Consider CAR-first reading

Open
#7,672 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Refactor
Clarity
Mostly clear
Activity status
Active
Tech stack
rust
Domain
databases

Research direction

Start with src/db/car/many.rs at lines 226-239, where ParityDb is checked before the CAR files. Measure both lookup orders on snapshot export and RPC-serving workloads, including lock contention and relevant performance metrics. Done means deciding which approach should be the default, with any option to reverse it supported if the measurements warrant that.

Written by the indexing model from the issue text.

Description

Type: Task

Summary

Right now we check ParityDb first, only then the CAR files.

https://github.com/ChainSafe/forest/blob/84b3faac005a6886eda0df9379c68f2aa80fffc5/src/db/car/many.rs#L226-L239

The issue is that this internally harasses an internal ParityDb RwLocks - 4 global ones and then some processing. Flipping it (and potentially replacing our RwLock with ArcSwap) could make the entire thing operate more smoothly with less lock contention. In theory this could make certain workloads slower (if operating purely on hot data); it's worth measuring on expected use-cases, e.g., snapshot export and RPC serving. Some additional metrics would be useful as well.

We could also have an option to flip back the behavior.

Completion Criteria

  • Decide whether ParityDb first or CAR-first approach is a better default.

Additional Links & Resources

Dominant language
Rust
Stars
700
Forks
201
Avg merge
1d 17h
Merged PRs (30d)
65

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ChainSafe/forest

All issues in ChainSafe/forest

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.