Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

[Feature] Add an offline tool to backfill historical SectionBloom data

Aperta
  • Qualcosa non va?
#6,958 5 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Una pull request collegata è già stata integrata.

  • #6973 di @317787106 — integrata

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
38/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
java
Ambito
cli, database

Direzione di ricerca

Start with PR #6390 and the existing SectionBloomStore logic, then trace how Toolkit commands access transactionRetStore, properties, and the section-bloom database. Confirm the command's range, concurrency, validation, idempotency, progress, summary, and failure behavior against the issue's requirements, with tests covering the listed cases.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

type:feature

Summary

Add an offline Toolkit command to rebuild historical SectionBloom data in the section-bloom database from existing records in transactionRetStore. This allows node operators to restore historical log filtering without resyncing the node or replaying the blockchain.

Problem

Motivation

SectionBloom data allows eth_getLogs to quickly identify blocks that may contain logs matching a contract address or topic using Bloom filters.

Current State

In v4.8.0 and earlier, SectionBloom data was not generated for blocks processed while node.jsonrpc.httpFullNodeEnable was disabled. In v4.8.1 and later, this setting no longer controls SectionBloom generation, and the data is always written for newly processed blocks.

Enabling the option later, or upgrading to a version that always writes SectionBloom data, does not repair missing historical data. As a result, eth_getLogs queries that filter historical ranges by address or topics may fail to find all matching logs.

Limitations or Risks

There is currently no offline tool for rebuilding this data from the local database. Operators may therefore need to resync the node or replay historical blocks, even when the required transaction results are already available in transactionRetStore.

Proposed Solution

Proposed Design

Add the following command to Toolkit:

java -jar Toolkit.jar db backfill-bloom

The command should read historical transaction results from transactionRetStore, calculate block Bloom filters using the same logic as SectionBloomStore, and create or update the corresponding records in the section-bloom database.

Expected usage:

java -jar Toolkit.jar db backfill-bloom \
  [-d <databaseDirectory>] \
  [-s <startBlock>] \
  [-e <endBlock>] \
  [-c <maxConcurrency>]
Option Description Default
-d Database directory. output-directory/database
-s, --start-block First block to process, inclusive. Earliest non-zero block available in transactionRetStore.
-e, --end-block Last block to process, inclusive. Latest solidified block recorded in the properties database.
-c, --max-concurrency Maximum processing concurrency. 8

Values outside the available block range should be adjusted to the actual database boundaries.

The command should process data by Section, with each Section containing 2,048 blocks. Actual concurrency should not exceed the number of Sections being processed.

The operation should be idempotent so that the same block range can be safely processed again after an interruption. Existing SectionBloom bits should be preserved when records are updated.

Progress and Summary

For long-running backfills, the command should display terminal progress and periodically write progress information to toolkit.log.

The final summary should include:

  • Number of scanned blocks.
  • Number of successfully processed blocks.
  • Number of blocks containing logs.
  • Number of errors.
  • Number of Bloom writes.
  • Elapsed time.
  • Processing rate.
  • Concurrency used.
Validation and Testing

The implementation should validate the database directory, required databases, block range, and concurrency value.

Unit tests should cover:

  • Parameter validation.
  • Automatic range detection and adjustment.
  • Missing or invalid databases.
  • Processing failures.
  • Progress and summary output.
  • Help output.
  • Safe reprocessing of the same block range.
Key Changes
  • Toolkit: Add the db backfill-bloom offline maintenance command.
  • Database access: Read historical transaction results from transactionRetStore and the latest solidified block from properties; create or update derived Bloom records in section-bloom.
  • Configuration and APIs: Existing node configuration and JSON-RPC APIs remain unchanged.

Operational Requirements and Risks

The FullNode and any other process accessing the database must be stopped before running the command because the database requires exclusive access. Multiple backfill processes must not operate on the same database concurrently.

The target blocks must have been processed while storage.transHistory.switch was enabled. Otherwise, transactionRetStore will not contain the historical transaction results required to rebuild SectionBloom data.

The backfill may generate significant disk I/O and CPU load when processing large block ranges. Operators should adjust concurrency according to their storage hardware and monitor disk latency and CPU usage during execution.

Impact

After the missing SectionBloom data is rebuilt, eth_getLogs can correctly filter the affected historical blocks by contract address and topics.

This feature introduces only an offline maintenance command. It does not add a new network interface or change the normal block-processing flow. The tool can only be run when the FullNode is stopped.

Compatibility

  • Breaking change: No.
  • Default behavior change: No.
  • Migration required: No.

Nodes without missing historical SectionBloom data do not need to run this command. Existing configurations and JSON-RPC APIs remain unchanged.

References

Use PR #6390: feat(toolkit): implement backfill SectionBloom function @h3110w0r1d-y as a reference. The implementation may be reworked or redesigned from there.

Additional Notes

  • Do you have ideas regarding implementation? Yes.
  • Are you willing to implement this feature? Yes.
Lingua principale
Java
Stelle
4.2k
Fork
1.8k
Merge medio
3g 21h
PR unite (30g)
13

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di tronprotocol/java-tron

Tutte le issue di tronprotocol/java-tron

Issue simili

Altre issue su Java

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.