Hacktoberfest 2026: die Issues, die Maintainer für den Oktober markiert haben – offen und einsteigerfreundlich. Hacktoberfest-Issues durchsuchen

[Bug] Subgraph startup rehashes the WASM module once per data source (minutes for subgraphs with millions of dynamic data sources)

Offen
  • Stimmt etwas nicht?
#6,731 1 Kommentar 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

@madumas arbeitet bereits daran.

Seit 09.10.2026.

  • #6732 von @madumas — offen

Bewertung

Schwierigkeit
3/5
Geschätzter Aufwand
Ein halber Tag
Anfängerfreundlichkeit
22/100
Issue-Typ
Bug
Klarheit
Klar beschrieben
Aktivitätsstatus
Veraltet
Tech-Stack
rust, wasm
Bereich
backend, performance

Rechercherichtung

Start in SubgraphInstance::new_host in core/src/subgraph/context/instance/mod.rs, where keccak256 runs over the full module bytes before the module_cache lookup. Check whether the bytes can be identified once per shared Arc or per template instead of per data source. Pull request #6732 is already open against this issue, so review or build on it rather than starting a parallel fix. Done means each distinct module is hashed once per runner start.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

Bug report

When a subgraph runner starts, SubgraphInstance::new_host
(core/src/subgraph/context/instance/mod.rs) computes keccak256 over the
full WASM module bytes for every data source, and only then looks the hash
up in module_cache:

let module_hash = alloy::primitives::keccak256(module_bytes.as_ref()).0;
if let Some(sender) = self.module_cache.get(&module_hash) {
    sender.clone()
} else { /* spawn_mapping … */ }

module_cache avoids recompiling the module, but not rehashing it. All data
sources created from the same template share one Arc of module bytes, so the
same bytes are hashed again for each of them. Startup time is therefore
O(number of data sources × module size), where it could be
O(number of data sources + number of distinct modules × module size).

This is paid on every runner start: node restart, subgraph restart after an
error, unassign/reassign, etc. Hashing the module for every data source dates
back to v0.19.0 (in another form since v0.17), and is still on master
(core/src/subgraph/context/instance/mod.rs:107 at 6838f4e3c).

Expected: each distinct module is hashed once per runner start.
Actual: each module is hashed once per data source.

Impact

On the Uniswap v3 subgraph on Base (deployment QmTEVcvbvWdW1LhNCdGpUFTxjxGmA34aRUksvX8TPjrQkZ),
with 1,770,112 data sources (all but one created from the pool template, whose
module is 66,103 bytes), startup takes about 4 minutes, nearly all of it in this
loop: graph-node logs nothing between Data source count at start and
forcing subgraph to use static filters (~236 s in the log below).

Relevant log output
2026-10-09T19:27:41.705 INFO Resolve subgraph files using IPFS, n_templates: 1, n_data_sources: 1, runner_index: 0, subgraph_id: QmTEVcvbvWdW1LhNCdGpUFTxjxGmA34aRUksvX8TPjrQkZ
2026-10-09T19:27:45.167 INFO Data source count at start: 1770112, runner_index: 0, subgraph_id: QmTEVcvbvWdW1LhNCdGpUFTxjxGmA34aRUksvX8TPjrQkZ
2026-10-09T19:31:41.628 INFO forcing subgraph to use static filters., runner_index: 0, subgraph_id: QmTEVcvbvWdW1LhNCdGpUFTxjxGmA34aRUksvX8TPjrQkZ
2026-10-09T19:31:41.688 INFO Start processing block, triggers: 66, runner_index: 0, subgraph_id: QmTEVcvbvWdW1LhNCdGpUFTxjxGmA34aRUksvX8TPjrQkZ
IPFS hash

QmTEVcvbvWdW1LhNCdGpUFTxjxGmA34aRUksvX8TPjrQkZ

Subgraph name or link to explorer

Uniswap v3 (Base)

Some information to help us out
  • Tick this box if this bug is caused by a regression found in the latest release.
  • Tick this box if this bug is specific to the hosted service.
  • I have searched the issue tracker to make sure this issue is not a duplicate.
OS information

Linux

Vorherrschende Sprache
Rust
Sterne
3.2k
Forks
1.1k
Keine gemergten PRs in 30 T.

Entwicklungsumgebung

In Codespaces öffnen

Startet den Dev-Container des Projekts im Browser, mit Ihrem eigenen GitHub-Konto.

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus graphprotocol/graph-node

Alle Issues in graphprotocol/graph-node

Ähnliche Issues

Weitere Issues zu Rust

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.