Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

[Bug Report] _BLOCK_LIST_ATTRS hardcoded name list silently drops Raven's blocks from composition-score / head-label analysis

Aperta
  • Qualcosa non va?
#1,791 2 commenti 0 reazioni 1 assegnatario Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@LightWork666 ci sta già lavorando.

Dal 22/9/2026.

Valutazione

Questa issue non è ancora stata valutata.

Descrizione

bug complexity-moderate TransformerBridge

Describe the bug

_enumerate_blocks() (transformer_bridge.py) walks a hardcoded tuple, _BLOCK_LIST_ATTRS = ("blocks", "encoder_blocks", "decoder_blocks", "L_blocks", "H_blocks") (bridge_core.py), checking self._modules.get(name) for each. Raven/Huginn's three block lists are registered under prelude, core_block, and coda -- none of which are in that tuple -- so _enumerate_blocks() silently returns [] for a Raven bridge, with no error.

This breaks every method built on it, silently:

  • blocks_with(submodule) -> always []
  • composition_layer_indices() -> always []
  • attn_head_labels -> always [] (should report labels for every attention head across prelude/core_block/coda)

Inconsistently, some other methods built on the same broken enumeration fail loudly instead of silently, because they separately guard against an empty result: all_composition_scores() raises ValueError("No attention layers found"), and stack_params_for() (used by QK_for_attn_layers()/OV_for_attn_layers()) raises ValueError("No blocks have submodule ..."). Both messages are misleading -- Raven's blocks clearly do have attention -- but at least they don't silently hand back wrong data the way attn_head_labels does.

This is the same class of fragility as #1769 (stop_at_layer's reject-list), which jlarson4 is already having fixed by moving to an allowlist/_modules-based check -- except this one is on the enumeration side rather than the rejection side, and its failure mode is silent wrongness rather than a silent no-op.

Reproduction (no model download needed)

import torch.nn as nn
from transformer_lens.model_bridge.transformer_bridge import TransformerBridge

# Mirrors Raven's actual registered block-list shape (prelude/core_block/coda).
fake = nn.Module()
fake.add_module("prelude", nn.ModuleList([nn.Linear(2, 2), nn.Linear(2, 2)]))
fake.add_module("core_block", nn.ModuleList([nn.Linear(2, 2) for _ in range(4)]))
fake.add_module("coda", nn.ModuleList([nn.Linear(2, 2), nn.Linear(2, 2)]))

result = TransformerBridge._enumerate_blocks(fake)
print(result)  # -> [] -- silently drops all 8 blocks, no error

System Info

  • Installed from source, dev-4.x
  • Pure Python logic, no weights involved; verified without downloading Raven's ~14GB checkpoint

Expected behaviour & fix pointers

Same direction as the fix in progress for #1769: replace the hardcoded name tuple with structural discovery. Every block-list component set via component_setup.py's list-item path carries hook_out_is_single_residual_stream = True on its items (already used for exactly this purpose in all_composition_scores()'s prerequisite check) -- _enumerate_blocks() could instead walk self._modules.items() and treat any nn.ModuleList whose first element is a GeneralizedComponent with that marker set as a block list, picking up prelude/core_block/coda (and any future non-standard adapter) automatically, with no name list to maintain.

Acceptance:

  • _enumerate_blocks() (and everything built on it: blocks_with, composition_layer_indices, attn_head_labels, stack_params_for, all_composition_scores) correctly discovers Raven's prelude/core_block/coda blocks, or raises a clear error consistently -- never silently returns an empty/wrong result
  • Existing blocks/encoder_blocks/decoder_blocks/L_blocks/H_blocks behavior is unchanged
  • A fast regression test (matching the reproduction above, no checkpoint download) pins the fix
  • make unit-test and uv run mypy . pass
Checklist
  • I have checked that there is no similar issue in the repo (required)
Lingua principale
Python
Stelle
3.9k
Fork
708
Merge medio
1g 17h
PR unite (30g)
70

Preparare l'ambiente

Apri in Codespaces

Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.

  • Nessun Dockerfile né file Docker Compose
  • Ha un modello di pull request
  • Nessuna guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di TransformerLensOrg/TransformerLens

Tutte le issue di TransformerLensOrg/TransformerLens

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.