[Proposal] Linear Artificial Tomography (LAT) scans

Aperta
  • Qualcosa non va?
#1,957 3 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
30/100
Tipo di issue
Funzionalità
Chiarezza
Da chiarire
Stato di attività
Attiva
Stack tecnologico
python
Ambito
machine-learning

Direzione di ricerca

The proposal adds a LAT utility for HookedTransformer using run_with_cache and resid_pre/resid_post projections. No specific files or tests are named; review the TransformerLens package and HookedTransformer hooks to find integration points. Done means a merged PR implementing the sketched API with direction extraction, scans, baselines, and visualization.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

enhancement seen_by_maintainers TransformerBridge

Summary

Add a utility for running LAT scans on HookedTransformer models. A scan finds a concept direction at each layer and projects activations onto it, producing a layer-by-token readout of where the concept is represented. TransformerLens has the hooks for this, but no standard implementation.

Motivation

LAT is a common tool in representation engineering, and each project currently reimplements it. I used it to study emotion-associated directions in Qwen3-0.6B, where alignment rose in later layers. Interpreting that result required controls for norm growth and anisotropy that a shared implementation could provide by default. I already have the code written, would have to just be ported to the repository.

Proposed functionality

Direction extraction: Build per-layer concept vectors from contrastive or stimulus-based prompt sets, with PCA or difference-of-means options.
Scanning: Run run_with_cache over input text and project resid_pre / resid_post onto the vectors, returning a layer × token score tensor.
Baselines: Built-in random-direction and norm-matched controls, plus optional centering or whitening from neutral-prompt statistics.
Visualization: Layer-wise alignment curves and token-level heatmaps.
Interventions: Optional steering or ablation along extracted vectors for causal checks.

Example usage


python
scan = lat_scan(model, concept_prompts, neutral_prompts, layers="all")
scores = scan.project(text)
scan.plot(baseline="random")

The API is only a sketch, and I’m open to whatever fits the library’s conventions.

Example of the image generated:

Image
Lingua principale
Python
Stelle
3.9k
Fork
708
Merge medio
1g 10h
PR unite (30g)
82

Preparare l'ambiente

Apri in Codespaces

Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.

  • Nessun Dockerfile né file Docker Compose
  • Ha un modello di pull request
  • Nessuna guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di TransformerLensOrg/TransformerLens

Tutte le issue di TransformerLensOrg/TransformerLens

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.