error with /attention_head_record endpoint
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 35/100
Research direction
Start by reproducing the request against /attention_head_record and read neuron_explainer/activation_server/read_routes.py, especially attention_head_record, convert_activation_records_to_token_and_attention_activations_lists, and normalize_attention_token_scalars. Compare this path with /neuron_record and inspect the failing scalar summary functions around lines 380-455. Done means the supplied request is handled successfully or returns a clear explanation of the dataset or input problem.
Written by the indexing model from the issue text.
Description
Hi Team,
I hit error when I tried following request body to /attention_head_record
{
"dst": "logits",
"layerIndex": 9,
"activationIndex": 8,
"datasets": [
"https://openaipublic.blob.core.windows.net/neuron-explainer/gpt2_small_data/collated-activations/"
]
}
Error as below:
File "/home/ruibn/debug/transformer-debugger/neuron_explainer/activation_server/read_routes.py", line 265, in attention_head_record
) = convert_activation_records_to_token_and_attention_activations_lists(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ruibn/debug/transformer-debugger/neuron_explainer/activation_server/read_routes.py", line 191, in convert_activation_records_to_token_and_attention_activations_lists
return normalize_attention_token_scalars(zipped_tokens_and_raw_attention_activations)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ruibn/debug/transformer-debugger/neuron_explainer/activation_server/read_routes.py", line 455, in normalize_attention_token_scalars
) = compute_scalar_summary_and_scale(compute_max_scalar_out, list_of_sequence_lists)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ruibn/debug/transformer-debugger/neuron_explainer/activation_server/read_routes.py", line 427, in compute_scalar_summary_and_scale
scalar_indexed_by_token_sequence_list: list[list[list[float]]] = [
^
File "/home/ruibn/debug/transformer-debugger/neuron_explainer/activation_server/read_routes.py", line 428, in
[compute_scalar_summary_function(sequence) for sequence in sequence_list]
File "/home/ruibn/debug/transformer-debugger/neuron_explainer/activation_server/read_routes.py", line 428, in
[compute_scalar_summary_function(sequence) for sequence in sequence_list]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ruibn/debug/transformer-debugger/neuron_explainer/activation_server/read_routes.py", line 421, in compute_max_scalar_out
return compute_summary_of_scalar_out(attn_token_sequence_list, _max_non_none)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/ruibn/debug/transformer-debugger/neuron_explainer/activation_server/read_routes.py", line 392, in compute_summary_of_scalar_out
scalar_out = [
^
File "/home/ruibn/debug/transformer-debugger/neuron_explainer/activation_server/read_routes.py", line 393, in
operation(
File "/home/ruibn/debug/transformer-debugger/neuron_explainer/activation_server/read_routes.py", line 380, in _max_non_none
return max([a for a in scalars if a is not None])
Any idea? Same request was working fine on /neuron_record.
Will it due to wrong dataset I used? @WuTheFWasThat
- Dominant language
- Python
- Stars
- 4.1k
- Forks
- 242
- PR merge metrics
- No merged PRs in 30d
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from openai/transformer-debugger
-
Difficulty 3/5 1-2 days Newbie friendliness 25/100
-
Difficulty 3/5 1-2 days Newbie friendliness 25/100
openai/transformer-debugger#21 · 5 comments · 1 reaction ·
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
-
Difficulty 4/5 3-5 days Newbie friendliness 20/100
openai/transformer-debugger#19 · 1 comment ·
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
openai/transformer-debugger#18 · 1 comment · 3 reactions ·
All issues in openai/transformer-debugger
Similar issues
-
bug good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
yunaremaia/proto-drift#22 ·
-
bug good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
yunaremaia/memwatch#96 ·
-
bug good first issue
Difficulty 1/5 Under an hour Newbie friendliness 91/100
yunaremaia/ci-gate-watch#34 ·
-
bug good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
yunaremaia/agent-capability-attestation#64 ·
Maintainers usually reply within 1 day
-
bug good first issue
Difficulty 1/5 Under an hour Newbie friendliness 86/100
yunaremaia/agent-call-graph#43 ·
Maintainers usually reply within 1 day