Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Long single tool calls end in loops, and the work before the loop is lost

Open
  • Is something wrong?
#261 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
python
Domain
ai-infra-agents

Research direction

Start with the recorded model streams from smoke runs 8b95baa, 4403ae8, fc8321d a/b/c, and the 84 ccc71fc sessions; compare loop start against output length, tool and field, escaping density, and recent tool choices. The issue does not name files or tests, so first locate how these streams are recorded and inspect the cited examples, especially smallrung_toml. Done means identifying a cause with evidence—or showing it cannot yet be separated—and recording a fix or no-fix decision with a measurement plan.

Written by the indexing model from the issue text.

Description

area/proxy performance reliability status/ready

What needs doing

Find out why long single tool calls end in loops. Name the cause with evidence before choosing a fix.

Why

The loop detector's timing is not the main cost of a loop (#215). Across 1,180 recorded model streams (smoke runs 8b95baa, 4403ae8, and fc8321d a/b/c), the detector cut 25 streams, all real loops. In the 21 of them with a clear loop start:

  • 916 s of generation came before each loop began. The cut discards all of it.
  • 291 s went to the loops themselves.

The clearest case is smallrung_toml (0 or 1 of 6 in every recent run). To add one branch of about 10 lines to _syntax_check_impl, the model re-emits the whole function, 257 lines and 9–13 KB, inside one structural_edit call. The loop then starts at 8.8–10 KB:

  • 8b95baa r1, turn 4: 111 s of content, then 0000….
  • 8b95baa r1, turn 7: the function was complete at 9.7 KB (124 s), including the new toml branch. Then the model started the whole function again from its docstring.

In the ccc71fc runs (84 sessions), 20 calls were cut. Nine were an edit_file old_str (tracked in #215); the rest were long structural_edit and insert_after bodies.

Hypotheses (none established)

  1. The size of what is re-emitted. A whole function is replaced to add one branch. Tool choice or guidance may steer the model there. For example, a retry message recommends structural_edit (#217).
  2. Output length. Degeneration may grow with output length under the sampling settings in use.
  3. Escaping load. Quote-heavy Python inside a JSON string (#217 counted 9 cuts of this kind).
  4. Channel. The fenced channel carries write_file bodies outside JSON, but structural_edit and insert_after bodies always go through JSON.

Done when

  • The recorded streams are measured on each hypothesis: loop start against output length, tool and field, escaping density, and the model's recent tool choices.
  • The cause is named with evidence, or the evidence says it cannot be separated yet.
  • A fix, or a decision not to fix, is recorded with its measurement plan.
Dominant language
Python
Stars
2.1k
Forks
197
Avg merge
14h 39m
Merged PRs (30d)
188

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from inferstep/ATLAS

All issues in inferstep/ATLAS

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.