Observation folding makes small models write placeholders like [URL obtido em obs_2]

Open
  • Is something wrong?
#27 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Active
Tech stack
python
Domain
ai, tooling

Research direction

Start by reading the observation-folding and verify-gate code, then trace how folded observations are exposed when mem is unavailable. Reproduce the collect-then-write failure with the reported pipeline and model, and check the three related issues (#24, #25, and #27). Done means choosing and implementing a behavior that lets the model recover needed values, then confirming the verify gate accepts the resulting output.

Written by the indexing model from the issue text.

Description

known-issue

Problem

With Rule 5 (K=2 observation folding), a small model that needs to copy data from earlier tool outputs into a later write sees only tombstones for older observations. In a pipeline run (searcher skill, ollama/gemma4:e4b), after several yt-channel/yt-search calls the model wrote:

arquitetura | [Título do vídeo sobre arquitetura geral] | [URL obtido em obs_2] | [Visualizações de obs_2]

It referenced obs_N as if the content were still visible, instead of running mem recall obs_N (it never did, across 3 gate failures) or re-running the search. The verify gate (youtube.com/watch count + oEmbed check) correctly rejected it, but the stage ended after 3 consecutive gate failures with no result.

Why it matters

Folding is correct for token economy, but the tombstone text use mem recall obs_3 to expand is not enough for a 9B model that is mid-way through a collect-then-write task, and mem was not even enabled for that skill, so recall was impossible.

Ideas (decision left to the implementer)

  1. When a verify gate fails and the written file contains obs_\d+ / bracketed placeholders, include a gate hint to re-run the tool and copy the value immediately.
  2. Make the fold tombstone aware of whether mem is enabled; if not, either keep the last N observations unfolded for that session or state that recall is unavailable.
  3. Keep observations unfolded while the model has not yet called a write/done command that consumes them (task-phase aware folding).
  4. Document the pattern: for collect-then-write tasks, use a tool that persists results itself (e.g. an append-to-file tool) instead of relying on the model to carry values across turns.

Related: #24, #25.

Dominant language
Python
Stars
3
Forks
1
Avg merge
52m
Merged PRs (30d)
166

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from igormcsouza/tether

All issues in igormcsouza/tether

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.