Spec capture: skip prompts that are already in the transcript, and capture specs written during the session

Offen
  • Stimmt etwas nicht?
#3,515 0 Kommentare 0 Reaktionen 1 zugewiesene Person Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

@migmartri arbeitet bereits daran.

Seit 05.10.2026.

Bewertung

Dieses Issue wurde noch nicht bewertet.

Beschreibung

component/CLI Feature

Follow-up on AI session spec capture (#3495, #3500). The chainloop trace hooks ask the coding agent to copy each spec source (ticket, document, image, prompt, approved plan) into .chainloop/specs/<session-id>/. The pre-push hook then records these files as evidence of the session. A review found the gaps below.

1. Decide if the prompt is stored as a spec

Today the session records the user's prompt as a spec entry with kind: text. The prompt is already part of the transcript, so it is not external context. This gives a duplicate record and makes the spec list look strange. For example, a spec material can contain only the first user message of the transcript, so it adds no information.

To do:

  • Decide if a prompt that is already in the transcript must be stored as spec evidence.
  • If no: do not capture it. Keep kind: text for a written plan that the session approved, or for text that is not in the transcript.
  • If yes: write down why, and show the difference in the UI between "prompt" and "external spec".

2. Capture specs that are written during the session

Capture depends on the agent copying each source into the spec folder at the start of the session. A spec that the agent creates or changes after that point is not captured. Example: the agent writes a design note in a local notes folder outside the repository, edits it many times, and subagents use it as their plan. The note is not in the evidence. The instruction says "if the task changes, add another file", but the agent does not do it.

To do:

  • Capture a spec document that is created or changed during the session, not only the sources that exist at the start.
  • Possible approaches: a reminder hook after the agent writes a planning document, detection of Write/Edit calls to known spec locations (for example a configured notes path or docs/specs/), or a check at push time that compares the files the agent read as a plan with the captured specs.
  • Support files outside the repository (local notes), because the URI is a local path.

How capture works today

Code: app/cli/pkg/action/trace_spec.go, trace_spec_materials.go, trace_hook_handler.go, trace_agent_hook.go, app/cli/internal/trace/spec/spec.go.

  1. Session start: the session-start hook creates .chainloop/specs/ and injects an instruction. The hook shows the instruction only while the session folder is empty. After the first file, a resumed session gets no reminder.
  2. During the session: no hook detects specs. The Write/Edit/Bash hooks record line attribution only.
  3. On git push: the pre-push hook reads all files in the folder of each attested session. It sorts them oldest first and keeps a maximum of 10. It redacts secrets from text files and stores images unchanged. Each file becomes one EVIDENCE material. The session material keeps only {kind, uri, digest, captured_at} for each file.
  4. Session end: the session-end hook deletes the spec folder of the session and its cached redactions.

Other gaps

  • A re-push does not record new specs. When all AI-assisted commits are already attested, the push stops ("all AI-assisted commits already attested, skipping push"). A spec added after a push is recorded only with the next AI-assisted commit.
  • Specs are deleted at session end. A push after the session ends attests the session without its spec.
  • The cap of 10 entries keeps the oldest files. In a long session, a spec added late is the first one dropped.
  • The prompt is captured on purpose. The instruction says "a written prompt … is kind text". The decision in part 1 changes the text of specCaptureInstruction, not the capture code.
  • A code comment is out of date. The package comment in spec.go says that pre-push deletes the files. The code deletes them at session end.

Acceptance criteria

  • There is a documented decision about prompt capture, and the implementation follows it.
  • A spec document that the agent writes during the session (also outside the repository) is captured as a document spec with its final content.
  • A spec added after the first push is recorded in the evidence, also when no new AI-assisted commit exists, or there is a documented decision not to do this.
  • A push after session end does not lose the spec, or there is a documented decision for this behavior.

🤖 Posted by Maximus bot (Claude Code) on behalf of @migmartri

Vorherrschende Sprache
Go
Sterne
582
Forks
61
Ø Merge
12 Std. 47 Min.
Gemergte PRs (30 T.)
137

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus chainloop-dev/chainloop

Alle Issues in chainloop-dev/chainloop

Ähnliche Issues

Weitere Issues zu Go

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.