Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

pi-coding-agent: Pi 1.4 system messages dropped, so LLM spans have no system prompt or tools

Open
  • Is something wrong?
#2,662 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@pcbeingused333 is already working on this.

Since Oct 25, 2026.

  • #2663 by @pcbeingused333 — open

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
45/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
node.js, typescript
Domain
observability

Research direction

Start in js/src/instrumentation/plugins/pi-coding-agent-plugin.ts: normalizePiMessage currently returns [] for role:"system" messages and normalizePiContextInput only reads context.systemPrompt. Read pi-agent-core's StreamFn/SystemMessage docs (content, sections, toolsAdded, toolsRemoved) and reproduce with Pi 1.4 by checking that an openai.responses.create span under AgentSession.prompt gains system input and metadata.tools. Done: system messages are normalized into input, tools are folded from toolsAdded/toolsRemoved with a fallback to context.tools, and executors are wrapped from agent.state.tools.

Written by the indexing model from the issue text.

Description

Summary

With @earendil-works/pi-coding-agent / pi-agent-core / pi-ai 1.4.0, the Pi Coding Agent integration (js/src/instrumentation/plugins/pi-coding-agent-plugin.ts, braintrust 3.39.0 and main at 50280e8b1) records LLM spans with no system prompt and no tool list.

Cause

Pi 1.4's agent loop calls the stream function with a normalized TranscriptContext. Its system prompt and tool declarations travel only in role: "system" messages (SystemMessage: content, sections, toolsAdded, toolsRemoved). pi-agent-core's StreamFn doc says: "the system prompt and tool declarations are carried by the transcript's system messages, never by context.systemPrompt or context.tools."

The plugin reads only those absent fields:

  • normalizePiContextInput adds a system message only from context.systemPrompt;
  • normalizePiMessage returns [] for a role: "system" message, so the transcript's system messages are dropped;
  • extractToolMetadata(context.tools) (LLM span metadata.tools) and wrapPiToolExecutors(context.tools) see undefined.

Observed

In our dev project, every openai.responses.create LLM span under AgentSession.prompt has an input of only user/assistant/tool messages and no metadata.tools; the prompt span has pi_coding_agent.active_tools (names only). There is no plugin option to recover this: spanCustomizers.onSpanExport runs after the data is gone.

Suggested fix

  • Normalize each SystemMessage into a { role: "system", content } input message, rendering content followed by its sections (later messages replace sections by name; null removes one), or at least keep the message's content and sections.
  • Fold toolsAdded / toolsRemoved across the transcript's system messages into the declared tool set and report it as metadata.tools, falling back to context.tools for callers that still pass it.
  • Wrap tool executors from the agent's tools (agent.state.tools) rather than context.tools.

We work around it by adding the agent state's systemPrompt and tools to the AgentSession.prompt span through a span customizer.

Dominant language
TypeScript
Stars
29
Forks
21
Avg merge
1d 13h
Merged PRs (30d)
67

Getting set up

  • Ships a Dockerfile or Docker Compose file
  • No pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from braintrustdata/braintrust-sdk-javascript

All issues in braintrustdata/braintrust-sdk-javascript

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.