Python remote agent example reports tool calls under `tool_trace`, which the decoder ignores
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 86/100
Research direction
Start with the two tool_trace occurrences in examples/remote_agent_python/server.py and compare them with the decoder in Domain.scala and the README wire format. Verify the response field is decoded as tool interactions; the issue is done when both example occurrences use tool_interactions.
Written by the indexing model from the issue text.
Description
examples/remote_agent_python/server.py returns tool calls under the key tool_trace (line 92, and in the comment on line 76):
"action": {
"type": "send",
"message": {...},
"tool_trace": [],
}
The response decoder reads tool_interactions as an optional field (Domain.scala:139):
toolInteractions <- c.downField("tool_interactions").as[Option[List[ToolInteraction]]]
The README's wire format also uses tool_interactions.
Because the field is optional and circe ignores unknown keys, a server built from this example decodes without error but reports no tool calls. A criterion with require_tool_call: true would then fail, and nothing would point to the mismatched key. The example's list is empty, so the example itself works. The problem shows up once someone adds real tools to it.
Suggested fix: rename the key to tool_interactions in both places in server.py. Optionally, have the decoder warn on an unknown tool_trace key, or reject it.
Found by reading the code at 6368051. I haven't run it end to end.
- Dominant language
- Scala
- Stars
- 0
- Forks
- 6
- Avg merge
- 16d 12h
- Merged PRs (30d)
- 1
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from salesforce/evalon
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
salesforce/evalon#10 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 58/100
salesforce/evalon#15 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 72/100
salesforce/evalon#13 ·
-
Failure classification by name string misclassifies harness failures on name collisionPossibly taken @hippi345 claimed this 14 days ago. Openbug
Difficulty 4/5 3-5 days Newbie friendliness 50/100
salesforce/evalon#11 ·
-
EvaluatedAgent crashes on directConversations.head when an observer-only agent receives eventsPossibly taken @hippi345 claimed this 14 days ago. Openbug
Difficulty 4/5 3-5 days Newbie friendliness 48/100
salesforce/evalon#9 ·
All issues in salesforce/evalon
Similar issues
-
GeoSeries scale and skew tests fail with pytest 9.2.0Possibly taken @jiayuasu claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
[BUG] Python unit CI fails on Ubuntu 26.04 runnerPossibly taken @parthosa claimed this today. Open
Difficulty 1/5 1-3 hours Newbie friendliness 83/100
NVIDIA/cudf-spark-tools#2207 ·
Maintainers usually reply within 2 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 73/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
canton-network/splice#8087 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day