Hacktoberfest 2026：维护者为十月标记出来的 issue，仍然开放、适合新手。 浏览 Hacktoberfest issue

Add messages to a response while it runs (steering between tool rounds)

未关闭
  • 哪里不对？
#360 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
5/5
预计耗时
一周以上
新手友好度
35/100
Issue 类型
功能
描述清晰度
基本清楚
活跃度
活跃
技术栈
swift
领域
api, backend

调研方向

Read the OpenAI provider's streaming and nonstreaming tool loops and the ToolExecutionDelegate definition; the issue cites a fork implementation at gtokman/AnyLanguageModel@8f2d9f8. Compare how each loop builds requests and transcript entries, then decide whether the library should support queued messages and what happens to messages still pending when a response ends. Done means a design is agreed on for the API shape, message types, and end-of-response semantics.

由索引模型根据 Issue 内容生成。

描述

A long response with tools can run for a minute or more: search, read some pages, compare, write. While it runs, the person often wants to adjust it ("skip Delta", "under $300", "actually make it Lisbon"). Today an app's only option is to cancel the response and start a new one, which throws away the lookups already done.

What we wanted was to send messages into the running response: after a round of tool calls, before the model's next request, add what the person has sent since, and the model reads it alongside the tool results.

Why it's easy to get wrong. The added messages have to go into the request and the transcript, in the same form and in the same place (after the round's tool outputs). If the in-flight request and a later replay of the transcript differ, a prompt cache that matches on exact prefixes (Anthropic's, for example) misses on every later turn. That's the same problem #325 fixed for tool calls. An app can't do this from outside: it doesn't control what goes into the next request mid-loop.

What we did. Our fork adds one method to ToolExecutionDelegate (gtokman/AnyLanguageModel@8f2d9f8):

/// Asked after each round of tool calls, before the model's next request.
/// The messages are added to the transcript as prompts, in order,
/// after the round's tool outputs. The default adds nothing.
func pendingMessages(in session: LanguageModelSession) async -> [String]

The OpenAI provider asks for messages after each round, in both the streaming and nonstreaming loops. It appends each one as a .prompt entry and as the matching user message in the request, so replay sends them exactly as they were sent. Our server keeps a per-conversation inbox. A small model also answers the person right away ("Got it, I'll leave Delta out."), and the main model is told it already said that, so it doesn't acknowledge twice.

The shape is a question for you. A delegate method fit our server, but there are other designs:

  • a method on the session to queue a message for the response in progress (session.enqueue("skip Delta")), which is closer to how an app thinks about it;
  • tying it to dynamic instructions or profiles (#344–#348), since those already re-resolve the request each round;
  • only Strings, or full Prompts (images, for example)?

And a semantics question: if the response ends with no further tool round, should messages still pending start a new turn, be returned to the caller, or be dropped? Our server interrupts the response if the model is writing its final text, and starts a new response from the messages (the lookups stay in the transcript; the unfinished text doesn't). Any that arrive as the reply closes go back to the app, which sends them as a normal message.

Does this belong in the library, and if so, which shape would you prefer? Happy to do the PR.

主要语言
Swift
星标
959
派生
99
平均合并
1 天 6 小时
30 天内合并 PR
59

环境准备

这个项目没有提供开发容器、Dockerfile 或贡献指南，环境需要你自己搭建：先看它的 README，通用步骤见我们的新手贡献指南。

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

huggingface/AnyLanguageModel 的其他 Issue

查看 huggingface/AnyLanguageModel 的全部 Issue

相似的 Issue

更多 Swift Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。