Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Run a round's tool calls at once

Aperta
  • Qualcosa non va?
#359 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
25/100
Tipo di issue
Funzionalità
Chiarezza
Da chiarire
Stato di attività
Attiva
Stack tecnologico
swift
Ambito
ai, backend

Direzione di ricerca

Start by reading the OpenAI provider's tool-call loop and the fork at gtokman/AnyLanguageModel@3bc39af. The issue leaves key design choices open: provider scope, whether concurrency is opt-in, cancellation behavior, and interaction with parallel_tool_calls. Done means agreeing on those behaviors and implementing concurrent calls while preserving result order and delegate callbacks.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

When a model calls several tools in one round, the providers run the calls one after another, so a round takes the sum of its calls' times. The calls can't depend on each other's results, because the model wrote all of them before seeing any output, so they could run at once and the round would take as long as its slowest call.

In our chat server (OpenAI Chat Completions through Vercel AI Gateway; the model often looks up several places or sites at once), this was most of the wait in multi-call rounds. We run a fork with the OpenAI provider's calls running concurrently: gtokman/AnyLanguageModel@3bc39af. It keeps what callers see the same:

  • results go back in the order the model gave the calls, both in the transcript and in the next request;
  • the delegate hears about each call as before (didGenerateToolCalls, toolCallDecision, and the per-call results);
  • the first error ends the round.

Two 300 ms calls in one round took 0.32 s instead of 0.62 s. In our server's benchmark, the median multi-call round went from 6.5 s to 1.3 s.

Before opening a PR, some questions for you:

  1. All providers, or opt-in? Our fork only changes the OpenAI provider. Upstream it probably belongs in every provider's tool loop, maybe through shared code. But some apps' tools may not be safe to run concurrently (shared state, rate limits, side effects that must happen in order). Should it be the default, a GenerationOptions setting, or something on the tool, such as a way to mark a tool as safe to run alongside others?
  2. Cancellation: when one call throws, should the others be cancelled, or allowed to finish before the error is reported? Our fork reports the first error; a tool with side effects might want its siblings to finish.
  3. OpenAI's parallel_tool_calls: it controls whether the model may request several calls in a round. This would be about running them. Should a setting for one affect the other?

Happy to do the PR in whichever shape you prefer.

Lingua principale
Swift
Stelle
959
Fork
99
Merge medio
1g 6h
PR unite (30g)
59

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di huggingface/AnyLanguageModel

Tutte le issue di huggingface/AnyLanguageModel

Issue simili

Altre issue su Swift

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.