Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

MLXLanguageModel tool calling: flaky repeat check, missing assistant tool-call turn, and tool-round limits

Abierto
  • ¿Algo no está bien?
#328 2 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
18/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
macos, swift
Área
api, backend

Línea de trabajo

Start in MLXLanguageModel.swift at the repeated-call check (around lines 1230 and 1496), where the key-order-dependent string is built, and compare it with ToolRoundLimit (#262). The issue also covers the missing assistant tool-call turn in pendingChat and the decodingFailure behaviour at the tool-round limit, which needs maintainer design decisions. Done would need the maintainers' answers to the three questions before any patch.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Version: 1.0.0 (80d50b0), mlx-swift-lm 3.32.3, MLX 4-bit models on an M3 Pro (macOS 27).

We measured how MLXLanguageModel handles multi-round tool calling with small local models, and found three things.
We'd like your view before going any further. Details are in the collapsed sections.

1. The repeated-call check depends on dictionary key order

MLXLanguageModel stops a response when a tool round repeats the previous round's calls. It compares strings built
with "\($0.function.name):\($0.function.arguments)", where arguments is a [String: MLXLMCommon.JSONValue]
(MLXLanguageModel.swift:1230-1233, :1496-1499).

A native Swift Dictionary seeds its hash table with its storage address (hashSeed(for:scale:) in the standard
library's HashTable.swift), so two identical calls can print their keys in different orders. As a result, the check
is reliable for tools with zero or one argument and a coin flip for tools with more.

Evidence

Controlled run.

  • Greedy sampling fixes the model's output, so between runs only Swift's hashing differs.
  • The prompts MLX builds are byte-identical with and without SWIFT_DETERMINISTIC_HASHING=1 (swift-jinja sorts keys).

Identical consecutive rounds, caught / missed:

model, tool (arguments) default SWIFT_DETERMINISTIC_HASHING=1
Qwen3-4B, check_status(job_id, region) (2) 4 / 16 20 / 0
Qwen3-1.7B, a search tool (pattern, file, context) (3) 19 / 12 20 / 0
Qwen3-4B, check_status(job_id) (1) 10 / 0 10 / 0

Comparing the arguments with sorted keys instead removes every miss (20 / 0 and 20 / 0), which confirms the cause.

What the check compared. We logged both strings just before the comparison. Here is one missed repeat
(Gemma 4 e2b):

previous: calculate:["a": MLXLMCommon.JSONValue.string("12"), "b": MLXLMCommon.JSONValue.string("7"), "operation": MLXLMCommon.JSONValue.string("multiply")]
current:  calculate:["b": MLXLMCommon.JSONValue.string("7"), "a": MLXLMCommon.JSONValue.string("12"), "operation": MLXLMCommon.JSONValue.string("multiply")]

Across 14 models and 603 identical consecutive rounds, the check missed 286 times. Every miss was the same text in a
different order.

Without MLX:

import Foundation
let text = Data(#"{"pattern":"report","file":"notes.md","context":"0"}"#.utf8)
let rounds = try (0..<100).map { _ in try JSONDecoder().decode([String: String].self, from: text) }
print(Set(rounds.map { "\($0)" }).count)  // 6 (20 of 20 runs); 1 with SWIFT_DETERMINISTIC_HASHING=1 or a single key

For comparison, ToolRoundLimit (#262) compares calls structurally for the remote providers, and
LlamaLanguageModel signs calls with .sortedKeys JSON.

2. The assistant turn that made a tool call is never sent back to the model

Between rounds, the chat is rebuilt from the transcript plus pendingChat, but the assistant turn carrying the calls
is never added:

  • pendingChat gets .assistant(text) only when the round produced text (:1217-1219, :1485-1487).
  • Results are appended as .tool(content), with no id and no name (:1267, :1525).
  • convertTranscriptToMLXChat drops .toolCalls entries (:1737-1739).

So the chat template receives [system, user, tool]: a result with no call before it. What happens next depends on
the template:

  • Gemma 4 (e2b, e4b, their QAT builds; the 26B-A4B QAT template behaves the same) drops the result.
    • The round-2 prompt is byte-identical to round 1, so the model repeats the same call until the repeat check or the
      8-round cap stops it.
    • Across 280 round transitions, the result never appeared in the prompt.
  • gpt-oss-20b's template raises Message has tool role, but there was no previous assistant message with a tool call!. We checked this at the template level only.
  • Ministral 3's template raises After the optional system message, conversation roles must alternate user and assistant roles except for tool calls and results. once two rounds in a response produced text (2 of 6 runs).
  • Qwen3, Qwen3.5, Llama 3.2 and LFM2 render the result, but without the call that produced it.
Evidence: the same build with only the assistant turn added to pendingChat

Default sampling, 3–5 runs per mode, respond and streamResponse:

model scenario 1.0.0 + assistant turn
Gemma 4 e2b calculator ("12 * 7 + 3") 6/6 failed 6/6 answered 87
Gemma 4 e2b search tool that never matches 6/6 failed 6/6 answered that nothing was found
Gemma 4 e4b calculator 6/6 failed 6/6 answered 87
Gemma 4 E2B QAT calculator 6/6 failed 5/6 answered 87
Gemma 4 E4B QAT calculator 6/6 failed 4/6 answered 87 (1 empty answer)
Ministral-3-3B calculator 2/6 raised the alternation error 0/6
Qwen3-1.7B search tool 10/10 re-sent their previous call verbatim 0/10
Qwen3-8B search tool 6/6 re-sent their previous call verbatim 0/6
Qwen3-4B search tool 8/10 ended by announcing another search ("I will attempt to find … by searching the knowledgebase files") 0/10

Gemma's prompt, decoded, for "What is 12 * 7 + 3?".

  • On 1.0.0: rounds 2–9 are identical to round 1 (144 tokens).
  • With the turn: round 2 is round 1 plus
    <|tool_call>call:calculate{a:<|"|>12<|"|>,b:<|"|>7<|"|>,operation:<|"|>multiply<|"|>}<tool_call|><|tool_response>response:{value:<|"|>"84"<|"|>}<tool_response|>,
    and the model's next call is add(84, 3).

Llama 3.x. Its templates reject an assistant message carrying more than one call, with This model only supports single tool-calls at once!. With all of a round's calls in one message, Llama-3.2-3B hit that on every round where it
called tools in parallel.

3. What happens when a tool-round limit is hit

Since #262, every provider stops a response after 8 tool rounds, or when a round repeats the previous one, by throwing
GenerationError.decodingFailure. What we observed:

  • The tool work is discarded. After a limit error the transcript holds only the prompt (88 of 88 errors on 1.0.0).
    .preserveTranscript keeps the tool entries only for streamResponse.
  • The delegate never sees the round that hits the limit, in 373 of 373 cases. So it can't .stop or substitute
    output there.
  • The error is hard to identify. Foundation Models documents decodingFailure as a deserialization failure, so
    callers can only recognize this case by matching on debugDescription.
  • The repeat rule mostly stops legitimate repeats.
    • We ran 336 responses with the assistant turn added (section 2), across 13 models. The rule stopped 118 of them.
    • 113 were legitimate: 74 re-checks of a job whose first check returned pending ("check again until it is
      done"), and 39 second die rolls when the prompt asked for two.
    • The loops that did run long (18 runs) were 5–8 different calls, which only the cap stops.
    • On plain 1.0.0, a one-argument polling tool is rejected on its first re-check every time (12/12).
  • Foundation Models allows these repeats. We ran the same tools and prompts through SystemLanguageModel, 30
    runs: no errors.
    • It re-checked the pending job with identical arguments and answered (12/12).
    • It rolled the die twice with identical calls (6/6).
    • Against a search tool that never matches, it stopped after 3–5 searches and said nothing was found (6/6).

This touches the question on #147 ("prevents llms to call exact same tool twice … Was it ever discussed?") and #280.

Questions

  1. Is the key-order dependence in the MLX repeat check (section 1) unintended? Should it compare calls the way
    ToolRoundLimit does?
  2. Is leaving the assistant tool-call turn (and tool_call_id/name) out of the MLX chat history (section 2)
    deliberate, for example for template compatibility? Or should the model see its own calls, as it does with
    llama.cpp and the remote providers?
  3. When a tool-round limit is hit, is throwing decodingFailure and losing the round's work the behavior you want?
    And should identical consecutive calls count as a loop at all, given that Foundation Models allows them?

Happy to share the harness, which logs every round through ToolExecutionDelegate, and the raw data.

Lenguaje dominante
Swift
Estrellas
948
Forks
99
Merge medio
19 h 50 min
PR fusionados (30 d)
51

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de huggingface/AnyLanguageModel

Todos los issues de huggingface/AnyLanguageModel

Issues similares

Más issues de Swift

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.