Hacktoberfest 2026: những issue maintainer đã đánh dấu cho tháng Mười, đang mở và phù hợp người mới. Xem issue Hacktoberfest

MLXLanguageModel tool calling: flaky repeat check, missing assistant tool-call turn, and tool-round limits

Đang mở
  • Có gì không ổn?
#328 2 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
4/5
Thời gian dự kiến
3-5 ngày
Mức phù hợp với người mới
18/100
Loại issue
Lỗi
Độ rõ ràng
Khá rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
macos, swift
Lĩnh vực
api, backend

Hướng nghiên cứu

Start in MLXLanguageModel.swift at the repeated-call check (around lines 1230 and 1496), where the key-order-dependent string is built, and compare it with ToolRoundLimit (#262). The issue also covers the missing assistant tool-call turn in pendingChat and the decodingFailure behaviour at the tool-round limit, which needs maintainer design decisions. Done would need the maintainers' answers to the three questions before any patch.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

Version: 1.0.0 (80d50b0), mlx-swift-lm 3.32.3, MLX 4-bit models on an M3 Pro (macOS 27).

We measured how MLXLanguageModel handles multi-round tool calling with small local models, and found three things.
We'd like your view before going any further. Details are in the collapsed sections.

1. The repeated-call check depends on dictionary key order

MLXLanguageModel stops a response when a tool round repeats the previous round's calls. It compares strings built
with "\($0.function.name):\($0.function.arguments)", where arguments is a [String: MLXLMCommon.JSONValue]
(MLXLanguageModel.swift:1230-1233, :1496-1499).

A native Swift Dictionary seeds its hash table with its storage address (hashSeed(for:scale:) in the standard
library's HashTable.swift), so two identical calls can print their keys in different orders. As a result, the check
is reliable for tools with zero or one argument and a coin flip for tools with more.

Evidence

Controlled run.

  • Greedy sampling fixes the model's output, so between runs only Swift's hashing differs.
  • The prompts MLX builds are byte-identical with and without SWIFT_DETERMINISTIC_HASHING=1 (swift-jinja sorts keys).

Identical consecutive rounds, caught / missed:

model, tool (arguments) default SWIFT_DETERMINISTIC_HASHING=1
Qwen3-4B, check_status(job_id, region) (2) 4 / 16 20 / 0
Qwen3-1.7B, a search tool (pattern, file, context) (3) 19 / 12 20 / 0
Qwen3-4B, check_status(job_id) (1) 10 / 0 10 / 0

Comparing the arguments with sorted keys instead removes every miss (20 / 0 and 20 / 0), which confirms the cause.

What the check compared. We logged both strings just before the comparison. Here is one missed repeat
(Gemma 4 e2b):

previous: calculate:["a": MLXLMCommon.JSONValue.string("12"), "b": MLXLMCommon.JSONValue.string("7"), "operation": MLXLMCommon.JSONValue.string("multiply")]
current:  calculate:["b": MLXLMCommon.JSONValue.string("7"), "a": MLXLMCommon.JSONValue.string("12"), "operation": MLXLMCommon.JSONValue.string("multiply")]

Across 14 models and 603 identical consecutive rounds, the check missed 286 times. Every miss was the same text in a
different order.

Without MLX:

import Foundation
let text = Data(#"{"pattern":"report","file":"notes.md","context":"0"}"#.utf8)
let rounds = try (0..<100).map { _ in try JSONDecoder().decode([String: String].self, from: text) }
print(Set(rounds.map { "\($0)" }).count)  // 6 (20 of 20 runs); 1 with SWIFT_DETERMINISTIC_HASHING=1 or a single key

For comparison, ToolRoundLimit (#262) compares calls structurally for the remote providers, and
LlamaLanguageModel signs calls with .sortedKeys JSON.

2. The assistant turn that made a tool call is never sent back to the model

Between rounds, the chat is rebuilt from the transcript plus pendingChat, but the assistant turn carrying the calls
is never added:

  • pendingChat gets .assistant(text) only when the round produced text (:1217-1219, :1485-1487).
  • Results are appended as .tool(content), with no id and no name (:1267, :1525).
  • convertTranscriptToMLXChat drops .toolCalls entries (:1737-1739).

So the chat template receives [system, user, tool]: a result with no call before it. What happens next depends on
the template:

  • Gemma 4 (e2b, e4b, their QAT builds; the 26B-A4B QAT template behaves the same) drops the result.
    • The round-2 prompt is byte-identical to round 1, so the model repeats the same call until the repeat check or the
      8-round cap stops it.
    • Across 280 round transitions, the result never appeared in the prompt.
  • gpt-oss-20b's template raises Message has tool role, but there was no previous assistant message with a tool call!. We checked this at the template level only.
  • Ministral 3's template raises After the optional system message, conversation roles must alternate user and assistant roles except for tool calls and results. once two rounds in a response produced text (2 of 6 runs).
  • Qwen3, Qwen3.5, Llama 3.2 and LFM2 render the result, but without the call that produced it.
Evidence: the same build with only the assistant turn added to pendingChat

Default sampling, 3–5 runs per mode, respond and streamResponse:

model scenario 1.0.0 + assistant turn
Gemma 4 e2b calculator ("12 * 7 + 3") 6/6 failed 6/6 answered 87
Gemma 4 e2b search tool that never matches 6/6 failed 6/6 answered that nothing was found
Gemma 4 e4b calculator 6/6 failed 6/6 answered 87
Gemma 4 E2B QAT calculator 6/6 failed 5/6 answered 87
Gemma 4 E4B QAT calculator 6/6 failed 4/6 answered 87 (1 empty answer)
Ministral-3-3B calculator 2/6 raised the alternation error 0/6
Qwen3-1.7B search tool 10/10 re-sent their previous call verbatim 0/10
Qwen3-8B search tool 6/6 re-sent their previous call verbatim 0/6
Qwen3-4B search tool 8/10 ended by announcing another search ("I will attempt to find … by searching the knowledgebase files") 0/10

Gemma's prompt, decoded, for "What is 12 * 7 + 3?".

  • On 1.0.0: rounds 2–9 are identical to round 1 (144 tokens).
  • With the turn: round 2 is round 1 plus
    <|tool_call>call:calculate{a:<|"|>12<|"|>,b:<|"|>7<|"|>,operation:<|"|>multiply<|"|>}<tool_call|><|tool_response>response:{value:<|"|>"84"<|"|>}<tool_response|>,
    and the model's next call is add(84, 3).

Llama 3.x. Its templates reject an assistant message carrying more than one call, with This model only supports single tool-calls at once!. With all of a round's calls in one message, Llama-3.2-3B hit that on every round where it
called tools in parallel.

3. What happens when a tool-round limit is hit

Since #262, every provider stops a response after 8 tool rounds, or when a round repeats the previous one, by throwing
GenerationError.decodingFailure. What we observed:

  • The tool work is discarded. After a limit error the transcript holds only the prompt (88 of 88 errors on 1.0.0).
    .preserveTranscript keeps the tool entries only for streamResponse.
  • The delegate never sees the round that hits the limit, in 373 of 373 cases. So it can't .stop or substitute
    output there.
  • The error is hard to identify. Foundation Models documents decodingFailure as a deserialization failure, so
    callers can only recognize this case by matching on debugDescription.
  • The repeat rule mostly stops legitimate repeats.
    • We ran 336 responses with the assistant turn added (section 2), across 13 models. The rule stopped 118 of them.
    • 113 were legitimate: 74 re-checks of a job whose first check returned pending ("check again until it is
      done"), and 39 second die rolls when the prompt asked for two.
    • The loops that did run long (18 runs) were 5–8 different calls, which only the cap stops.
    • On plain 1.0.0, a one-argument polling tool is rejected on its first re-check every time (12/12).
  • Foundation Models allows these repeats. We ran the same tools and prompts through SystemLanguageModel, 30
    runs: no errors.
    • It re-checked the pending job with identical arguments and answered (12/12).
    • It rolled the die twice with identical calls (6/6).
    • Against a search tool that never matches, it stopped after 3–5 searches and said nothing was found (6/6).

This touches the question on #147 ("prevents llms to call exact same tool twice … Was it ever discussed?") and #280.

Questions

  1. Is the key-order dependence in the MLX repeat check (section 1) unintended? Should it compare calls the way
    ToolRoundLimit does?
  2. Is leaving the assistant tool-call turn (and tool_call_id/name) out of the MLX chat history (section 2)
    deliberate, for example for template compatibility? Or should the model see its own calls, as it does with
    llama.cpp and the remote providers?
  3. When a tool-round limit is hit, is throwing decodingFailure and losing the round's work the behavior you want?
    And should identical consecutive calls count as a loop at all, given that Foundation Models allows them?

Happy to share the harness, which logs every round through ToolExecutionDelegate, and the raw data.

Ngôn ngữ chính
Swift
Star
948
Fork
99
Merge trung bình
19 giờ 50 phút
Pull request đã merge (30 ngày)
51

Chuẩn bị môi trường

Dự án này không cung cấp dev container, Dockerfile hay hướng dẫn đóng góp, nên bạn cần tự thiết lập môi trường: hãy bắt đầu từ README và xem hướng dẫn đóng góp lần đầu của chúng tôi để biết các bước chung.

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của huggingface/AnyLanguageModel

Tất cả issue của huggingface/AnyLanguageModel

Issue tương tự

Thêm issue về Swift

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.