mlx_lm.server assigns 36-char UUID tool-call ids, which Mistral v3 templates reject (length-9 constraint)

Start in server.py at ToolCallFormatter._format, where missing tool-call IDs are generated, and review the related tool-call handling. Add a regression case in tests/test_server.py covering a 9-character alphanumeric ID and the Mistral template reproduction. Done means the follow-up tool exchange renders successfully without violating the stated ID constraint.

Summary

When a tool call comes back without an id, mlx_lm.server assigns str(uuid.uuid4()). Mistral v3 chat templates require tool-call ids to be exactly 9 alphanumeric characters and raise otherwise. A multi-turn tool exchange with these models therefore fails on the follow-up request: rendering the prior assistant tool call through the template raises, and the server returns HTTP 404.

Environment

• mlx-lm 0.31.3 and current main
• Python 3.12, macOS arm64
• Affected: Mistral checkpoints whose template enforces the 9-char id rule. Confirmed on mlx-community/Ministral-8B-Instruct-2410-4bit; Mistral-7B-Instruct-v0.3, Mixtral, and Mistral-Nemo share the same template family.
• Not affected: Devstral-Small-2-...-2512, whose template has no id-length constraint.

Reproduction (template only, version-independent)

import uuid
from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained("mlx-community/Ministral-8B-Instruct-2410-4bit")
tools = [{"type": "function", "function": {"name": "calculator",
          "parameters": {"type": "object",
          "properties": {"expression": {"type": "string"}}, "required": ["expression"]}}}]

bad = str(uuid.uuid4())          # 36 chars, what server.py assigns
msgs = [
    {"role": "user", "content": "What is 2+3?"},
    {"role": "assistant", "content": "", "tool_calls": [
        {"id": bad, "type": "function",
         "function": {"name": "calculator", "arguments": "{\"expression\": \"2+3\"}"}}]},
    {"role": "tool", "tool_call_id": bad, "name": "calculator", "content": "5"},
]
tok.apply_chat_template(msgs, tools=tools, tokenize=False)
# -> TemplateError: Tool call IDs should be alphanumeric strings with length 9!

# A 9-char alphanumeric id renders cleanly:
good = uuid.uuid4().hex[:9]      # e.g. "0819261ae"
# ... same messages with `good` instead of `bad` -> renders fine

End to end this shows up as HTTP 404 on the second /v1/chat/completions call (the one that carries the prior tool call and its result), because the server renders the history through apply_chat_template and the template raises.

Root cause

ToolCallFormatter._format in server.py:

tc_id = tc.pop("id", None) or str(uuid.uuid4())

str(uuid.uuid4()) is 36 characters with hyphens, which violates the Mistral v3 constraint of 9 alphanumeric characters.

Suggested direction

Two options, and I would like a maintainer's view before sending a patch:

1. Make the generated id 9 alphanumeric characters globally, for example uuid.uuid4().hex[:9]. This satisfies the Mistral templates and stays valid for templates that do not constrain the id.
2. Generate the constrained id only for templates that require it, leaving the current id elsewhere.

Option 1 is simpler and I have not found a template that rejects a short alphanumeric id, but option 2 is more conservative. Happy to open a PR with whichever you prefer, plus a tests/test_server.py case.

Note on ordering

This is the last step of the Mistral tool-calling round-trip: a follow-up turn is only reached after the first turn returns a tool call, which today also depends on #1307 and #1373 (the EOS flush) and on the parser accepting the model's output (sibling issue). Reported separately because the root cause, the file, and the fix are distinct, and the reproduction above hits the template directly.

Related

• #1307 and #1373
• #1374 (Mistral JSON-list tool-call parser format, the earlier step of the same round-trip)