Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

[M3 Bug] Tool calling: array entries silently dropped into an invented `item` key

Aperta
  • Qualcosa non va?
#29 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
25/100
Tipo di issue
Bug
Chiarezza
Da chiarire
Stato di attività
Tranquilla
Ambito
api

Direzione di ricerca

Non sono stati identificati file del repository o test. Inizia riproducendo il problema attraverso gli endpoint elencati usando gli item_ids controllati e l'esperimento con le parole del prompt; il lavoro sarebbe completato con una causa confermata, una correzione dell'implementazione o una risposta strict-mode supportata e la convalida che le voci degli array non vengano più scartate.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug
Which inference path did you use?

MiniMax API

Inference parameters

API defaults (no temperature / top_p / top_k set). Also reproduced with temperature=0 - see Additional context.

Prompt / input
Endpoint: POST https://api.minimax.io/v1/chat/completions
tool_choice: {"type": "function", "function": {"name": "final_result"}}

Tool parameters schema:


{"type":"object","properties":{"groups":{"type":"array","items":{"type":"object","properties":{"item_ids":{"type":"array","items":{"type":"string"}}},"required":["item_ids"]}}},"required":["groups"]}


User message:


    Sort the following items into groups, where each group contains items that belong to the same category:
    id a1: apple
    id a2: banana
    id b1: carrot
    id b2: potato
    id c1: hammer
    id c2: screwdriver
    Explain which items belong together and why, then return the grouping.
Expected behavior

Three groups (fruit / vegetables / tools), all six ids present:

{"groups":[{"item_ids":["a1","a2"]},{"item_ids":["b1","b2"]},{"item_ids":["c1","c2"]}]}
Actual behavior

Roughly 15–35% of calls return only the first entry inside the array and flatten the rest into an invented top-level key item that does not exist in the schema.
The middle group is lost entirely — only 4 of the 6 ids survive:

{"groups":[{"item_ids":["a1","a2"]}],"item":{"item_ids":["c1","c2"]}}

The malformed output is byte-identical across independent failing calls.

There is no error signal. The response is a normal 200 OK with finish_reason: "tool_calls", syntactically valid JSON, and base_resp: {"status_code": 0}. Token counts look normal (nothing truncated). Because the surviving JSON is well-formed, standard clients that ignore unexpected keys (e.g. pydantic's default extra="ignore") drop the item key silently - so entries vanish with no exception and no retry.

The visible <think> block shows the model reasoning out the correct three groups in prose, then emitting the malformed structure anyway.

Additional context

Frequency: 7/20 (35%) on chat completions in the sharpest run; 10–35% across runs. Failures cluster — within a batch of concurrent requests it is often all-or-nothing rather than an even split, which may point to serving-side batching or routing correlating outcomes across simultaneous requests.

Not transport. Same signature on all three endpoints (N=20 each):

Endpoint Violations
/v1/chat/completions 4/20
/v1/responses 2/20
/anthropic/v1/messages 3/20

Not sampling. temperature: 0 does not suppress it — 6/20 at temp 0 vs 3/20 at default, interleaved in a single batch so both arms share the same serving window.

Worse variant. Give the array element two fields and groups stops being an array at all, with both inner field names leaking to the top level:

{"groups":{"item":{"item_ids":["a1","a2"]},"category":"Fruits"},"item":{"item_ids":["c1","c2"]},"category":"Tools"}

Questions: Is this known / is a fix planned? Is there a supported strict or constrained-decoding mode for tool calls?

Update 2026-07-28: I re-tested this with a controlled experiment and the original diagnosis was too broad. It is not array-of-objects in general. Two conditions have to hold together:

  1. an element property named item_ids (or similar item* name), and
  2. a prompt that never uses the word "item".

2×2 factorial, three independent serving windows, ~176 calls, requests throttled to 4 concurrent and all four cells interleaved so they share a window:

element property prompt uses "item" prompt avoids "item"
item_ids 0/45 12/45 (27%) — per round 3/15, 4/15, 5/15
member_ids 0/42 0/44

The whole effect is confined to one cell, so this is an interaction - neither factor does anything on its own. Two independent fixes each take it to zero:

  • rename the property - member_ids or element_ids is clean (0/86 across both prompts), same schema shape otherwise;
  • or ground the name in the prompt - keeping item_ids but ending the prompt with "Each group must list the id strings of its items." gives 0/45.

Only the second sentence differs between the two prompt arms:

prompt A (clean):   Sort the following items into groups ... Explain which items
                    belong together and why, then return the grouping.
                    Each group must list the id strings of its items.

prompt B (27% bad): Sort the following products into groups ... Explain which
                    products belong together and why, then return the grouping.
                    Each group must list the id strings of its members.

Everything else - the schema, the six ids, tool_choice forcing the call - is unchanged from the original report. The malformed output is also unchanged: one group inside groups, one moved to a top-level item, the middle group gone. Every invented key I have observed, in every run, is literally item.

Guess at the mechanism, since it might point somewhere useful: items is the JSON Schema keyword that describes array elements. A property with that name and no anchor in the prompt looks like it gets conflated with the keyword, so the model emits a sibling item holding a single element instead of appending it to the array.

Realistic schemas with no item* property have been clean in everything I have run since — 26/26 calls on one real workload, and 56/56 on a larger schema across three different ways of encoding its numeric vectors (arrays, flattened scalars, delimited strings, all 0 violations). So restructuring a schema does not help; renaming the colliding property does.

This is no longer a problem for us, since we can easily work around it. I would love to hear what the cause/mechanism of this bug is though - I'm really curious.

Lingua principale
Nessun dato sulla lingua
Stelle
491
Fork
59
Nessuna PR unita negli ultimi 30g

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di MiniMax-AI/MiniMax-M3

Tutte le issue di MiniMax-AI/MiniMax-M3

Issue simili

Altre issue su Backend & API Design

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.