Agent loop stops after first tool call with non-Claude models (GLM-5): generateSandboxedUi never invoked

Reproduce the issue using apps/agent/src/model.py and its build_model() configuration for GLM-5. Trace the tool-call continuation after plan_visualization through deepagents, CopilotKitMiddleware, and ag-ui-langgraph, using the reported _generate/_agenerate and HTTP-request observations as checkpoints. Done means a second LLM request invokes generateSandboxedUi and the UI renders an iframe.

Summary

When using a non-Claude OpenAI-compatible model (tested with GLM-5 via Zhipu coding endpoint https://open.bigmodel.cn/api/coding/paas/v4), the agent correctly calls plan_visualization on the first LLM turn but never proceeds to a second LLM turn to call generateSandboxedUi. The result is that the user sees the assistant's text description of the widget that "would" be built, but no sandboxed iframe is ever rendered (0 iframes in the DOM).

The README warns that "weaker models will produce broken layouts", but this is a different failure mode: GLM-5 successfully generates a complete, valid widget when prompted directly (9.6 KB of JS, all 6 generateSandboxedUi parameters emitted in the correct order). The failure is not model capability — it's that the agent loop doesn't give the model a second turn after the plan_visualization tool returns.

Environment

• Repo commit: 457e60c (current main as of 2026-07-18)
• Model: glm-5 (Zhipu coding endpoint), wired in via langchain_openai.ChatOpenAI with LLM_MODEL=glm-5, LLM_PROVIDER=openai
• Python 3.13, Node 24, pnpm 10, uv
• macOS Darwin 25.5.0
• Same behavior on fresh thread (cleared localStorage + reopened browser)

Repro

1. Replace apps/agent/src/model.py build_model() so that LLM_MODEL=glm-5 + LLM_PROVIDER=openai routes to ChatOpenAI with the Zhipu base_url.
2. make setup + make dev.
3. Open http://localhost:3000, type "Draw binary search visualization".
4. GLM-5 replies with text + calls plan_visualization. UI shows the "✓ Plan: visualization" pill.
5. Nothing else happens. No iframe, no generateSandboxedUi activity event.

Evidence

What the agent loop sees (first LLM turn) — captured by monkey-patching bind_tools on the model:

[MODEL_DEBUG] bind_tools (19): ['write_todos', 'ls', 'read_file', 'write_file', 'edit_file', 'glob', 'grep', 'task', 'query_data', 'plan_visualization', 'manage_todos', 'get_todos', 'generate_form', 'generateSandboxedUi', 'toggleTheme', 'pieChart', 'barChart', 'scheduleTime', 'render_a2ui']

generateSandboxedUi IS in the offered tools. GLM-5 returns:

finish_reason: tool_calls
content: "I'll create an interactive binary search visualization that shows how the algorithm narrows down the search space step by step. Let me plan and build this for you."
tool_calls: [plan_visualization(approach=..., technology=..., key_elements=[...])]

After plan_visualization returns its string, no further LLM call is made — verified by hooking both _generate / _agenerate and the underlying httpx client; zero additional HTTP requests to /chat/completions after the first turn.

Direct replay against GLM-5 API with the same system prompt + same tool list + the plan_visualization tool result already in messages:

messages = [
    {"role":"system","content":"You MUST call generateSandboxedUi now that plan is done."},
    {"role":"user","content":"Draw binary search visualization"},
    {"role":"assistant","content":"I'll build this.","tool_calls":[{"id":"call_1",...,"function":{"name":"plan_visualization","arguments":"..."}}]},
    {"role":"tool","tool_call_id":"call_1","content":"Plan: step-by-step\nTech: SVG\n- array\n- pointers\n- controls"},
]

GLM-5 returns:

finish_reason: tool_calls
content: "" (empty)
tool_calls: [generateSandboxedUi(
    initialHeight=580,
    placeholderMessages=[3 items],
    css="<3201 chars>",
    html="<2086 chars>",
    jsFunctions="<9673 chars>",   # full binary search algorithm + UI
    jsExpressions=["<347 chars>"]
)]
usage: completion_tokens=4839 (incl. 87 reasoning_tokens)

So the model is fully capable of producing the widget in a single shot once the plan_visualization tool result is fed back — but the agent loop never feeds it back.

Hypothesis

When finish_reason=tool_calls and content is non-empty simultaneously (GLM-5's default style — it emits a short sentence alongside the tool call), some layer in deepagents + CopilotKitMiddleware + ag-ui-langgraph appears to treat the message as a final response and does not run the tool-→-LLM-continue cycle. Claude models, per the README's recommended config, apparently emit plan + widget in a single turn or in a way that sidesteps this path; GLM emits them across two turns and gets stuck between.

I tried to nail down the exact layer but the LangChain 1.x callback surface (on_chat_model_start etc.) doesn't fire for the calls that GLM-5 actually answers, so I couldn't trace whether deepagents ran the second turn and it was dropped at the CopilotKit middleware, or whether deepagents itself never re-entered the model.

Suggested fixes / things to try

1. Force a second LLM turn after plan_visualization even when the first turn's message has both content and tool_calls. The current behavior looks like a short-circuit on "has content → treat as final".
2. Or allow plan_visualization to be skipped so a model that prefers to call generateSandboxedUi directly can do so in a single turn.
3. Or document explicitly that non-Claude models must emit plan_visualization and generateSandboxedUi in a single assistant turn, with a system-prompt nudge that makes GLM/etc. do that.
4. Investigate whether ag-ui-langgraph or CopilotKitMiddleware is the layer that drops the continuation — happy to dig further if a maintainer can point me at where to instrument.

Reproducibility script

I have a self-contained Python script that hits GLM-5 directly with the same tool schemas and prompts, reproducing the "2-turn happy path" reliably — can paste it in if useful.

Why I'm reporting this

The project is marketed as "open-source showcase for AI-generated UI" and the README explicitly supports gpt-* fallback via LLM_MODEL. Any user trying it with a non-Claude OpenAI-compatible endpoint (GLM / DeepSeek / Qwen / Kimi / MiniMax / etc.) will hit this silent half-failure and conclude the model is too weak, when in fact it's an agent-loop bug. Worth either fixing or documenting.