glm-5.3 is listed as an OpenHands-provider model, but the OpenHands LLM proxy rejects it

Open
#6,924 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
46/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python

Research direction

Start by querying the OpenHands proxy’s /v1/models with a regular OpenHands LLM key and record which GLM models it serves. Then read openhands-sdk/openhands/sdk/llm/utils/verified_models.py and tests/sdk/llm/test_model_list.py; remove only unserved GLM entries from VERIFIED_OPENHANDS_MODELS, keeping the other provider lists unchanged. Done means the model-list tests pass and /llm/models/verified no longer offers removed models under OpenHands.

Written by the indexing model from the issue text.

Description

priority:medium ready-for-dev

What happened
On Agent Canvas 1.41.0 (local): Settings → LLM → Add LLM Profile → provider OpenHands → model glm-5.3 → Save. It fails with:

litellm.BadRequestError: Litellm_proxyException - /chat/completions: Invalid model name passed in model=glm-5.3. Call `/v1/models` to view available models for your key.

Why it happens
glm-5.3 is in VERIFIED_OPENHANDS_MODELS, the list of models the openhands/ provider serves, so Canvas offers it in the model picker. But the OpenHands LLM proxy doesn't serve it for a regular OpenHands LLM key.

https://github.com/OpenHands/software-agent-sdk/blob/98c8cd30ddfa7282f29bb95eb1df62f1c28c39e1/openhands-sdk/openhands/sdk/llm/utils/verified_models.py#L209

Expected
Either serve glm-5.3 on the proxy, or remove it from VERIFIED_OPENHANDS_MODELS so it isn't offered.


OpenHands AI triage

The following comments and acceptance criteria were added by the OpenHands AI agent.

Triage

Verified on main: glm-5.3, glm-5.3-flash, glm-5.3-flashx and glm-5.2 are in VERIFIED_OPENHANDS_MODELS (openhands-sdk/openhands/sdk/llm/utils/verified_models.py), which feeds both the model picker (agent-server GET /llm/models/verified) and the openhands/ provider allowlist (openhands_provider.py). The OpenHands LLM proxy is a separate deployed service; this repository cannot add models to it, so "serve glm-5.3 on the proxy" is a non-goal here. The repository-owned fix is to keep the openhands/ list aligned with what the proxy actually serves, which is the resolution used for the identical claude-opus-4-5 failure (PR #3421): remove the proxy-unserved entries from VERIFIED_OPENHANDS_MODELS only.

The GLM entries entered this list via the rolling model-list automation, whose evidence was LiteLLM model_cost plus OpenRouter availability — neither confirms OpenHands-proxy serving; kimi-for-coding is already excluded from the OpenHands list for exactly this reason. The four GLM entries stay in VERIFIED_GLM_MODELS, VERIFIED_OPENROUTER_MODELS and VERIFIED_DIGITALOCEAN_MODELS, and model_features.py is untouched, so direct GLM/BYOK, OpenRouter and DigitalOcean users are unaffected. The served-set question is settled empirically, not by policy: whoever implements must first query the proxy's /v1/models with a regular OpenHands LLM key.

Non-goals: changing the deployed OpenHands LLM proxy or its catalog; changing VERIFIED_GLM_MODELS, VERIFIED_OPENROUTER_MODELS, VERIFIED_DIGITALOCEAN_MODELS or model_features.py; changing how the openhands/ provider allowlist or /llm/models/verified works; adding curation automation; touching the clients/typescript surfaces (no contract change — the list is data).

If the proxy turns out to serve glm-5.3 (i.e., the report's environment was a transient proxy issue), stop and re-triage instead of applying the removal — the served-set observation in the implementation evidence decides which entries go.

Acceptance Criteria
  • Before any list change, the proxy catalog is queried with a regular OpenHands LLM key (GET https://llm-proxy.app.all-hands.dev/v1/models), and the query result (which, if any, of glm-5.3, glm-5.3-flash, glm-5.3-flashx, glm-5.2 appear) is recorded in the PR.
  • Every GLM entry in VERIFIED_OPENHANDS_MODELS that the queried proxy catalog does not serve is removed, and every entry that it does serve is kept. If the proxy serves glm-5.3, the issue is instead closed as not reproducible with a comment recording the catalog evidence.
  • VERIFIED_GLM_MODELS, VERIFIED_OPENROUTER_MODELS and VERIFIED_DIGITALOCEAN_MODELS still contain the GLM entries (unchanged), so direct-provider and OpenRouter/DigitalOcean pickers keep offering them.
  • uv run pytest tests/sdk/llm/test_model_list.py passes, including test_openhands_models_all_have_provider_list and test_verified_lists_keep_two_latest_versions_per_line.
  • A saved profile using openhands/glm-5.3 from a pre-fix build still loads after upgrading (no migration error); if openhands/glm-5.3 was removed from the list, such a profile's next LLM call surfaces the proxy's invalid-model error as before (the list is data only; no runtime allowlist behavior change is added or removed).
  • GET /llm/models/verified on an agent-server running the fixed build no longer offers the removed GLM models under the openhands provider, demonstrated with the route response.
Dominant language
Python
Stars
1.4k
Forks
797
Avg merge
1d 17h
Merged PRs (30d)
354

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from OpenHands/software-agent-sdk

All issues in OpenHands/software-agent-sdk

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.