[Feature]: Discover available models from OpenAI-compatible custom base_url for the model picker and drift validation
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 43/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- openapi, python
- Domain
- api, backend-api-design
Research direction
Start by reading the agent-server route and profile validation code, then inspect the server-side fetch patterns in runtime_metadata.py and openrouter.py. Add the upstream catalog endpoint with normalized model IDs and warning-only validation; confirm both response shapes, auth handling, redacted failures, and the OpenAPI and TypeScript client updates. Done means the listed acceptance criteria pass, including the local HTTP stand-in tests.
Written by the indexing model from the issue text.
Description
Summary
Add a way for the model picker / LLM profile editor to discover available models from an OpenAI-compatible custom base_url (fetch GET {base_url}/v1/models), instead of only offering LiteLLM's static model list (GET /api/llm/models) and the hardcoded verified lists.
Problem
When an LLM profile points at a local proxy or gateway (base_url set, e.g. LiteLLM proxy, one-api/new-api style aggregators, or a self-hosted router), the set of valid model IDs is determined by that upstream, not by LiteLLM's static catalog. Today:
- The settings UI model picker is populated from
GET /api/llm/models(LiteLLM static list, ~3.8k entries) plusGET /api/llm/models/verified. Models that exist only behind the custom endpoint are not listed; conversely, listed names may not exist upstream. - If the saved profile
modelreferences an upstream-only ID, nothing validates it against the upstream catalog. A mismatch surfaces only at conversation time as an upstream400 model_not_found, which is non-retryable and kills the conversation (LLMBadRequestError,classification.kind=config).
Concrete incident from our deployment: a profile pinned to one upstream's model ID kept working while a router in front of it redirected traffic to a different provider with a strict pass-through model mapping — the stale model ID then failed every new conversation with 400 model_not_found until manually re-pointed. A catalog check against base_url would have detected the drift immediately.
Related work
- #6863 / #6862 — provider connections becoming authoritative for routing and model ID normalization. This issue complements that work on the discovery/validation side: knowing which IDs the endpoint actually serves.
- The SDK already live-fetches metadata for one special-cased provider (OpenRouter, hardcoded
openrouter.aiendpoint insdk/llm/utils/providers/openrouter.py). This request generalizes that pattern to any OpenAI-compatiblebase_url. - LiteLLM itself exposes
client.models.list()for OpenAI-compatible endpoints, so the underlying client capability exists.
Proposal
- Catalog endpoint on the agent-server, e.g.
GET /api/llm/models/upstream?base_url=...(or accept a profile name): fetch{base_url}/v1/modelsserver-side, normalize both OpenAI format ({"data":[{"id":...}]}) and common variants, return{models: [...]}. No secrets leave the server; the profile's stored API key is used for authenticated endpoints, with an anonymous fallback for open endpoints. - Model picker integration: when a profile has a
base_url, offer "models from this endpoint" (live fetch) alongside the static LiteLLM list, with a clear source label. - Optional drift check: when saving/activating a profile, optionally validate
modelagainst the fetched catalog (warning, not a hard error — some gateways lazily accept IDs).
Notes on format variance
Real-world gateways vary: some return strict OpenAI data[] format, others return flat arrays like {"models":[{"slug":...,"display_name":...}]} (the OpenAI Codex CLI native catalog format). Accepting both (and maybe id/slug/name keys) covers most deployments; a normalization helper would also let proxy projects add an OpenAI-compatible models endpoint cheaply.
Environment
- OpenHands Agent Server 1.62.0 / SDK 1.62.0 (WSL2, local Agent Canvas)
- Proxy in front: third-party LLM router providing OpenAI-compatible
/v1/responses+ native/_v1/models
This issue was created by an AI agent (OpenHands) on behalf of @JohnieXu.
OpenHands AI triage
The following comments and acceptance criteria were added by the OpenHands AI agent.
Triage
Bounded scope: this repository owns the Python SDK and the agent-server REST API (AGENTS.md), so the discovery endpoint and the profile-validation hook belong here. Add one additive agent-server endpoint, GET /api/llm/models/upstream, that takes a base_url (or a profile name) server-side, fetches that endpoint's model catalog with httpx (same server-side fetch pattern as runtime_metadata.py/openrouter.py), normalizes both OpenAI data[].id and flat models[] (id/slug/name) shapes, and returns {models: [...]}. The request uses the stored profile/provider-connection API key server-side when one exists, with an anonymous fallback; no secrets leave the server and none may appear in responses, logs, or errors. POST /api/llm/profiles/{name}/validate may add an optional, non-blocking catalog check of the profile's model against the fetched catalog (warning-only, mirroring _TRANSIENT_ERROR_TYPES treatment). Model-picker UI presentation is a Canvas change (OpenHands/OpenHands); the endpoint must land here first per the SDK → OpenAPI → clients/typescript → Canvas flow.
Non-goals: no change to GET /api/llm/models, GET /api/llm/models/verified, or VERIFIED_MODELS/unverified_models.py catalogs; no new LiteLLM provider integration; no hard blocking on save/activation for unknown model IDs (some gateways lazily accept IDs); no rewrite of stored profiles; no Canvas UI work in this repository.
Acceptance Criteria
-
GET /api/llm/models/upstreamaccepts abase_urlquery parameter (or a stored profile name) and returns HTTP 200 with{"models": [...]}listing the model IDs the upstream endpoint serves, for both OpenAI{"data":[{"id":...}]}and flat{"models":[...]}response shapes (normalizingid/slug/namekeys). - When the profile or its linked provider connection has an API key, the upstream catalog request is sent server-side with that key; the key never appears in the response body, logs, or error text.
- Anonymous endpoints reachable without credentials are fetched without an API key instead of failing.
- The upstream request is bounded by a finite timeout; timeout, DNS failure, non-2xx status, and unparseable bodies return a typed 4xx/5xx error (not a hang and not a raw 500) with a redacted message.
-
POST /api/llm/profiles/{name}/validateoptionally reports a catalog mismatch between the profile'smodeland the fetched upstream catalog as a non-blocking warning; a failed catalog fetch alone does not fail validation. - The change is additive: no existing
/api/**operation or response schema changes (the agent-server REST breakage check passes), andGET /api/llm/modelsplus/models/verifiedbehavior is unchanged. - The endpoint appears in the agent-server OpenAPI schema and is mirrored in
clients/typescript/per the SDK → OpenAPI → TypeScript client flow. - Automated tests cover the route against a local HTTP stand-in: both response shapes, authenticated and anonymous fetch, timeout/non-2xx/malformed-body errors, and the warning-only validate behavior; live coverage where credentials are available is not required.
- Dominant language
- Python
- Stars
- 1.4k
- Forks
- 797
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 354
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from OpenHands/software-agent-sdk
-
[Bug]: `list_directory` reports a complete listing as truncated when a directory has exactly 500 entriesPossibly taken @balabommablock-cpu claimed this 29 days ago. Openbug priority:low ready-for-dev
Difficulty 2/5 1-3 hours Newbie friendliness 87/100
OpenHands/software-agent-sdk#6133 ·
Maintainers usually reply within 1 day
-
[Bug]: Command hook that exits 1 is treated as blocking when its stdout is a JSON `deny` or `continue: false`Possibly taken A pull request linked to this issue is open or already merged. Openbug priority:low ready-for-dev
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
OpenHands/software-agent-sdk#6116 · 3 comments ·
Maintainers usually reply within 1 day
-
[Bug]: TypeScript GET-with-body requests remain pending after a truncated responsePossibly taken @BlueRaddish claimed this 34 days ago. Openbug priority:low ready-for-dev
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
OpenHands/software-agent-sdk#6059 ·
Maintainers usually reply within 1 day
-
[Bug]: Document whether `prompt_tokens` includes `cache_read_tokens` in accumulated token usagePossibly taken @Jaxon1216 claimed this 36 days ago. Openpriority:low ready-for-dev
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
OpenHands/software-agent-sdk#5975 ·
Maintainers usually reply within 1 day
-
priority:low
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
OpenHands/software-agent-sdk#5708 · 1 comment ·
Maintainers usually reply within 1 day
All issues in OpenHands/software-agent-sdk
Similar issues
-
[Bug]: reme CLI crashes when the running server used a relative config pathPossibly taken @nurlanmnn claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
agentscope-ai/ReMe#770 ·
Maintainers usually reply within 1 day
-
Pi: paused model calls are reported as "not signed in" when the model's provider comes from an extensionPossibly taken @balcsida claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 73/100
microsoft/TokenEconomizer#288 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 67/100
sickn33/agentic-awesome-skills#2437 ·
Maintainers usually reply within 1 day
-
site-bug triage
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
ethereum/consensus-specs#5846 ·
Maintainers usually reply within 1 day