[Engine] Native /v1/models still has no context window on 0.2.13 — engine reports n_ctx, the host gateway drops it

Aperta Adatta ai principianti
#5,836 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 4 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
62/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
cpp, javascript, openapi
Ambito
api

Direzione di ricerca

La correzione si trova in host/server/index.mjs, nel ramo native di GET /v1/models, dove ogni istanza viene mappata a una voce con campi meta. Leggi lì la mappatura e il calcolo esistente di max_context in localInstances, oltre a host/server/model-providers.mjs per la denominazione context_length. È completato quando le voci includono context_length, max_model_len e meta.n_ctx dal valore del motore, con le chiavi omesse se il valore è sconosciuto, verificato con un curl contro il listener dell'host.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Area: [Engine] (speed, kernels, models, the server and API)
Desktop app, Terminal or source?: Desktop app (Windows)


What happened

I loaded Qwen3.8-Flash-Next IQ3_XXS (131072 context, images on) in the desktop app, then asked the host gateway for the model list:

GET http://127.0.0.1:47831/v1/models
{"object":"list","data":[{"id":"iq3_xxs","object":"model","owned_by":"strata",
 "meta":{"instanceId":"…","gpus":[0],"label":"Qwen3.8-Flash-Next IQ3_XXS","aliases":["iq3_xxs"]}}]}

There is no n_ctx, no context_length, no max_model_len — nothing a client can size a context window from.

The engine behind it does report the value, three ways:

GET http://127.0.0.1:18500/v1/models   →  "meta": {"n_ctx": 131072}
GET http://127.0.0.1:18500/props       →  "default_generation_settings": {"n_ctx": 131072}
GET http://127.0.0.1:18500/metrics     →  "engine": {"max_context": 131072}

So the data exists at the engine and is dropped when the host builds its own model list. What I expected instead: the host's /v1/models entries carry the loaded instance's context window under meta.n_ctx plus context_length and max_model_len (the names #1882 proposes), on the primary entry and on alias entries.

Why it matters: OpenAI-compatible agent clients size their context window from this list. When the key is missing they fall back to their own default — in my case 1M against a 128K engine — so they never compact and the conversation is hard-truncated at 131072 instead of being summarized before the limit. Working around it means hand-configuring the context window in every client, on every machine, with no way to keep them consistent.

GET http://<lan-ip>:47832/v1/models (the share listener) returns the same meta with exactly those four keys, so sharing hits the same gap.

This is the same report as #3931, which was closed by the stale-issue sweep rather than by a fix, so I re-tested on 0.2.13 as that comment asked. Still reproduces.


Smallest steps or command that reproduce it

  1. Load IQ3_XXS in the app (Settings > Models > Load).
  2. curl -s -H "Authorization: Bearer $KEY" http://127.0.0.1:47831/v1/models → the response above, with no context key.
  3. For comparison, read the engine directly: curl -s -H "Authorization: Bearer $ENGINE_KEY" http://127.0.0.1:18500/v1/models → meta.n_ctx: 131072.
    ($KEY is the serving key from /api/serving; $ENGINE_KEY is api_key in native-instances/<instanceId>.json.)

Reproduction status: Reproduced locally

Strata version: Strata 0.2.13 · engine 0.1.59 (windows-x86_64-cuda)

Graphics card, memory and system: RTX 5060 Ti 16 GB · 96 GB RAM · Windows 11 Pro

Model, size and settings: Qwen3.8-Flash-Next IQ3_XXS · 131072 context · pictures on

Logs: can attach strata-iq3_xxs.log if useful — nothing in it looks related, the engine reports the value fine.


Where the fix would live (for the maintainers)

host/server/index.mjs, the native branch of /v1/models (line ~2145 in 0.2.13). The instance already holds the value and the same file computes it elsewhere for localInstances as i.metrics?.engine?.max_context ?? i.context; host/server/model-providers.mjs:143 already reads context_length from remote providers, so the naming convention is in the project.

Current:

.map(i=>({id:i.apiModelId,object:'model',owned_by:'strata',meta:{instanceId:i.id,gpus:i.gpus,label:i.label,aliases:i.aliases}}))

Proposed:

.map(i=>{const ctx=i.metrics?.engine?.max_context??i.context??0;return{id:i.apiModelId,object:'model',owned_by:'strata',
  context_length:ctx||undefined,max_model_len:ctx||undefined,max_input_tokens:ctx||undefined,
  meta:{n_ctx:ctx||undefined,instanceId:i.id,gpus:i.gpus,label:i.label,aliases:i.aliases}}})

Keys are omitted rather than sent as 0 when the value is not known yet (metrics are polled every 2s and can be briefly null right after load). I can send this as a PR if you prefer a patch over an issue.

Lingua principale
C++
Stelle
11.6k
Fork
1k
Merge medio
18g 13h
PR unite (30g)
4

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di Niko1221/Strata

Tutte le issue di Niko1221/Strata

Issue simili

Altre issue su C++

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.