Feature Request: Router: upstream clients' GET /models?reload=1 is silently inert, and a client can't detect which reload route exists

Aperta
#185 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

@dannyward630 ci sta già lavorando.

Dal 29/10/2026.

  • #193 di @dannyward630 — aperta

Valutazione

Difficoltà
3/5
Tempo stimato
1-2 giorni
Idoneità per principianti
25/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Ferma
Stack tecnologico
cpp, python
Ambito
api, backend

Direzione di ricerca

Start with the router's GET /props and GET /models handlers in the server sources, and find where reload=1 is read and where POST /models/reload is registered. The helper _get_model_ids(is_reload=...) in tests/unit/test_router.py (called around lines 511 and 523) and the note near lines 408-412 show the current behaviour. Work is already under way in open PR #193, so check it before starting. Done means a router props field names the reload endpoint, or the README documents the difference, and the router tests pass.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

enhancement
Prerequisites
  • I am running the latest code. Mention the version if possible as well.
  • I carefully followed the README.md.
  • I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
  • I reviewed the Discussions, and have a new and useful enhancement to share.
Feature Description

Please give clients a way to tell which request reloads the router preset. In order of preference:

  1. Advertise the capability in GET /props (router props), e.g. a list of supported router endpoints or a flag like "preset_reload": "POST /models/reload". Then a client picks the right request once and never guesses.
  2. At minimum, a line in the README next to POST /models/reload saying upstream does this with GET /models?reload=1 and that the query form is intentionally inert here.

I am not asking you to make GET mutate state. I see test_router.py:408-412 and the README line stating that GET /models is read-only, and the reasoning is sound.

Motivation

BeeLlama's router reloads the preset on POST /models/reload. Upstream llama.cpp reloads it on GET /models?reload=1 and has no POST route (checked across all of tools/server at the commit BeeLlama branches from, 465e49b9, and at current master: models/reload appears only in a stale docstring in tests/unit/test_router.py). A client written for upstream sends GET /models?reload=1, gets 200 with a normal listing, and concludes the reload happened. Nothing reloaded: the file on disk is right, the model still starts with the old arguments, and only a router restart fixes it.

Measured on 0.4.7 (build 11860), Windows 10, one model, all unloaded:

status.args of model X     -> --n-cpu-moe 14
edit the preset to n-cpu-moe = 15
GET /models?reload=1       -> 200, status.args still --n-cpu-moe 14
POST /models/reload        -> 200 {"success": true}, status.args --n-cpu-moe 15

Same with a newly added section: invisible after the GET, present after the POST. preview-v0.4.8 behaves the same.

Affected client: LlamaForge — https://github.com/dadwritestech/LlamaForge/issues/37 (every Save, Scan, auto-tune round and preset apply goes through the query form there).

Related: test_router_models_status_args_preset keeps status.args/status.preset for exactly this kind of client compatibility (#117). This is the same class of problem, one layer up.

Possible Implementation

A field in router GET /props naming the reload endpoint would be enough; clients can fall back to today's behaviour when it is absent.

Minor, unrelated: the helper _get_model_ids(is_reload=...) in tests/unit/test_router.py still carries upstream's ?reload=1 branch and is called with is_reload=True at :511 and :523, where it now just lists models without reloading. Harmless, but the name claims something it no longer does.

Lingua principale
C++
Stelle
1.2k
Fork
80
Merge medio
4g 20h
PR unite (30g)
2

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di Anbeeld/beellama.cpp

Tutte le issue di Anbeeld/beellama.cpp

Issue simili

Altre issue su C++

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.