Feature Request: Router: upstream clients' GET /models?reload=1 is silently inert, and a client can't detect which reload route exists

Offen
#185 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

@dannyward630 arbeitet bereits daran.

Seit 29.10.2026.

  • #193 von @dannyward630 — offen

Bewertung

Schwierigkeit
3/5
Geschätzter Aufwand
1-2 Tage
Anfängerfreundlichkeit
25/100
Issue-Typ
Feature
Klarheit
Größtenteils klar
Aktivitätsstatus
Veraltet
Tech-Stack
cpp, python
Bereich
api, backend

Rechercherichtung

Start with the router's GET /props and GET /models handlers in the server sources, and find where reload=1 is read and where POST /models/reload is registered. The helper _get_model_ids(is_reload=...) in tests/unit/test_router.py (called around lines 511 and 523) and the note near lines 408-412 show the current behaviour. Work is already under way in open PR #193, so check it before starting. Done means a router props field names the reload endpoint, or the README documents the difference, and the router tests pass.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

enhancement
Prerequisites
  • I am running the latest code. Mention the version if possible as well.
  • I carefully followed the README.md.
  • I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
  • I reviewed the Discussions, and have a new and useful enhancement to share.
Feature Description

Please give clients a way to tell which request reloads the router preset. In order of preference:

  1. Advertise the capability in GET /props (router props), e.g. a list of supported router endpoints or a flag like "preset_reload": "POST /models/reload". Then a client picks the right request once and never guesses.
  2. At minimum, a line in the README next to POST /models/reload saying upstream does this with GET /models?reload=1 and that the query form is intentionally inert here.

I am not asking you to make GET mutate state. I see test_router.py:408-412 and the README line stating that GET /models is read-only, and the reasoning is sound.

Motivation

BeeLlama's router reloads the preset on POST /models/reload. Upstream llama.cpp reloads it on GET /models?reload=1 and has no POST route (checked across all of tools/server at the commit BeeLlama branches from, 465e49b9, and at current master: models/reload appears only in a stale docstring in tests/unit/test_router.py). A client written for upstream sends GET /models?reload=1, gets 200 with a normal listing, and concludes the reload happened. Nothing reloaded: the file on disk is right, the model still starts with the old arguments, and only a router restart fixes it.

Measured on 0.4.7 (build 11860), Windows 10, one model, all unloaded:

status.args of model X     -> --n-cpu-moe 14
edit the preset to n-cpu-moe = 15
GET /models?reload=1       -> 200, status.args still --n-cpu-moe 14
POST /models/reload        -> 200 {"success": true}, status.args --n-cpu-moe 15

Same with a newly added section: invisible after the GET, present after the POST. preview-v0.4.8 behaves the same.

Affected client: LlamaForge — https://github.com/dadwritestech/LlamaForge/issues/37 (every Save, Scan, auto-tune round and preset apply goes through the query form there).

Related: test_router_models_status_args_preset keeps status.args/status.preset for exactly this kind of client compatibility (#117). This is the same class of problem, one layer up.

Possible Implementation

A field in router GET /props naming the reload endpoint would be enough; clients can fall back to today's behaviour when it is absent.

Minor, unrelated: the helper _get_model_ids(is_reload=...) in tests/unit/test_router.py still carries upstream's ?reload=1 branch and is called with is_reload=True at :511 and :523, where it now just lists models without reloading. Harmless, but the name claims something it no longer does.

Vorherrschende Sprache
C++
Sterne
1.2k
Forks
80
Ø Merge
4 T. 20 Std.
Gemergte PRs (30 T.)
2

Entwicklungsumgebung

Dieses Projekt bietet weder Dev-Container noch Dockerfile noch Beitragsleitfaden – die Einrichtung liegt bei Ihnen. Beginnen Sie mit der README; die allgemeinen Schritte stehen in unserem Leitfaden für den ersten Beitrag.

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus Anbeeld/beellama.cpp

Alle Issues in Anbeeld/beellama.cpp

Ähnliche Issues

Weitere Issues zu C++

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.