Feature Request: Router: upstream clients' GET /models?reload=1 is silently inert, and a client can't detect which reload route exists
Évaluation
- Difficulté
- 3/5
- Temps estimé
- 1-2 jours
- Accessibilité débutants
- 25/100
Piste de recherche
Start with the router's GET /props and GET /models handlers in the server sources, and find where reload=1 is read and where POST /models/reload is registered. The helper _get_model_ids(is_reload=...) in tests/unit/test_router.py (called around lines 511 and 523) and the note near lines 408-412 show the current behaviour. Work is already under way in open PR #193, so check it before starting. Done means a router props field names the reload endpoint, or the README documents the difference, and the router tests pass.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Description
Prerequisites
- I am running the latest code. Mention the version if possible as well.
- I carefully followed the README.md.
- I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
- I reviewed the Discussions, and have a new and useful enhancement to share.
Feature Description
Please give clients a way to tell which request reloads the router preset. In order of preference:
- Advertise the capability in
GET /props(router props), e.g. a list of supported router endpoints or a flag like"preset_reload": "POST /models/reload". Then a client picks the right request once and never guesses. - At minimum, a line in the README next to
POST /models/reloadsaying upstream does this withGET /models?reload=1and that the query form is intentionally inert here.
I am not asking you to make GET mutate state. I see test_router.py:408-412 and the README line stating that GET /models is read-only, and the reasoning is sound.
Motivation
BeeLlama's router reloads the preset on POST /models/reload. Upstream llama.cpp reloads it on GET /models?reload=1 and has no POST route (checked across all of tools/server at the commit BeeLlama branches from, 465e49b9, and at current master: models/reload appears only in a stale docstring in tests/unit/test_router.py). A client written for upstream sends GET /models?reload=1, gets 200 with a normal listing, and concludes the reload happened. Nothing reloaded: the file on disk is right, the model still starts with the old arguments, and only a router restart fixes it.
Measured on 0.4.7 (build 11860), Windows 10, one model, all unloaded:
status.args of model X -> --n-cpu-moe 14
edit the preset to n-cpu-moe = 15
GET /models?reload=1 -> 200, status.args still --n-cpu-moe 14
POST /models/reload -> 200 {"success": true}, status.args --n-cpu-moe 15
Same with a newly added section: invisible after the GET, present after the POST. preview-v0.4.8 behaves the same.
Affected client: LlamaForge — https://github.com/dadwritestech/LlamaForge/issues/37 (every Save, Scan, auto-tune round and preset apply goes through the query form there).
Related: test_router_models_status_args_preset keeps status.args/status.preset for exactly this kind of client compatibility (#117). This is the same class of problem, one layer up.
Possible Implementation
A field in router GET /props naming the reload endpoint would be enough; clients can fall back to today's behaviour when it is absent.
Minor, unrelated: the helper _get_model_ids(is_reload=...) in tests/unit/test_router.py still carries upstream's ?reload=1 branch and is called with is_reload=True at :511 and :523, where it now just lists models without reloading. Harmless, but the name claims something it no longer does.
- Langage dominant
- C++
- Étoiles
- 1.2k
- Forks
- 80
- Merge moyen
- 4 j 20 h
- PR mergées (30 j)
- 2
Préparer son environnement
Ce projet ne fournit ni conteneur de développement, ni Dockerfile, ni guide de contribution : l'installation est à votre charge. Commencez par son README, et consultez notre guide de la première contribution pour les étapes générales.
Par où commencer
- Lisez l'issue en entier, puis le guide de contribution du projet.
- Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
- Forkez le dépôt et travaillez sur une branche.
- Ouvrez une pull request qui référence le numéro de l'issue.
Autres issues de Anbeeld/beellama.cpp
-
[BUG] KVarN assertion failure in build_inp_mem_hybrid_iswa() due to uninitialized self_kvarn_rotOuverte
Difficulté 2/5 1-3 heures Accessibilité débutants 68/100
Anbeeld/beellama.cpp#205 ·
-
Difficulté 3/5 Une demi-journée Accessibilité débutants 58/100
Anbeeld/beellama.cpp#206 ·
-
Difficulté 3/5 1-2 jours Accessibilité débutants 57/100
Anbeeld/beellama.cpp#204 ·
-
Difficulté 4/5 3-5 jours Accessibilité débutants 45/100
Anbeeld/beellama.cpp#203 · 2 commentaires ·
-
Difficulté 4/5 3-5 jours Accessibilité débutants 45/100
Anbeeld/beellama.cpp#202 · 1 commentaire ·
Toutes les issues de Anbeeld/beellama.cpp
Issues similaires
-
Difficulté 2/5 1-3 heures Accessibilité débutants 72/100
CachyOS/packageinstaller#144 ·
Les mainteneurs répondent en général sous 1 jour
-
documentation
Difficulté 1/5 Moins d'une heure Accessibilité débutants 82/100
TomerAberbach/profiler-md#951 ·
Les mainteneurs répondent en général sous 1 jour
-
[Bug] Default key routing (JavaStringHash) picks a different partition than the Java client for about half of keysPeut-être pris @0z5a l’a pris aujourd’hui. Ouverte
Difficulté 2/5 1-3 heures Accessibilité débutants 82/100
apache/pulsar-client-go#1575 ·
Les mainteneurs répondent en général sous 2 jours
-
Difficulté 2/5 1-3 heures Accessibilité débutants 72/100
pocoproject/poco#5764 ·
Les mainteneurs répondent en général sous 2 jours
-
bug cpp-core-guidelines low-priority
Difficulté 2/5 1-3 heures Accessibilité débutants 62/100
NatronGitHub/openfx-supportext#164 ·
Les mainteneurs répondent en général sous 1 jour