Feature Request: Router: upstream clients' GET /models?reload=1 is silently inert, and a client can't detect which reload route exists

未关闭
#185 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

@dannyward630 已经在做这个了。

开始于 2026年10月29日。

  • #193 来自 @dannyward630 —— 未关闭

评估

难度
3/5
预计耗时
1-2 天
新手友好度
25/100
Issue 类型
功能
描述清晰度
基本清楚
活跃度
停滞
技术栈
cpp, python
领域
api, backend

调研方向

Start with the router's GET /props and GET /models handlers in the server sources, and find where reload=1 is read and where POST /models/reload is registered. The helper _get_model_ids(is_reload=...) in tests/unit/test_router.py (called around lines 511 and 523) and the note near lines 408-412 show the current behaviour. Work is already under way in open PR #193, so check it before starting. Done means a router props field names the reload endpoint, or the README documents the difference, and the router tests pass.

由索引模型根据 Issue 内容生成。

描述

enhancement
Prerequisites
  • I am running the latest code. Mention the version if possible as well.
  • I carefully followed the README.md.
  • I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
  • I reviewed the Discussions, and have a new and useful enhancement to share.
Feature Description

Please give clients a way to tell which request reloads the router preset. In order of preference:

  1. Advertise the capability in GET /props (router props), e.g. a list of supported router endpoints or a flag like "preset_reload": "POST /models/reload". Then a client picks the right request once and never guesses.
  2. At minimum, a line in the README next to POST /models/reload saying upstream does this with GET /models?reload=1 and that the query form is intentionally inert here.

I am not asking you to make GET mutate state. I see test_router.py:408-412 and the README line stating that GET /models is read-only, and the reasoning is sound.

Motivation

BeeLlama's router reloads the preset on POST /models/reload. Upstream llama.cpp reloads it on GET /models?reload=1 and has no POST route (checked across all of tools/server at the commit BeeLlama branches from, 465e49b9, and at current master: models/reload appears only in a stale docstring in tests/unit/test_router.py). A client written for upstream sends GET /models?reload=1, gets 200 with a normal listing, and concludes the reload happened. Nothing reloaded: the file on disk is right, the model still starts with the old arguments, and only a router restart fixes it.

Measured on 0.4.7 (build 11860), Windows 10, one model, all unloaded:

status.args of model X     -> --n-cpu-moe 14
edit the preset to n-cpu-moe = 15
GET /models?reload=1       -> 200, status.args still --n-cpu-moe 14
POST /models/reload        -> 200 {"success": true}, status.args --n-cpu-moe 15

Same with a newly added section: invisible after the GET, present after the POST. preview-v0.4.8 behaves the same.

Affected client: LlamaForge — https://github.com/dadwritestech/LlamaForge/issues/37 (every Save, Scan, auto-tune round and preset apply goes through the query form there).

Related: test_router_models_status_args_preset keeps status.args/status.preset for exactly this kind of client compatibility (#117). This is the same class of problem, one layer up.

Possible Implementation

A field in router GET /props naming the reload endpoint would be enough; clients can fall back to today's behaviour when it is absent.

Minor, unrelated: the helper _get_model_ids(is_reload=...) in tests/unit/test_router.py still carries upstream's ?reload=1 branch and is called with is_reload=True at :511 and :523, where it now just lists models without reloading. Harmless, but the name claims something it no longer does.

主要语言
C++
星标
1.2k
派生
80
平均合并
4 天 20 小时
30 天内合并 PR
2

环境准备

这个项目没有提供开发容器、Dockerfile 或贡献指南,环境需要你自己搭建:先看它的 README,通用步骤见我们的新手贡献指南。

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

Anbeeld/beellama.cpp 的其他 Issue

查看 Anbeeld/beellama.cpp 的全部 Issue

相似的 Issue

更多 C++ Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。