max_tokens and best_of reject a value equal to their configured limit
维护者通常 2 天内回复
评估
调研方向
Inspect the validation checks in openai_api_handler.cpp for max_tokens and best_of, and compare them with openai_completions.cpp and openai_responses.cpp. Locate the existing limit-validation tests, then verify that all four fields accept values equal to their configured limits. The issue is already covered by an open pull request, so confirm its scope and test coverage before starting.
由索引模型根据 Issue 内容生成。
描述
Describe the bug
docs/llm/reference.md describes max_tokens_limit and best_of_limit as the maximum value
accepted by the endpoint. The check for max_tokens and best_of rejects a request that sends
exactly that value, while the other token-limit fields accept it:
| field | check | accepts a value equal to the limit |
|---|---|---|
| max_tokens | !(value < limit) (openai_api_handler.cpp) |
no |
| best_of | !(value < limit) (openai_api_handler.cpp) |
no |
| max_completion_tokens | value > limit (openai_completions.cpp) |
yes |
| max_output_tokens | value > limit (openai_responses.cpp) |
yes |
With max_tokens_limit: 1000 and the default best_of_limit: 20, running the validation code with
a value equal to the limit gives:
max_tokens 1000 -> max_tokens exceeds limit provided in graph config: 1000
max_completion_tokens 1000 -> accepted
max_output_tokens 1000 -> accepted
best_of 20 -> best_of exceeds limit provided in graph config: 20
When max_tokens is omitted the server fills in the limit itself (1000), a value that an explicit
request cannot send.
No test sends a value equal to either limit.
To Reproduce
- Graph with
max_tokens_limit: 1000(or the defaultbest_of_limitof 20). - POST /v3/chat/completions with
"max_tokens": 1000, or"best_of": 20. - 400
... exceeds limit provided in graph config.
Expected behavior
A value equal to the configured limit is accepted, for all four fields.
Logs
None.
Configuration
- OVMS version: main @ eec7daac
- LLM servable with max_tokens_limit or best_of_limit
- N/A
- N/A
- N/A
Additional context
I have a patch with tests and will open a PR.
- 主要语言
- C++
- 星标
- 952
- 派生
- 285
- 平均合并
- 3 天 12 小时
- 30 天内合并 PR
- 47
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
openvinotoolkit/model_server 的其他 Issue
-
Rerank with an empty documents array or object documents fails with an internal tokenizer error可能已有人在做 @Daksha1611 于 3 天前认领。 未关闭
难度 4/5 1-2 天 新手友好度 24/100
openvinotoolkit/model_server#4671 ·
维护者通常 2 天内回复
-
Downloading an image from a URL has no timeout可能已有人在做 @Daksha1611 于 4 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 35/100
openvinotoolkit/model_server#4669 ·
维护者通常 2 天内回复
-
/v1/tokenize returns a flat list when the text array holds a single string可能已有人在做 @Daksha1611 于 5 天前认领。 未关闭
难度 3/5 1-2 天 新手友好度 25/100
openvinotoolkit/model_server#4664 ·
维护者通常 2 天内回复
-
/v3/audio/speech ignores response_format and always returns 32-bit float WAV可能已有人在做 @michalkulakowski 于 23 天前认领。 未关闭
难度 3/5 1-2 天 新手友好度 58/100
openvinotoolkit/model_server#4613 · 3 条评论 · 已指派 1 人 ·
维护者通常 2 天内回复
-
enhancement
难度 3/5 1-2 天 新手友好度 68/100
openvinotoolkit/model_server#4609 · 5 条评论 ·
维护者通常 2 天内回复
查看 openvinotoolkit/model_server 的全部 Issue