verl 0.7.x embedded vLLM server cannot load custom tool parser plugins (tool_parser_plugin import step skipped)
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 68/100
Research direction
Start with agentlightning/verl/vllm_compat.py and the embedded vLLM server entry point, vLLMHttpServer.run_server, using the compatibility work from #590 as context. Verify the custom tool-parser plugin is imported before server initialization when configured, while missing or unset plugin attributes remain a no-op; confirm the embedded server can start with the documented custom-parser configuration.
Written by the indexing model from the issue text.
Description
Summary
On the verl 0.7.x/0.8.x line that Agent Lightning pins, a custom vLLM tool-call parser plugin (--tool-parser-plugin / config tool_parser_plugin) can never be used with the embedded vLLM OpenAI server: verl's custom server entrypoint skips the plugin-import step that vllm serve performs, so the server dies at startup with:
ValueError: --enable-auto-tool-choice requires tool_parser:'<name>' which has not been registered
This matters for agent RL on models whose tool-call format has no built-in vLLM parser (in our case a MiniCPM-family model) — exactly the population Agent Lightning serves.
Root cause
vllm serve's entrypoint (vllm/entrypoints/openai/api_server.py,run_server) imports the plugin before building the app:if args.tool_parser_plugin: ToolParserManager.import_tool_parser(args.tool_parser_plugin).- verl 0.7.x's
vLLMHttpServer.run_server(https://github.com/volcengine/verl/blob/v0.7.1/verl/workers/rollout/vllm_rollout/vllm_async_server.py#L441) builds the engine and app itself and never performs that import, while still honoring--tool-call-parservalidation ininit_app_state— so the parser name from the plugin cannot resolve. - verl upstream is not a viable venue:
mainremoved server-side tool-parsing args entirely (tool parsing is expected client-side in AgentLoop; see volcengine/verl#6560 rejected and volcengine/verl#6844 closed), and 0.7.x is a frozen release line.
Fix we run in production
A 4-line patch at the top of verl's run_server (applied to site-packages):
if getattr(args, "tool_parser_plugin", None):
from vllm.entrypoints.openai.tool_parsers import ToolParserManager
ToolParserManager.import_tool_parser(args.tool_parser_plugin)
With this, actor_rollout_ref.rollout.multi_turn.tool_config-style configs with a custom parser plugin work end-to-end (running in our GRPO training since 09-08).
Proposal
This fits the same pattern as #589 / PR #590: a runtime shim in agentlightning/verl/vllm_compat.py that wraps vLLMHttpServer.run_server to perform the plugin import before delegating (strict no-op when no plugin is configured or the attribute doesn't exist). I'm happy to submit that PR — it builds on the compat module introduced in #590, so I'd stack it once #590 has a verdict.
- Dominant language
- Python
- Stars
- 18.4k
- Forks
- 1.6k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 27
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from microsoft/agent-lightning
-
bug v0.x verl
Difficulty 1/5 Under an hour Newbie friendliness 68/100
microsoft/agent-lightning#492 ·
-
enhancement store v0.x
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
microsoft/agent-lightning#372 · 10 comments ·
-
Difficulty 3/5 1-2 days Newbie friendliness 72/100
microsoft/agent-lightning#593 · 6 comments ·
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
microsoft/agent-lightning#570 ·
-
Difficulty 5/5 Over a week Newbie friendliness 45/100
microsoft/agent-lightning#567 · 2 comments ·
All issues in microsoft/agent-lightning
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100