good first issue
仓库指标
- Star
- (28,442 star)
- PR 合并指标
- (平均合并 2天 1小时) (30 天内合并 1,000 个 PR)
描述
Checklist
- 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
- 2. Please use English, otherwise it will be closed.
Motivation
Currently, when eagle is in use, the running batch size is constrained by the draft_extend phase because low-latency dispatch constrains the number of dispatched tokens but two-batch overlap (TBO) is not supported in the draft_extend phase. Enabling TBO can double the running batch size.
Related resources
No response