higress-group/higress
ai-load-balancer 考虑支持 SGLang 吗？ || ai-load-balancer Consider supporting SGLang?
开放
#3,392 创建于 2026年1月26日
help wanted
仓库指标
- 星标
- (9,036 个星标)
- PR 合并指标
- (平均合并 7天 9小时) (30 天内合并 18 个 PR)
描述
Why do you need it?
目前看支持了 vLLM，想问下后续有考虑支持 SGLang、Triton TensorRT-LLM 等其它推理引擎吗？ https://github.com/alibaba/higress/tree/main/plugins/wasm-go/extensions/ai-load-balancer/endpoint_metrics/backend/vllm
How could it be?
目前项目同时使用 vLLM 和 SGLang，若监控指标负载均衡能同时兼容两者，会更加便捷一些。
Why do you need it?
At present, vLLM is supported. I would like to ask if you will consider supporting other inference engines such as SGLang and Triton TensorRT-LLM in the future? https://github.com/alibaba/higress/tree/main/plugins/wasm-go/extensions/ai-load-balancer/endpoint_metrics/backend/vllm
How could it be?
The current project uses both vLLM and SGLang. It would be more convenient if the monitoring indicator load balancing could be compatible with both.