higress-group/higress
ai-load-balancer 考虑支持 SGLang 吗？ || ai-load-balancer Consider supporting SGLang?
オープン
#3,392 opened on 2026/01/26
help wanted
Repository metrics
- Stars
- (9,036 個のスター)
- PR merge metrics
- (平均マージ 7d 9h) (30d で 18 merged PRs)
説明
Why do you need it?
目前看支持了 vLLM，想问下后续有考虑支持 SGLang、Triton TensorRT-LLM 等其它推理引擎吗？ https://github.com/alibaba/higress/tree/main/plugins/wasm-go/extensions/ai-load-balancer/endpoint_metrics/backend/vllm
How could it be?
目前项目同时使用 vLLM 和 SGLang，若监控指标负载均衡能同时兼容两者，会更加便捷一些。
Why do you need it?
At present, vLLM is supported. I would like to ask if you will consider supporting other inference engines such as SGLang and Triton TensorRT-LLM in the future? https://github.com/alibaba/higress/tree/main/plugins/wasm-go/extensions/ai-load-balancer/endpoint_metrics/backend/vllm
How could it be?
The current project uses both vLLM and SGLang. It would be more convenient if the monitoring indicator load balancing could be compatible with both.