sgl-project/sglang

[Feature] request for "classify" method for sglang router

已關閉

#15,240 建立於 2025年12月16日

 (10 則留言) (0 個反應) (0 位負責人)Python (6,216 個分叉)auto 404
good first issue

倉庫指標

星標
 (28,442 顆星)
PR 合併指標
 (平均合併 2天 1小時) (30 天內合併 1,000 個 PR)

描述

Checklist

Motivation

In RL, we may launch a router to dispatch reward computation requests for both generative rewards (via /generate) and discriminative rewards (via /classify). I note that the "classify" api has not been implemented in sglang router (https://docs.sglang.io/advanced_features/router.html#api-surface). Hope future release will include this feat.

In addition to this, I want to report a bug that in the latest sglang router, the router fails to handle "v1/embeddings" requests after the servers sleep and then wake up.

# step 1: Launch multiple workers and one router
# step 2: Sleep servers and then wake up them
# step 3: Post "v1/embeddings" requests to router

For sglang-router==0.2.4 curl http://{router_address}/v1/models {"object":"list","data":[{"id":"unknown","object":"model","owned_by":"local"}]} and the reward computation also fails

For sglang-router==0.2.2 all behaviors are correct

@slin1237 @zhaochenyang20

Related resources

No response

貢獻者指南