Multi-modalcigood first issueperformance
倉庫指標
- Star
- (28,442 star)
- PR 合併指標
- (平均合併 2天 1小時) (30 天內合併 1,000 個 PR)
描述
Checklist
- 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
- 2. Please use English, otherwise it will be closed.
Motivation
- All existing Multi-modal CI operations are executed on a single GPU, without considering the scenario of Tensor Parallelism (TP). There is a requirement to introduce test cases for TP-2/4 configurations.
- The payload for VLM CI is relatively low. Stress tests are necessary, and simultaneously, it is crucial to investigate whether there are any memory leaks.
- At present, the majority of the VLM CI is only to vision. It is essential to expand its scope to audio.
- Welcome to add more.
Related resources
The test is under test/srt/test_vision_openai_server_x
Related PRs: