[Feature] Profile the update weights from disk API of SGLang Diffusion · sgl-project/sglang

(3 comentários) (0 reação) (0 responsável)Python (6.216 forks)auto 404

good first issue

Métricas do repositório

Stars: (28.442 estrelas)
Métricas de merge de PR: (Mesclagem média 1d 13h) (1.000 fundiu PRs em 30d)

Description

Checklist

If this is not a feature request but a general question, please start a discussion at https://github.com/sgl-project/sglang/discussions. Otherwise, it will be closed.
Please use English. Otherwise, it will be closed.

Motivation

As we did in this comment:

https://github.com/sgl-project/sglang/pull/18306/#issuecomment-3898841774

We should profile the actual time breakdown in the update weights from disk.

Ideally speaking, 7B models' update should be within 1s (no considering save to disk time) in this https://github.com/zhaochenyang20/Awesome-ML-SYS-Tutorial/blob/main/sglang/latency-accelerate-for-weight-updates/readme.md

Related resources

No response

Guia do colaborador

Direção de pesquisa: Analise a função de atualização de pesos do disco medindo a divisão do tempo de leitura, análise e carregamento dos pesos. Compare com os benchmarks de latência esperados.
Pilha de tecnologia: python
Domain: backend
Tipo Issue: Funcionalidade
Difficulty: 2
Tempo estimado: Meio dia
Status da atividade: Ativo
Clarity: Claro
Prerequisites: Python
Simpatia para novatos: 60