[Question] GLM5----MTP training support
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 30/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Quiet
- Tech stack
- python
- Domain
- distributed-systems, machine-learning
Research direction
Start with scripts/models/glm5-744B-A40B.sh at the referenced line, then trace how MTP training and allgather_cp are configured for GLM-5. No test or implementation entry point is identified; done would require a clarified scope and a maintainer-approved support plan or implementation.
Written by the indexing model from the issue text.
Description
Your Question
Any plan to support training the MTP layers jointly with the main model for GLM-5?
MTP training is incompatible with allgather_cp
https://github.com/THUDM/slime/blob/f65c6e8191044cc71291a1a87431f752420ffe91/scripts/models/glm5-744B-A40B.sh#L54
What I've Tried
Environment (if relevant)
- slime version:
- Python version:
- PyTorch version:
- CUDA/ROCm version:
- GPU type and count:
- OS:
Additional Context
No response
Pre-submission Checklist
- I have read the CONTRIBUTING.md and understand the collaboration scope.
- I have read the documentation and FAQ and my question is not answered there.
- I have searched for existing issues and my question has not been asked before.
- Dominant language
- Python
- Stars
- 8.8k
- Forks
- 1.3k
- Avg merge
- 20h 44m
- Merged PRs (30d)
- 25
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from THUDM/slime
-
[Bug] Math reward accepts distinct large integers after float normalizationPossibly taken @lll-hhh claimed this 5 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
gpqa: when no option letter can be extracted, a single-letter gold label is substring-matched against the whole reply, so a reply that never answers can score 1.0Possibly taken @shaurya416 claimed this 41 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Maintainers usually reply within 1 day
-
[Bug] shipped dynamic-sampling filters crash on fan-out groups (list[list[Sample]]) from multi-turn agent rolloutsPossibly taken @QIU-Guanzong claimed this 52 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Maintainers usually reply within 1 day
-
[Bug] W&B initialization fails with wandb 0.28.2 because wandb.util.generate_id was removedPossibly taken @cyyecao-lappland claimed this 70 days ago. Openbug
Difficulty 1/5 1-3 hours Newbie friendliness 90/100
THUDM/slime#2332 · 1 comment ·
Maintainers usually reply within 1 day
-
[Bug] solution_str = sample.prompt + sample.responsePossibly taken @mvanhorn claimed this 135 days ago. Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
THUDM/slime#1829 · 1 comment ·
Maintainers usually reply within 1 day
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
huggingface/deep-rl-class#723 ·
-
[tool] bug: Qwen3 XML tool parser raises on `...` argument values (Ellipsis is not JSON serializable)Possibly taken @super-yuki claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 87/100
verl-project/verl#8559 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
TencentCloud/octop-harness#228 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
google/osv-scalibr#2585 ·
Maintainers usually reply within 2 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
NousResearch/hermes-agent#149323 ·
Maintainers usually reply within 1 day