One example, multiple config files
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 52/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- python
- Domain
- machine-learning
Research direction
Start in the DeepSpeed-ZenFlow/finetuning example directory and inspect the existing training script, run.sh, and configuration file. Add the proposed z2_config.json, z3_config.json, z2_offload_config.json, and zenflow_config.json variants, then make run.sh accept a config-file input so each DeepSpeed mode can be launched from the same example.
Written by the indexing model from the issue text.
Description
I'm wondering if we can take the ZenFlow finetuning example, and extend this example into a test bed of different DeepSpeed technologies.
The ZenFlow finetuning example:
https://github.com/deepspeedai/DeepSpeedExamples/tree/master/training/DeepSpeed-ZenFlow/finetuning
The reason is that DeepSpeed is mostly config driven, which means with the same training script, it only takes a config file change to run DeepSpeed training in different mode.
How about put different config file in the directory, i.e. "z2_config.json", "z3_config.json", "z2_offload_config.json", "zenflow_config.json", etc. into the directory, then change the run.sh to make it take the config as input? This might be a good start point for anyone to quickstart deepspeed with a real example.
- Dominant language
- Python
- Stars
- 6.8k
- Forks
- 1.1k
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 1
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from deepspeedai/DeepSpeedExamples
-
Difficulty 2/5 1-3 hours Newbie friendliness 52/100
deepspeedai/DeepSpeedExamples#996 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
deepspeedai/DeepSpeedExamples#995 ·
-
moe example 404 Open
Difficulty 4/5 3-5 days Newbie friendliness 25/100
deepspeedai/DeepSpeedExamples#984 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 42/100
deepspeedai/DeepSpeedExamples#979 · 6 comments ·
-
Difficulty 4/5 3-5 days Newbie friendliness 38/100
deepspeedai/DeepSpeedExamples#969 · 1 comment ·
All issues in deepspeedai/DeepSpeedExamples
Similar issues
-
sponsored
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
Diaoul/subliminal#1382 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 92/100
-
triage/confirmed
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
agentscope-ai/agentscope#2775 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100