Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

DeepSpeed - Q4 2026 Roadmap

Aperta
  • Qualcosa non va?
#8,672 14 commenti 3 reazioni 5 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 3 giorni

@delock ci sta già lavorando.

Dal 26/9/2026.

Valutazione

Questa issue non è ancora stata valutata.

Descrizione

roadmap

This is the roadmap for DeepSpeed Q4 2026. Feedback is welcome — please leave comments on this issue or join the #2026q4-roadmap channel on the DeepSpeed Slack.

Some of the items are the extension of the Q3 roadmap: https://github.com/deepspeedai/DeepSpeed/issues/8104

Tagging all the PoCs (code maintainers) for each workstream, the PoCs can coordinate developer meetings, sort out implementation plan, and are responsible for updating the status.

New feature and enhancement

Mixture-of-Expert (MoE) Training Scaling and Kernel Development

Mixture-of-Expert (MoE) is the mainstream model architecture for frontier AI models. Scaling MoE training requires further improvement on the MoE backend of DeepSpeed, enabling better efficiency, scalability and performance.

  • Pipelined Communication & Compute (More Efficient All-to-All Communications and Reduce HBM activation spikes) (PoC: @PKUWZP )
  • Block-Sparse Kernels for efficient token dropping (PoC: @PKUWZP )
  • Hardware-Aware Memory Optimization & Model Sharding (PoC: @PKUWZP )
  • Expert Loading Balance & Hotspot Mitigations (PoC: @PKUWZP )
  • Resiliency & Silence Data Corruption Detection (PoC: @PKUWZP )
  • AutoEP Performance Optimization & Parallel Folding (AutoEP + AutoTP) (PoC: @tohtana @pengdurice )
  • MoE gradient norm and clipping (PoC @pengdurice )
  • FP8 GEMM for MoE (PoC @pengdurice )
  • Improvements of preset and configuration definitions for better coverage of MoE variants (PoC @pengdurice )

DeepCompile

  • Automatic optimization pass orchestration (@tohtana @sfc-gh-truwase @pengdurice )
  • Expert parallel support (@tohtana @pengdurice )
  • AutoSP with ZeRO (@tohtana )
  • Model coverage improvements including VL/VLA models (@tohtana @sfc-gh-truwase , optionally @PKUWZP )
  • Save/load optimized IR graphs (@tohtana )
  • Robustness improvements (@tohtana )

New Accelerators Support (Q4)

  • DeepSpeed TPU accelerators support and performance optimization (@PKUWZP @tohtana @sfc-gh-truwase )
  • Apple Silicon Accelerators support for DeepSpeed (@PKUWZP )
  • Performance Optimization on Emerging AI Accelerators (@PKUWZP @delock )

DeepSpeed HybridEngine Optimization (Q4)

  • Achieving on-par performance with small batch size compared to mainstream LLM Inference Engines (@delock @PKUWZP )
  • Continuous Batching Rollout support (@delock )
  • Kernel Injection for co-located rollout support (@delock )

Enable training of large scale models

  • Enable autoEP, autoTP, Ulysses under Z3 to unblock large scale model training ( POC @pengdurice @tohtana )
  • Test and verify support of SOTA models (model specific kernels, attention implementations can go to DeepSpeedExamples) ( POC @pengdurice )
  • Muon optimizer offload( POC @delock @pengdurice )

On-Policy Distillation (OPD) Trainer

  • Systems Optimization for Large model OPD across multi-accelerators. (@delock @minjiazhang @PKUWZP )
  • Coupled with HybridEngine Optimization for end-to-end benchmarking (@delock @minjiazhang @PKUWZP )
  • Documentation for OPD trainer onboarding (@delock @PKUWZP )

Code quality and health (Optional)

  • Continue deprecating legacy codes (PoC @sfc-gh-truwase )
  • Refactor the current code base, i.e. unified process group placement, unified optimizer initialization, reduce usages of globals, fix inconsistent code and doc descriptions. (PoC @sfc-gh-truwase @pengdurice )

AI Agent-driven Distributed Training Optimization (Optional)

  • DeepSpeed-native Agentic training optimization (TBD, tentatively @PKUWZP )

Agent Sandbox enablement for Agentic post-training capabilities (Optional)

  • Implement Agent Sandbox Environment for DeepSpeed to enable Agentic Post-training (@PKUWZP )

Universal Checkpointing (UCP) feature extension (Optional)

  • Affine mapping as Semantic free UCP info (Phase 1) (@delock )
  • UCP info support Yuan model TP sharding. (@delock )
Lingua principale
Python
Stelle
43.3k
Fork
5.1k
Merge medio
4g 21h
PR unite (30g)
64

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di deepspeedai/DeepSpeed

Tutte le issue di deepspeedai/DeepSpeed

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.