Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

Expert Parallelism for MoEs

Abierto
  • ¿Algo no está bien?
#165 1 comentario 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
8/100
Tipo de issue
Nueva funcionalidad
Claridad
Bastante claro
Estado de actividad
Estancado
Stack tecnológico
python
Área
distributed-systems, machine-learning

Línea de trabajo

Start with the existing MoE layer and router code in Fast-LLM, then read the EPLB repo to see what eplb.rebalance_experts expects as input. The issue calls for a 2-layer, 12-expert test with 4 redundant experts across 8 GPUs, with expert load checked for balance. The design is open and the issue is nearly two years old with no assignee, so confirm scope with maintainers first.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

enhancement need update

🎯 Goal (What & Why)

Integrate Expert Parallelism (EP) into Fast-LLM's MoE architecture.

Motivations:

  • Support Larger MoE Models: Enable training and inference for bigger models than Mixtral-8x7B, exceeding single-GPU memory constraints.
  • Improve GPU Load Balancing: Use DeepSeek's EPLB (eplb.rebalance_experts) to dynamically balance GPU loads via redundant expert replication and heuristic packing, see https://github.com/deepseek-ai/EPLB/tree/main.
  • Reduce Communication Overhead: Initially apply global load balancing, and eventually implement hierarchical load balancing to colocate expert groups.

🚀 Execution Plan

Step 1: Minimal Viable Product (MVP)

  1. Integrate eplb.rebalance_experts with Fast-LLM:

    • Add support for expert parallelism only in MoE layers.
    • Implement global load balancing for distributing expert replicas across GPUs.
    • Provide a minimal configuration interface, reusing existing MoE router logic in Fast-LLM.

  2. Validation & Testing:

    • Test with a small MoE, say 2-layers containing 12 experts per layer, using 4 redundant experts and distributing them across 8 GPUs.
    • Verify that the expert load is balanced and that performance is maintained during inference.

  3. Documentation:

    • Update Fast-LLM's documentation to include usage examples and configuration guidelines for enabling EP.
Step 2: Additional Optimizations (Post-MVP)
  • Support for Hierarchical Load Balancing: When the number of server nodes divides the number of expert groups.
  • Advanced Routing Strategies: Introduce DeepSeek's group-limited expert routing.
  • Performance Enhancements: Explore fused operations and other optimizations for speed.

📌 Acceptance Criteria (Must-Haves for Completion)

  • EP must be functional and tested in Fast-LLM's MoE layers.
  • The implementation must be documented clearly, with examples and edge cases covered.
  • The PR must include a performance/impact summary, demonstrating memory and compute efficiency gains.
  • No refactors unless directly necessary for feature completion.

🛠️ Project Management

  • Assign the project to the Fast-LLM project.
  • Set the Estimate field (in days) in the GitHub project.
  • Use the Size field to categorize the PR size (Small/Medium/Large).
  • Assign an owner when opening the issue.
Lenguaje dominante
Python
Estrellas
337
Forks
52
Sin PR fusionados en 30 d

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de ServiceNow/Fast-LLM

Todos los issues de ServiceNow/Fast-LLM

Issues similares

Más issues de Python

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.