huggingface/accelerate

[Community Contributions] examples on distributed inference using 馃 Accelerate

Open

#3,078 opened on Sep 4, 2024

View on GitHub
聽(4 comments)聽(2 reactions)聽(0 assignees)Python聽(5,805 stars)聽(626 forks)batch import
contributions-welcomegood first issuewip

Description

The inference/distributed directory houses examples on running distributed inference with accelerate:

  • Phi2 for language generation
  • Stable Diffusion for image generation

The strategy followed there is to load an entire model onto each GPU and sending chunks of a batch through each GPU鈥檚 model copy at a time. Synthetic data generation has become an essential toolkit for every ML Engineer. So, it'd be beneficial to extend these examples to include some more use cases:

  • Image captioning
  • Speech data generation

Some nice to haves:

  • Include artifact serialization as done in this
  • Keep the artifact serialization code under a thread to not block GPU execution

How can you help?

You could help us contribute an example on any of the above-mentioned use cases or you can come up with your own 馃 Help us make the art of synthetic data generation scalable, easy, and accessible.

Contributor guide

Tech stack
pythonpytorch
Domain
machine learningai
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Basic PyTorchFamiliarity with Accelerate
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
60
Research direction
The existing examples are in the `inference/distributed` directory. The issue suggests adding examples for image captioning and speech data generation. A contributor should look at the existing Phi2 and Stable Diffusion examples to understand the pattern, then implement a new example following that pattern, optionally including artifact serialization as shown in the linked gist.