[feature request] A more flexible `free_memory` function

Open
  • Is something wrong?
#925 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Stale
Tech stack
python
Domain
machine-learning

Research direction

Start with src/accelerate/accelerator.py around lines 1788-1799 and read the existing free_memory implementation and its handling of Accelerator internals. Define the behavior needed for repeatedly preparing and unloading custom cached objects, then verify that the proposed interface supports the example loop without accessing internal objects individually.

Written by the indexing model from the issue text.

Description

enhancement feature request

After looking at the existing issues and PRs, I'm opening this feature request since I couldn't find my use case.
https://github.com/huggingface/accelerate/blob/7889ba6b6dd094e72fa222194146a40426fb3d5a/src/accelerate/accelerator.py#L1788-L1799

  1. In my case, I'm using a custom cache system instead of regular dataloaders and I need to load and unload the same objects multiple times in the Accelerator for each iteration to free up memory. For now, I need to access the different internal objects of the Accelerator class individually. This could happen in different non-standard training loop (e.g. reloading a model and corresponding optimizers, ...).
  2. It only needs a quick modification of the free_memory function. I'm not sure how you would implement that to be coherent with the whole library though.
  3. This feature could be used as followed:
 # load cache
 # prepare it with the Accelerator
 for iteration in range(nb_iterations):
   # take random batch from cache
   # Forward pass
   if iteration % reload_frequency == 0:
         # unload the cache from the Accelerator to free up memory
         # load new cache
         # prepare it with the Accelerator

Thank you for this library by the way. It makes things much easier for everyone!

Dominant language
Python
Stars
9.9k
Forks
1.6k
Avg merge
20h 23m
Merged PRs (30d)
12

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from huggingface/accelerate

All issues in huggingface/accelerate

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.