accelerate autocast on mps device

Open
  • Is something wrong?
#1,396 9 comments 1 reaction 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Stale
Tech stack
python, pytorch
Domain
backend, machine-learning

Research direction

The reported entry point is Accelerator.autocast(), with its current implementation shown in the issue; begin by tracing how it selects torch autocast contexts for fp16 and bf16. Check the examples/ folder and a representative task to verify the MPS path, and consider the work done when mixed-precision autocast runs correctly on the reported M2 macOS environment.

Written by the indexing model from the issue text.

Description

feature request
System Info
accelerate==0.18.0
system==M2 macos 13.3.1
Information
  • The official example scripts
  • My own modified scripts
Tasks
  • One of the scripts in the examples/ folder of Accelerate or an officially supported no_trainer script in the examples folder of the transformers repo (such as run_no_trainer_glue.py)
  • My own task or dataset (give details below)
Reproduction

Down below is the autocast implement. The default for fp16 is cuda, I wonder is it possible to adapt on macos mps device. Thanks very much.

    def autocast(self):
        """
        Will apply automatic mixed-precision inside the block inside this context manager, if it is enabled. Nothing
        different will happen otherwise.

        Example:

        ```python
        >>> from accelerate import Accelerator

        >>> accelerator = Accelerator(mixed_precision="fp16")
        >>> with accelerator.autocast():
        ...     train()
        ```
        """
        if self.native_amp:
            if self.mixed_precision == "fp16" and is_torch_version(">=", "1.10"):
                autocast_context = torch.cuda.amp.autocast(dtype=torch.float16)
            elif self.mixed_precision == "bf16":
                if self.distributed_type in [DistributedType.NO, DistributedType.MULTI_CPU, DistributedType.MULTI_GPU]:
                    autocast_context = torch.autocast(dtype=torch.bfloat16, device_type=self.device.type)
            else:
                autocast_context = torch.cuda.amp.autocast()

            autocast_context.__enter__()
            yield
            autocast_context.__exit__(*sys.exc_info())
        else:
            yield
Expected behavior

Omission

Dominant language
Python
Stars
9.9k
Forks
1.6k
Avg merge
20h 23m
Merged PRs (30d)
12

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from huggingface/accelerate

All issues in huggingface/accelerate

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.