Plan to support FSDP2?

Maintainers usually reply within 1 day

Assessment

This issue has not been assessed yet.

Description

Getting set up

First steps

1. Read the whole issue, then the project's contributing guide.
2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
3. Fork the repository and make your change on a branch.
4. Open a pull request that references the issue number.

More from huggingface/accelerate

• FSDP2: prepare() keeps the whole unsharded model alive during sharding when cpu_ram_efficient_loading=False

  Possibly taken A pull request linked to this issue is open or already merged. Open

  ⋯Report a problem with this issue

  • Is something wrong?

  • Already taken
  • Not beginner-friendly
  • Stale or invalid
  • Something else…
  • Send Cancel

  Difficulty 2/5 1-3 hours Newbie friendliness 78/100

  huggingface/accelerate#4511 · Opened Nov 8, 2026

  Maintainers usually reply within 1 day

• BatchSamplerShard padding mutates retained sampler batches and changes original loader samples

  Possibly taken @davrens101 claimed this 4 days ago. Open

  ⋯Report a problem with this issue

  • Is something wrong?

  • Already taken
  • Not beginner-friendly
  • Stale or invalid
  • Something else…
  • Send Cancel

  Difficulty 2/5 1-3 hours Newbie friendliness 75/100

  huggingface/accelerate#4505 · Opened Nov 6, 2026

  Maintainers usually reply within 1 day

• accelerator.reduce(tensor, "mean") fails on integer tensors when more than one process is used

  Possibly taken @MaiAditya claimed this 12 days ago. Open

  ⋯Report a problem with this issue

  • Is something wrong?

  • Already taken
  • Not beginner-friendly
  • Stale or invalid
  • Something else…
  • Send Cancel

  Difficulty 2/5 1-3 hours Newbie friendliness 78/100

  huggingface/accelerate#4469 · Opened Oct 29, 2026

  Maintainers usually reply within 1 day

• merge_fsdp_weights with remove_checkpoint_dir=True deletes the merged model when output_path is inside checkpoint_dir

  Possibly taken @assass1nlike claimed this 1 day ago. Open

  ⋯Report a problem with this issue

  • Is something wrong?

  • Already taken
  • Not beginner-friendly
  • Stale or invalid
  • Something else…
  • Send Cancel

  Difficulty 2/5 1-3 hours Newbie friendliness 25/100

  huggingface/accelerate#4514 · Opened Nov 9, 2026

  Maintainers usually reply within 1 day

• FSDP2 QLoRA: with `bnb_4bit_quant_storage=bf16` (the storage the #3987 warning recommends), the all-gather alters packed 4-bit weights at world size ≥ 2

  Open

  ⋯Report a problem with this issue

  • Is something wrong?

  • Already taken
  • Not beginner-friendly
  • Stale or invalid
  • Something else…
  • Send Cancel

  Difficulty 4/5 3-5 days Newbie friendliness 42/100

  huggingface/accelerate#4510 · Opened Nov 8, 2026

  Maintainers usually reply within 1 day

All issues in huggingface/accelerate

Similar issues

• Admin Memory Inspector shows "truncated" when the graph has exactly node_limit nodes

  Open

  ⋯Report a problem with this issue

  • Is something wrong?

  • Already taken
  • Not beginner-friendly
  • Stale or invalid
  • Something else…
  • Send Cancel

  Difficulty 2/5 1-3 hours Newbie friendliness 77/100

  Significant-Gravitas/AutoGPT#16733 · Opened Nov 11, 2026

  Maintainers usually reply within 1 day

• New Notion API Changelog Entry: November 10, 2026

  Open

  ⋯Report a problem with this issue

  • Is something wrong?

  • Already taken
  • Not beginner-friendly
  • Stale or invalid
  • Something else…
  • Send Cancel
  changelog investigate

  Difficulty 1/5 Under an hour Newbie friendliness 74/100

  ramnes/notion-sdk-py#444 · Opened Nov 11, 2026

• An ordered comparison on a JSON number is unknown: make the rule prominent for pack authors

  Open

  ⋯Report a problem with this issue

  • Is something wrong?

  • Already taken
  • Not beginner-friendly
  • Stale or invalid
  • Something else…
  • Send Cancel
  documentation

  Difficulty 2/5 1-3 hours Newbie friendliness 69/100

  Judgment-Pack/judgment-pack-spec#299 · Opened Nov 10, 2026

  Maintainers usually reply within 1 day

• Fix typed Python dividend-history example

  Open

  ⋯Report a problem with this issue

  • Is something wrong?

  • Already taken
  • Not beginner-friendly
  • Stale or invalid
  • Something else…
  • Send Cancel

  Difficulty 1/5 Under an hour Newbie friendliness 90/100

  QuantConnect/Documentation#2858 · Opened Nov 10, 2026

  Maintainers usually reply within 1 day

• script.toml's test command misses the type refusals of op_int and read_op_code

  Open

  ⋯Report a problem with this issue

  • Is something wrong?

  • Already taken
  • Not beginner-friendly
  • Stale or invalid
  • Something else…
  • Send Cancel
  good first issue

  Difficulty 2/5 1-3 hours Newbie friendliness 90/100

  btclib-org/btclib#3047 · Opened Nov 10, 2026

  Maintainers usually reply within 1 day

More Python issues