Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

load_pretrained_model deep-copies the whole state dict, doubling peak load memory

Chiusa Adatta ai principianti
  • Qualcosa non va?
#3,728 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
88/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
python, pytorch
Ambito
machine-learning

Direzione di ricerca

Inizia da funasr/train_utils/load_pretrained_model.py e segui il percorso di caricamento del checkpoint di AutoModel attorno a torch.load e alla deepcopy ridondante. Rimuovi la seconda copia non necessaria dello state-dict, quindi esegui la riproduzione con il checkpoint grande indicato e conferma che il caricamento continui a segnalare che tutte le chiavi corrispondono, mentre l'RSS di picco non include più una seconda copia delle dimensioni del checkpoint.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug needs triage

🐛 Bug

load_pretrained_model() deep-copies the entire checkpoint state dict on every model load. The copy is redundant, and it holds a second full copy of the weights in memory for the duration of the load, so peak host memory is roughly doubled by the checkpoint size.

Loading iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online (220M params, 956 tensors, 840MB model.pt) peaks at 3461 MB RSS. With the copy removed the same load peaks at 2624 MB — a 837MB difference that matches the checkpoint size.

The practical consequence is that a model which fits in memory can still fail to load, and container memory limits have to be set to twice the checkpoint size.

To Reproduce

  1. Install with: pip install funasr modelscope kaldi-native-fbank
  2. Run: load any large checkpoint through AutoModel and sample peak RSS
  3. See: no exception on a roomy host — the symptom is peak RSS; on a constrained host it is an OOM kill
pip install funasr==1.4.16 modelscope kaldi-native-fbank

python - <<'PY'
import resource, time
from funasr import AutoModel

t0 = time.time()
AutoModel(
    model="iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online",
    device="cpu",
    disable_update=True,
)
peak = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1024
print(f"{time.time() - t0:.1f}s   peak RSS {peak:.0f} MB")
PY

Code sample

funasr/train_utils/load_pretrained_model.py:

ori_state = torch.load(path, map_location=map_location)   # fresh, local, never read again

src_state = copy.deepcopy(ori_state)                     # every tensor copied a second time
src_state = src_state["state_dict"] if "state_dict" in src_state else src_state
src_state = src_state["model_state_dict"] if "model_state_dict" in src_state else src_state
src_state = src_state["model"] if "model" in src_state else src_state

The copy is not needed:

  • ori_state is produced inside the function by torch.load, so it has a single owner and is not shared with the caller.
  • ori_state is not referenced again after the deepcopy line.
  • src_state is only read from that point on. The loop below reads src_state[k_src].shape and rebinds references in dst_state (dst_state[k] = src_state[k_src]); the data actually reaches the model through obj.load_state_dict(dst_state, strict=True).
  • No tensor is mutated in place anywhere on this path, so the deep-copied version and the shared version are equivalent.

Expected behavior

Loading a checkpoint should not require memory for two copies of it. A 220M-parameter model should not need ~2x its own weight size in headroom.

Error logs

No exception. Measured with resource.getrusage(RUSAGE_SELF).ru_maxrss, same machine, same checkpoint, back to back:

funasr 1.4.16              : LOAD 13.3s   peak RSS 3461 MB   <All keys matched successfully>
same, deep copy removed    : LOAD 26.9s   peak RSS 2624 MB   <All keys matched successfully>

The 837MB difference tracks the 840MB checkpoint, which is the copy being dropped. The wall-clock column is noisy and is not part of the report; the memory difference is the reproducible result. All keys matched successfully on both sides is the correctness check.

Environment

  • OS: Linux 6.6.87.2-microsoft-standard-WSL2 (Ubuntu userspace)
  • Python version: 3.12.14
  • FunASR version: 1.4.16 (latest release; main at 41778c4 is identical here)
  • ModelScope version: 1.40.1
  • PyTorch version: 2.14.0+cu126
  • Install method: pip
  • Device: cpu for the load measurement
  • GPU model: NVIDIA GeForce RTX 4060 Laptop GPU
  • CUDA version: 12.6

Audio details

Not audio-related. The model used for the measurement is iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online (220M params, 956 tensors, 840MB model.pt). The overhead scales with checkpoint size, so it should reproduce with any large FunASR checkpoint, including the LLM-ASR models.

Lingua principale
Python
Stelle
20.4k
Fork
2k
Merge medio
9h 48m
PR unite (30g)
168

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di modelscope/FunASR

Tutte le issue di modelscope/FunASR

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.