Hacktoberfest 2026: die Issues, die Maintainer für den Oktober markiert haben – offen und einsteigerfreundlich. Hacktoberfest-Issues durchsuchen

load_pretrained_model deep-copies the whole state dict, doubling peak load memory

Geschlossen Anfängerfreundlich
  • Stimmt etwas nicht?
#3,728 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
2/5
Geschätzter Aufwand
1-3 Stunden
Anfängerfreundlichkeit
88/100
Issue-Typ
Bug
Klarheit
Klar beschrieben
Aktivitätsstatus
Aktiv
Tech-Stack
python, pytorch
Bereich
machine-learning

Rechercherichtung

Beginne in funasr/train_utils/load_pretrained_model.py und verfolge den Checkpoint-Ladepfad von AutoModel rund um torch.load und die redundante deepcopy. Entferne die unnötige zweite State-Dict-Kopie. Führe anschließend die Reproduktion mit dem benannten großen Checkpoint aus und bestätige, dass beim Laden weiterhin gemeldet wird, dass alle Schlüssel übereinstimmen, während der maximale RSS keinen zweiten Checkpoint-großen Speicherbereich mehr enthält.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

bug needs triage

🐛 Bug

load_pretrained_model() deep-copies the entire checkpoint state dict on every model load. The copy is redundant, and it holds a second full copy of the weights in memory for the duration of the load, so peak host memory is roughly doubled by the checkpoint size.

Loading iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online (220M params, 956 tensors, 840MB model.pt) peaks at 3461 MB RSS. With the copy removed the same load peaks at 2624 MB — a 837MB difference that matches the checkpoint size.

The practical consequence is that a model which fits in memory can still fail to load, and container memory limits have to be set to twice the checkpoint size.

To Reproduce

  1. Install with: pip install funasr modelscope kaldi-native-fbank
  2. Run: load any large checkpoint through AutoModel and sample peak RSS
  3. See: no exception on a roomy host — the symptom is peak RSS; on a constrained host it is an OOM kill
pip install funasr==1.4.16 modelscope kaldi-native-fbank

python - <<'PY'
import resource, time
from funasr import AutoModel

t0 = time.time()
AutoModel(
    model="iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online",
    device="cpu",
    disable_update=True,
)
peak = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1024
print(f"{time.time() - t0:.1f}s   peak RSS {peak:.0f} MB")
PY

Code sample

funasr/train_utils/load_pretrained_model.py:

ori_state = torch.load(path, map_location=map_location)   # fresh, local, never read again

src_state = copy.deepcopy(ori_state)                     # every tensor copied a second time
src_state = src_state["state_dict"] if "state_dict" in src_state else src_state
src_state = src_state["model_state_dict"] if "model_state_dict" in src_state else src_state
src_state = src_state["model"] if "model" in src_state else src_state

The copy is not needed:

  • ori_state is produced inside the function by torch.load, so it has a single owner and is not shared with the caller.
  • ori_state is not referenced again after the deepcopy line.
  • src_state is only read from that point on. The loop below reads src_state[k_src].shape and rebinds references in dst_state (dst_state[k] = src_state[k_src]); the data actually reaches the model through obj.load_state_dict(dst_state, strict=True).
  • No tensor is mutated in place anywhere on this path, so the deep-copied version and the shared version are equivalent.

Expected behavior

Loading a checkpoint should not require memory for two copies of it. A 220M-parameter model should not need ~2x its own weight size in headroom.

Error logs

No exception. Measured with resource.getrusage(RUSAGE_SELF).ru_maxrss, same machine, same checkpoint, back to back:

funasr 1.4.16              : LOAD 13.3s   peak RSS 3461 MB   <All keys matched successfully>
same, deep copy removed    : LOAD 26.9s   peak RSS 2624 MB   <All keys matched successfully>

The 837MB difference tracks the 840MB checkpoint, which is the copy being dropped. The wall-clock column is noisy and is not part of the report; the memory difference is the reproducible result. All keys matched successfully on both sides is the correctness check.

Environment

  • OS: Linux 6.6.87.2-microsoft-standard-WSL2 (Ubuntu userspace)
  • Python version: 3.12.14
  • FunASR version: 1.4.16 (latest release; main at 41778c4 is identical here)
  • ModelScope version: 1.40.1
  • PyTorch version: 2.14.0+cu126
  • Install method: pip
  • Device: cpu for the load measurement
  • GPU model: NVIDIA GeForce RTX 4060 Laptop GPU
  • CUDA version: 12.6

Audio details

Not audio-related. The model used for the measurement is iic/speech_paraformer-large_asr_nat-zh-cn-16k-common-vocab8404-online (220M params, 956 tensors, 840MB model.pt). The overhead scales with checkpoint size, so it should reproduce with any large FunASR checkpoint, including the LLM-ASR models.

Vorherrschende Sprache
Python
Sterne
20.7k
Forks
2.1k
Ø Merge
23 Std. 18 Min.
Gemergte PRs (30 T.)
38

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus modelscope/FunASR

Alle Issues in modelscope/FunASR

Ähnliche Issues

Weitere Issues zu Python

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.