feat(core,llm): support ArrayBuffer, custom ModelDataLoader, and external .ptd weights in loadModel

Aperta
  • Qualcosa non va?
#1,629 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@Rasaboun ci sta già lavorando.

Dal 10/11/2026.

  • #1630 di @Rasaboun — aperta

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
1/100
Tipo di issue
Funzionalità
Chiarezza
Specificata chiaramente
Stato di attività
Ferma
Stack tecnologico
pytorch, react-native, typescript
Ambito
ai, mobile

Direzione di ricerca

The issue identifies src/core/model.ts and the LLM runner, task, hook, and model loading APIs as the main entry points. Start by reviewing the linked PR #1630 and its tests to see the implementation scope and expected behavior. Done means the proposed model sources and external weight loading work through the core and LLM APIs, with the cited tests and documentation updates.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

core feature request
Problem description

loadModel(modelPath, options) and the LLM task APIs (createLLMRunner, createLLMChatSession, createLLMTranslator, useLLMChatSession) currently require a plain local .pte file path on disk and do not expose ExecuTorch's data_map_loader for separated .ptd weight archives.

This blocks three common production use cases:

  1. In-memory models (ArrayBuffer): loading small .pte programs directly from memory without writing a temporary file to disk.
  2. Separated .ptd external weight archives: ExecuTorch supports exporting a small .pte program alongside an external .ptd named-data archive (Module(data_loader, ..., data_map_loader)), where backend delegates such as XNNPACK unpack named weight segments during method initialization and immediately free the source buffers.
  3. Chunked / on-demand loading (DataLoader): apps that store models in custom containers or encrypted chunks cannot materialize a 200 MB–2 GB model into a temporary plaintext file or a single full-file ArrayBuffer without doubling peak/steady-state resident memory. They need a random-access loader (size() + load(offset, size)) so ExecuTorch only requests the byte ranges it needs and frees each FreeableBuffer segment as soon as the backend unpacks it.
Proposed solution

Extend the core loadModel primitive and thread it through the LLM runner/task layer while staying 100% backward-compatible with existing path-based callers:

  1. Add ModelDataLoader and ModelSource in src/core/model.ts: 
    export type ModelDataLoader = {
  size(): number;
  load(offset: number, size: number): ArrayBuffer;
};

export type ModelSource = string | ArrayBuffer | ModelDataLoader;
  2. Extend LoadModelOptions and loadModel: 
    export type LoadModelOptions = {
  readonly eagerLoadMethods?: boolean;
  readonly dataMapPath?: ModelSource;
};

export function loadModel(modelSource: ModelSource, options?: LoadModelOptions): Model;
    • string continues to use MmapDataLoader (NoMlock).
    • ArrayBuffer uses a 64-byte-aligned owning BufferDataLoader (model.path === '<buffer>').
    • ModelDataLoader uses a JSI-backed DataLoader (model.path === '<loader>') that returns 64-byte-aligned FreeableBuffers with a deallocation callback so ExecuTorch frees each segment once unpacked, and detaches its JS function/runtime handles before loadModel() returns so no jsi::Runtime* or JS closure outlives the load call.
  3. Thread ModelSource and dataMapPath through LLM APIs & hooks:
    • createLLMRunner(modelPath: ModelSource, tokenizerPath: string, modalities?, runtime?, options?: LoadModelOptions)
    • LLMModel<TSource extends ModelSource = string> (defaulting to string so models.llm.* registry entries remain typed as string URLs, while createLLMChatSession, createLLMTranslator, and useLLMChatSession accept ModelSource and dataMapPath).
    • Preserve ArrayBuffer and ModelDataLoader reference identity in useModel and useResourceDownload config memoization, and skip ModelDataLoader objects in download() traversal.
Alternative solutions
  • Writing temporary .pte / .ptd files to disk before calling loadModel(path): undesirable for encrypted/protected model assets (writes plaintext weights to flash storage) and adds unnecessary disk I/O and storage pressure.
  • Passing only whole ArrayBuffers: works for small .pte programs, but for large models it keeps a full copy of the raw weights in memory alongside the backend's packed weights (~170+ MB extra RSS on a 180 MB model). Supporting both ArrayBuffer and ModelDataLoader plus .ptd external weights lets XNNPACK free each weight segment right after packing.
Benefits to React Native ExecuTorch
  • Unlocks separated .ptd weight archives (data_map_loader) across loadModel and LLM tasks.
  • Enables zero-temp-file in-memory loading (ArrayBuffer) and low-memory streaming/chunked loading (ModelDataLoader) for custom asset containers or encrypted models.
  • Lives cleanly in core/model with no breaking changes to existing APIs or models registry types.
Additional context

I have opened PR #1630 implementing this with host C++ GoogleTest (ModelTest), Jest API tests, and documentation updates.

Lingua principale
TypeScript
Stelle
1.8k
Fork
101
Merge medio
1g 10h
PR unite (30g)
51

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di software-mansion/react-native-executorch

Tutte le issue di software-mansion/react-native-executorch

Issue simili

Altre issue su TypeScript

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.