feat(core,llm): support ArrayBuffer, custom ModelDataLoader, and external .ptd weights in loadModel

Ouverte
  • Quelque chose ne va pas ?
#1,629 1 commentaire 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 1 jour

@Rasaboun y travaille déjà.

Depuis le 10/11/2026.

  • #1630 par @Rasaboun — ouverte

Évaluation

Difficulté
5/5
Temps estimé
Plus d'une semaine
Accessibilité débutants
1/100
Type d'issue
Fonctionnalité
Clarté
Clairement spécifiée
Activité
À l'abandon
Stack technique
pytorch, react-native, typescript
Domaine
ai, mobile

Piste de recherche

The issue identifies src/core/model.ts and the LLM runner, task, hook, and model loading APIs as the main entry points. Start by reviewing the linked PR #1630 and its tests to see the implementation scope and expected behavior. Done means the proposed model sources and external weight loading work through the core and LLM APIs, with the cited tests and documentation updates.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

core feature request
Problem description

loadModel(modelPath, options) and the LLM task APIs (createLLMRunner, createLLMChatSession, createLLMTranslator, useLLMChatSession) currently require a plain local .pte file path on disk and do not expose ExecuTorch's data_map_loader for separated .ptd weight archives.

This blocks three common production use cases:

  1. In-memory models (ArrayBuffer): loading small .pte programs directly from memory without writing a temporary file to disk.
  2. Separated .ptd external weight archives: ExecuTorch supports exporting a small .pte program alongside an external .ptd named-data archive (Module(data_loader, ..., data_map_loader)), where backend delegates such as XNNPACK unpack named weight segments during method initialization and immediately free the source buffers.
  3. Chunked / on-demand loading (DataLoader): apps that store models in custom containers or encrypted chunks cannot materialize a 200 MB–2 GB model into a temporary plaintext file or a single full-file ArrayBuffer without doubling peak/steady-state resident memory. They need a random-access loader (size() + load(offset, size)) so ExecuTorch only requests the byte ranges it needs and frees each FreeableBuffer segment as soon as the backend unpacks it.
Proposed solution

Extend the core loadModel primitive and thread it through the LLM runner/task layer while staying 100% backward-compatible with existing path-based callers:

  1. Add ModelDataLoader and ModelSource in src/core/model.ts: 
    export type ModelDataLoader = {
  size(): number;
  load(offset: number, size: number): ArrayBuffer;
};

export type ModelSource = string | ArrayBuffer | ModelDataLoader;
  2. Extend LoadModelOptions and loadModel: 
    export type LoadModelOptions = {
  readonly eagerLoadMethods?: boolean;
  readonly dataMapPath?: ModelSource;
};

export function loadModel(modelSource: ModelSource, options?: LoadModelOptions): Model;
    • string continues to use MmapDataLoader (NoMlock).
    • ArrayBuffer uses a 64-byte-aligned owning BufferDataLoader (model.path === '<buffer>').
    • ModelDataLoader uses a JSI-backed DataLoader (model.path === '<loader>') that returns 64-byte-aligned FreeableBuffers with a deallocation callback so ExecuTorch frees each segment once unpacked, and detaches its JS function/runtime handles before loadModel() returns so no jsi::Runtime* or JS closure outlives the load call.
  3. Thread ModelSource and dataMapPath through LLM APIs & hooks:
    • createLLMRunner(modelPath: ModelSource, tokenizerPath: string, modalities?, runtime?, options?: LoadModelOptions)
    • LLMModel<TSource extends ModelSource = string> (defaulting to string so models.llm.* registry entries remain typed as string URLs, while createLLMChatSession, createLLMTranslator, and useLLMChatSession accept ModelSource and dataMapPath).
    • Preserve ArrayBuffer and ModelDataLoader reference identity in useModel and useResourceDownload config memoization, and skip ModelDataLoader objects in download() traversal.
Alternative solutions
  • Writing temporary .pte / .ptd files to disk before calling loadModel(path): undesirable for encrypted/protected model assets (writes plaintext weights to flash storage) and adds unnecessary disk I/O and storage pressure.
  • Passing only whole ArrayBuffers: works for small .pte programs, but for large models it keeps a full copy of the raw weights in memory alongside the backend's packed weights (~170+ MB extra RSS on a 180 MB model). Supporting both ArrayBuffer and ModelDataLoader plus .ptd external weights lets XNNPACK free each weight segment right after packing.
Benefits to React Native ExecuTorch
  • Unlocks separated .ptd weight archives (data_map_loader) across loadModel and LLM tasks.
  • Enables zero-temp-file in-memory loading (ArrayBuffer) and low-memory streaming/chunked loading (ModelDataLoader) for custom asset containers or encrypted models.
  • Lives cleanly in core/model with no breaking changes to existing APIs or models registry types.
Additional context

I have opened PR #1630 implementing this with host C++ GoogleTest (ModelTest), Jest API tests, and documentation updates.

Langage dominant
TypeScript
Étoiles
1.8k
Forks
101
Merge moyen
1 j 10 h
PR mergées (30 j)
51

Préparer son environnement

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de software-mansion/react-native-executorch

Toutes les issues de software-mansion/react-native-executorch

Issues similaires

Plus d'issues TypeScript

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.