feat(core,llm): support ArrayBuffer, custom ModelDataLoader, and external .ptd weights in loadModel

Đang mở
  • Có gì không ổn?
#1,629 1 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

@Rasaboun đang làm issue này rồi.

Từ ngày 10/11/2026.

  • #1630 của @Rasaboun — đang mở

Đánh giá

Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức phù hợp với người mới
1/100
Loại issue
Tính năng
Độ rõ ràng
Đặc tả rõ ràng
Mức độ hoạt động
Đình trệ
Công nghệ
pytorch, react-native, typescript
Lĩnh vực
ai, mobile

Hướng nghiên cứu

The issue identifies src/core/model.ts and the LLM runner, task, hook, and model loading APIs as the main entry points. Start by reviewing the linked PR #1630 and its tests to see the implementation scope and expected behavior. Done means the proposed model sources and external weight loading work through the core and LLM APIs, with the cited tests and documentation updates.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

core feature request
Problem description

loadModel(modelPath, options) and the LLM task APIs (createLLMRunner, createLLMChatSession, createLLMTranslator, useLLMChatSession) currently require a plain local .pte file path on disk and do not expose ExecuTorch's data_map_loader for separated .ptd weight archives.

This blocks three common production use cases:

  1. In-memory models (ArrayBuffer): loading small .pte programs directly from memory without writing a temporary file to disk.
  2. Separated .ptd external weight archives: ExecuTorch supports exporting a small .pte program alongside an external .ptd named-data archive (Module(data_loader, ..., data_map_loader)), where backend delegates such as XNNPACK unpack named weight segments during method initialization and immediately free the source buffers.
  3. Chunked / on-demand loading (DataLoader): apps that store models in custom containers or encrypted chunks cannot materialize a 200 MB–2 GB model into a temporary plaintext file or a single full-file ArrayBuffer without doubling peak/steady-state resident memory. They need a random-access loader (size() + load(offset, size)) so ExecuTorch only requests the byte ranges it needs and frees each FreeableBuffer segment as soon as the backend unpacks it.
Proposed solution

Extend the core loadModel primitive and thread it through the LLM runner/task layer while staying 100% backward-compatible with existing path-based callers:

  1. Add ModelDataLoader and ModelSource in src/core/model.ts: 
    export type ModelDataLoader = {
  size(): number;
  load(offset: number, size: number): ArrayBuffer;
};

export type ModelSource = string | ArrayBuffer | ModelDataLoader;
  2. Extend LoadModelOptions and loadModel: 
    export type LoadModelOptions = {
  readonly eagerLoadMethods?: boolean;
  readonly dataMapPath?: ModelSource;
};

export function loadModel(modelSource: ModelSource, options?: LoadModelOptions): Model;
    • string continues to use MmapDataLoader (NoMlock).
    • ArrayBuffer uses a 64-byte-aligned owning BufferDataLoader (model.path === '<buffer>').
    • ModelDataLoader uses a JSI-backed DataLoader (model.path === '<loader>') that returns 64-byte-aligned FreeableBuffers with a deallocation callback so ExecuTorch frees each segment once unpacked, and detaches its JS function/runtime handles before loadModel() returns so no jsi::Runtime* or JS closure outlives the load call.
  3. Thread ModelSource and dataMapPath through LLM APIs & hooks:
    • createLLMRunner(modelPath: ModelSource, tokenizerPath: string, modalities?, runtime?, options?: LoadModelOptions)
    • LLMModel<TSource extends ModelSource = string> (defaulting to string so models.llm.* registry entries remain typed as string URLs, while createLLMChatSession, createLLMTranslator, and useLLMChatSession accept ModelSource and dataMapPath).
    • Preserve ArrayBuffer and ModelDataLoader reference identity in useModel and useResourceDownload config memoization, and skip ModelDataLoader objects in download() traversal.
Alternative solutions
  • Writing temporary .pte / .ptd files to disk before calling loadModel(path): undesirable for encrypted/protected model assets (writes plaintext weights to flash storage) and adds unnecessary disk I/O and storage pressure.
  • Passing only whole ArrayBuffers: works for small .pte programs, but for large models it keeps a full copy of the raw weights in memory alongside the backend's packed weights (~170+ MB extra RSS on a 180 MB model). Supporting both ArrayBuffer and ModelDataLoader plus .ptd external weights lets XNNPACK free each weight segment right after packing.
Benefits to React Native ExecuTorch
  • Unlocks separated .ptd weight archives (data_map_loader) across loadModel and LLM tasks.
  • Enables zero-temp-file in-memory loading (ArrayBuffer) and low-memory streaming/chunked loading (ModelDataLoader) for custom asset containers or encrypted models.
  • Lives cleanly in core/model with no breaking changes to existing APIs or models registry types.
Additional context

I have opened PR #1630 implementing this with host C++ GoogleTest (ModelTest), Jest API tests, and documentation updates.

Ngôn ngữ chính
TypeScript
Star
1.8k
Fork
101
Merge trung bình
1 ngày 10 giờ
Pull request đã merge (30 ngày)
51

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của software-mansion/react-native-executorch

Tất cả issue của software-mansion/react-native-executorch

Issue tương tự

Thêm issue về TypeScript

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.