Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Proposal: Support stateful model for WebNN

Aperta
  • Qualcosa non va?
#966 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
25/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
javascript
Ambito
api, machine-learning

Direzione di ricerca

Start by reading the proposal in this issue, especially the MLBuildOptions, MLGraph, and MLContext API changes and the stated stateful graph behavior. The issue names no implementation files or tests, so first locate the WebNN specification sections and related tests for graph building and dispatch. Done means defining stateful cache behavior, fallback behavior, validation, and reset semantics clearly enough to implement and test.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Problem

A language model generates one token per dispatch(). Each step reads the KV cache built by all earlier steps and adds to it. WebNN is stateless: nothing carries over from one dispatch to the next. So the cache has to be a graph input and a graph output of every dispatch. This is expensive,because each decode step copies the whole cache in and out, and usually allocates a new one. Step time grows with the conversation, even though each step only computes one token.

Reusing MLTensors doesn't solve the problem. A tensor can't be both an input and an output of the same dispatch, so the graph still writes a full new copy of the cache every step. More importantly, WebNN has no way to say "this output is the next dispatch's input". Without that, a backend can't keep the cache on the device and update it in place.

Proposal

The graph is built as it is today, with the cache as an ordinary input and output. At build time, the page lists which input/output pairs carry the cache and asks for a stateful graph:

dictionary MLKVCachePair {
  required DOMString input;   // graph input that provides the cache
  required DOMString output;  // graph output that stores the updated cache
};

dictionary MLBuildOptions {
  boolean useStateful = false;
  sequence<MLKVCachePair> kvCachePairs = [];
};

interface MLGraphBuilder {
  // The existing build() gains an optional options argument.
  Promise<MLGraph> build(MLNamedOperands outputs,
                         optional MLBuildOptions options = {});
};

partial interface MLGraph {
  readonly attribute boolean isStateful;
};

partial interface MLContext {
  undefined resetKVCache(MLGraph graph);
};

const graph = await builder.build(outputs, {
  useStateful: true,
  kvCachePairs: [
    {input: 'past_key_0', output: 'present_key_0'},
    {input: 'past_value_0', output: 'present_value_0'},
    // ... one pair for the keys and one for the values of each layer
  ],
});

if (graph.isStateful) {
  // Bind only the real inputs and outputs. The cache stays inside the graph.
  context.dispatch(graph, {input_ids, position_ids, attention_mask}, {logits});
  // Start a new conversation.
  context.resetKVCache(graph);
} else {
  // The backend has no state support: bind past_* and present_* and carry the cache as today.
}

How a stateful graph behaves:

  • It is a request. useStateful asks for a stateful graph, and isStateful says whether the page got one. A backend without state support builds the same graph stateless, so the page still runs everywhere.
  • The cache leaves the graph's interface. In a stateful graph, the page doesn't bind the paired inputs and outputs. Binding one is a TypeError.
  • The cache belongs to the graph. It starts empty and each dispatch updates it, in the order the dispatches are issued. It is kept until resetKVCache() is called or the graph is destroyed, and it is never shared with another graph. Its capacity is fixed, and a dispatch that would go beyond it fails.
  • The cache is hidden. No API reads its contents. The backend can keep it in whatever form suits the device, and the user's conversation is not exposed to script.
  • No new operators. Exported language models already have these pairs, such as past_key_values.* and present.* in ONNX, so a converter only needs to list them.
Lingua principale
Bikeshed
Stelle
566
Fork
72
Merge medio
6g 5h
PR unite (30g)
1

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di webmachinelearning/webnn

Tutte le issue di webmachinelearning/webnn

Issue simili

Altre issue su Backend & API Design

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.