Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Proposal: Support build-time metadata to describe how a graph will be driven

Open
  • Is something wrong?
#965 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
javascript, web-dev
Domain
api, machine-learning

Research direction

Start by reading the WebNN build() definition and the surrounding Web IDL dictionaries in the spec. Work out how the proposed metadata members and optional build options fit the API, including the stated hint behavior and validation rules. Done means the spec describes the two language-model metadata fields and how backends may handle them.

Written by the indexing model from the issue text.

Description

Problem

WebNN can mark a dimension as dynamic (#883), but that doesn't tell the backend how the page will use it. For example, a dynamic seq_len doesn't say how long the longest prompt will be.

Some backends, especially NPUs, compile fixed-size models up front, so they need these numbers before the first dispatch. For a language model, an NPU backend has to decide at compile time:

  • the longest prompt it will accept
  • how much KV cache to reserve for the reply

Only the page knows these numbers, but today WebNN gives the page no way to pass these numbers at build time, so the backend can only guess. For example, OpenVINO's NPU path (NPUW) assumes by default a prompt of at most 1024 tokens and 128 tokens for the reply. A longer prompt fails. A larger default would waste memory and compile time for pages that don't need it.

Proposal

Add an optional options to build(), which contains metadata member:

dictionary MLModelMetadata {
  [EnforceRange] unsigned long maxPromptLength;   // longest prompt, in tokens, in one dispatch
  [EnforceRange] unsigned long minResponseLength; // tokens reserved for the reply
};

dictionary MLBuildOptions {
  MLModelMetadata metadata = {};
};

interface MLGraphBuilder {
  // The existing build() gains an optional options argument.
  Promise<MLGraph> build(MLNamedOperands outputs,
                         optional MLBuildOptions options = {});
};

const graph = await builder.build({logits}, {
  metadata: {maxPromptLength: 4096, minResponseLength: 256},
});

How metadata behaves:

  • Its members are defined by the spec, not chosen by the page. Like operators, each member is added by the Working Group. A page can't pass arbitrary keys, and there are no vendor-specific members. This issue proposes only the two language-model members above. Others can be proposed later in the same way.
  • It is a hint. It never changes what the graph computes. It only affects performance and resource limits, such as how long a context fits. A backend may ignore a member, but must not fail because of it.
  • It describes the workload, not the device. "The longest prompt is 4096 tokens", not "use tile size 64".
Dominant language
Bikeshed
Stars
566
Forks
72
Avg merge
6d 5h
Merged PRs (30d)
1

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from webmachinelearning/webnn

All issues in webmachinelearning/webnn

Similar issues

More Backend & API Design issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.