feat: support respanai/span-02 typed decision model

Aperta
#892 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
25/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python, rust

Direzione di ricerca

Start by reading the existing SIE Gemma serving infrastructure and process/HTTP adapter patterns, then inspect the linked model card, Dockerfile, launch recipe, and wire schema at the pinned revision. Assess whether a standard export or validated conversion can reuse existing serving, with the native engine as the parity reference. Done means documenting supported capabilities and requirements, exercising the decision API, and reproducing the specified parity, usage, readiness, quality, and latency checks; no measurements are included in the issue.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Request

Add respanai/span-02 as an SIE typed-decision model, with documented capabilities and a reproducible comparison against its native server.

Upstream: https://huggingface.co/respanai/span-02

Inspected revision: 63cbdd774391f1b7d9bc3829179efe996d0191ce.

Why this needs an integration assessment

The model card describes a Gemma 4 26B-A4B fine-tune with approximately 4B active parameters and a 131,072-token per-question context. It supports choice, noul, score, and number, plus optional images and reasoning. These capabilities make it relevant to evaluating long agent traces and other structured decision tasks.

The publisher documents the included Rust/CUDA engine as the supported route for this quantized artifact, rather than ordinary Transformers or vLLM loading. The shipped Dockerfile targets Blackwell sm_120a. The launch recipe also loads a speculative draft model and defines deterministic reasoning settings. Treat the official launch configuration as part of the reference behavior, not just the weights. No alternative-backend loading test was performed for this issue.

Sources: model card, Dockerfile, launch recipe, wire schema.

Proposed scope

Assess reuse of existing SIE Gemma serving infrastructure through a compatible standard export or validated conversion, alongside the option of wrapping the native engine with an existing process/HTTP adapter pattern. The native engine remains the parity reference; its documented use does not prove that a new SIE engine is necessary. Prefer reusable serving infrastructure over adding a model-specific execution framework, and do not assume the published packed weights are directly loadable elsewhere.

The initial milestone can cover text choice/noul/score through SIE's decision API. Explicitly document and reject unsupported capabilities if number, images, or reasoning controls are deferred; do not silently ignore them. Engine support and a hosted Serverless rollout should be separate deliverables.

Acceptance criteria

  • Pin weights, engine, tokenizer, draft model and launch settings; document hardware, memory, dependency and license requirements.
  • Add model discovery/capability metadata and exercise the supported decision API through the SIE SDK.
  • Compare native and SIE outputs on fixed requests: option order, probabilities, confidence, ordinal scores, multi-question requests and boundary lengths. State numerical tolerances before measuring parity.
  • Preserve the meaning of reasoning settings, including that upstream budget_ms=0 does not mean reasoning is disabled. Record deterministic reference settings separately from latency-oriented profiles.
  • Report input, reasoning and output usage correctly, including repeated-state accounting. Validate readiness, cancellation, overlength rejection and overload handling without silent truncation or hidden retries.
  • Measure quality and latency on the same inputs and hardware as the native reference. Attribute upstream benchmark claims to upstream until reproduced; measure long-context behavior rather than inferring it from the advertised window.

No inference or performance measurements were performed for this request.

Lingua principale
Python
Stelle
3.5k
Fork
328
Merge medio
16h 25m
PR unite (30g)
121

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di superlinked/sie

Tutte le issue di superlinked/sie

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.