feat: support respanai/span-02 typed decision model

Offen
#892 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
5/5
Geschätzter Aufwand
Über eine Woche
Anfängerfreundlichkeit
25/100
Issue-Typ
Feature
Klarheit
Größtenteils klar
Aktivitätsstatus
Aktiv
Tech-Stack
python, rust

Rechercherichtung

Start by reading the existing SIE Gemma serving infrastructure and process/HTTP adapter patterns, then inspect the linked model card, Dockerfile, launch recipe, and wire schema at the pinned revision. Assess whether a standard export or validated conversion can reuse existing serving, with the native engine as the parity reference. Done means documenting supported capabilities and requirements, exercising the decision API, and reproducing the specified parity, usage, readiness, quality, and latency checks; no measurements are included in the issue.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

Request

Add respanai/span-02 as an SIE typed-decision model, with documented capabilities and a reproducible comparison against its native server.

Upstream: https://huggingface.co/respanai/span-02

Inspected revision: 63cbdd774391f1b7d9bc3829179efe996d0191ce.

Why this needs an integration assessment

The model card describes a Gemma 4 26B-A4B fine-tune with approximately 4B active parameters and a 131,072-token per-question context. It supports choice, noul, score, and number, plus optional images and reasoning. These capabilities make it relevant to evaluating long agent traces and other structured decision tasks.

The publisher documents the included Rust/CUDA engine as the supported route for this quantized artifact, rather than ordinary Transformers or vLLM loading. The shipped Dockerfile targets Blackwell sm_120a. The launch recipe also loads a speculative draft model and defines deterministic reasoning settings. Treat the official launch configuration as part of the reference behavior, not just the weights. No alternative-backend loading test was performed for this issue.

Sources: model card, Dockerfile, launch recipe, wire schema.

Proposed scope

Assess reuse of existing SIE Gemma serving infrastructure through a compatible standard export or validated conversion, alongside the option of wrapping the native engine with an existing process/HTTP adapter pattern. The native engine remains the parity reference; its documented use does not prove that a new SIE engine is necessary. Prefer reusable serving infrastructure over adding a model-specific execution framework, and do not assume the published packed weights are directly loadable elsewhere.

The initial milestone can cover text choice/noul/score through SIE's decision API. Explicitly document and reject unsupported capabilities if number, images, or reasoning controls are deferred; do not silently ignore them. Engine support and a hosted Serverless rollout should be separate deliverables.

Acceptance criteria

  • Pin weights, engine, tokenizer, draft model and launch settings; document hardware, memory, dependency and license requirements.
  • Add model discovery/capability metadata and exercise the supported decision API through the SIE SDK.
  • Compare native and SIE outputs on fixed requests: option order, probabilities, confidence, ordinal scores, multi-question requests and boundary lengths. State numerical tolerances before measuring parity.
  • Preserve the meaning of reasoning settings, including that upstream budget_ms=0 does not mean reasoning is disabled. Record deterministic reference settings separately from latency-oriented profiles.
  • Report input, reasoning and output usage correctly, including repeated-state accounting. Validate readiness, cancellation, overlength rejection and overload handling without silent truncation or hidden retries.
  • Measure quality and latency on the same inputs and hardware as the native reference. Attribute upstream benchmark claims to upstream until reproduced; measure long-context behavior rather than inferring it from the advertised window.

No inference or performance measurements were performed for this request.

Vorherrschende Sprache
Python
Sterne
3.5k
Forks
328
Ø Merge
16 Std. 25 Min.
Gemergte PRs (30 T.)
121

Entwicklungsumgebung

Dieses Projekt bietet weder Dev-Container noch Dockerfile noch Beitragsleitfaden – die Einrichtung liegt bei Ihnen. Beginnen Sie mit der README; die allgemeinen Schritte stehen in unserem Leitfaden für den ersten Beitrag.

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus superlinked/sie

Alle Issues in superlinked/sie

Ähnliche Issues

Weitere Issues zu Python

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.