[SKEEP-006]: Input front ends and model identity
I maintainer di solito rispondono entro 1 giorno
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Idoneità per principianti
- 15/100
- Tipo di issue
- Funzionalità
- Chiarezza
- Abbastanza chiara
- Stato di attività
- Attiva
- Stack tecnologico
- huggingface, kotlin
- Ambito
- backend, machine-learning
Direzione di ricerca
Read the draft proposal at docs/modules/skeep/pages/006-input-frontends-and-model-identity.adoc and its registration in docs/modules/skeep/nav.adoc. The proposal is still a draft, and implementation is already underway in a fork; wait for the maintainer to accept it and review the linked implementation work before taking on any sub-issue. Done means the proposal’s status and follow-up issues reflect the agreed scope.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Proposal document: docs/modules/skeep/pages/006-input-frontends-and-model-identity.adoc — prepared in a personal fork: https://github.com/michalharakal/SKaiNET-core-upstream/blob/feature/skeep-006-input-frontends/docs/modules/skeep/pages/006-input-frontends-and-model-identity.adoc
Status: Draft
Branch: feature/skeep-006-input-frontends (fork; the upstream pull request follows once the #1321 good-first-issue lanes have landed, so the contributors working them are not disturbed)
Trigger
Public Kotlin API (new modules skainet-model-identity, skainet-frontend-core, skainet-frontend-text; the sk.ainet.io.tokenizer package becomes deprecated aliases) · compatibility/migration policy (three-release deprecation of the superseded identity switches and of the SKaiNET-transformers legacy tokenizer stack) · runtime integration (the resolver is what model loaders, Java callers and cartridge facades use).
Summary
A checkpoint is weights, an identity and the recipe that turns raw input into the tensors the weights expect. Today the recipe is re-derived downstream: SKaiNET-transformers carries a second tokenizer stack that disagrees with the engine's, model identity is split three ways in the engine and a fourth way downstream, and audio/image preprocessing has no home at all. The proposal introduces FrontEnd<I, O> with Tokenizer as its first implementor, a string-based ModelIdentity with an immutable ModelFamilyRegistry that weight mapping and front ends both key on, and FrontEndResolver as the single entry point. Algorithms live in the engine, golden-tested against the Hugging Face tokenizers library; families (downstream) contribute policy only. Audio and image features follow on the same contract in later lanes.
Related DARC features
- #1321 — tokenizer.json parity for BPE encoders (the lane fixes #1323–#1326 are folded into the moved implementations; the lanes stay open for their contributors and are closed by the upstream PR of this proposal or by their own PRs, whichever lands first)
- SKaiNET-developers/SKaiNET-transformers#471 — legacy tokenizer defects and parity gate (its lanes become the migration gate)
- SKaiNET-developers/SKaiNET-transformers#330 — two GGUF tokenizer paths disagree (answered by
ModelFamilyDescriptor+FrontEndResolver)
Sub-issues
Filed once the proposal is Accepted: engine F2 (contract, identity, resolver — implemented in the fork branch), F4 log-mel front end, F5 image front ends; SKaiNET-transformers F3 (consume engine tokenizers — implemented in the fork branch feature/skeep-006-frontend-consumer of https://github.com/michalharakal/SKaiNET-transformers), F6 audio families.
Status upkeep
- Proposal registered in
docs/modules/skeep/nav.adocand the "Current Proposals" table (fork branch) - Maintainer moved status to
Accepted - Implementation PR(s) linked here and flipped the proposal's
Status:toImplemented
- Lingua principale
- Kotlin
- Stelle
- 52
- Fork
- 15
- Merge medio
- 1g 8h
- PR unite (30g)
- 37
Preparare l'ambiente
- Nessun Dockerfile né file Docker Compose
- Nessun modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di SKaiNET-developers/SKaiNET
-
coding good first issue platform size:xs skill:js sub-issue
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
SKaiNET-developers/SKaiNET#1232 ·
I maintainer di solito rispondono entro 1 giorno
-
compute-backend dsl/api enhancement size:m skill:kotlin-core skill:numerics
Difficoltà 5/5 Più di una settimana Idoneità per principianti 18/100
SKaiNET-developers/SKaiNET#1343 ·
I maintainer di solito rispondono entro 1 giorno
-
compute-backend dsl/api enhancement size:m skill:kotlin-core skill:numerics
Difficoltà 5/5 Più di una settimana Idoneità per principianti 12/100
SKaiNET-developers/SKaiNET#1342 ·
I maintainer di solito rispondono entro 1 giorno
-
assessment size:s skill:review sub-issue
Difficoltà 4/5 1-2 giorni Idoneità per principianti 18/100
SKaiNET-developers/SKaiNET#1330 ·
I maintainer di solito rispondono entro 1 giorno
-
documentation good first issue size:s skill:docs sub-issue
Difficoltà 3/5 1-2 giorni Idoneità per principianti 78/100
SKaiNET-developers/SKaiNET#1329 ·
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di SKaiNET-developers/SKaiNET
Issue simili
-
afk-ok area:data-quality importer size:S
Difficoltà 2/5 1-3 ore Idoneità per principianti 74/100
enorm-labs/event-junkie#3301 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 65/100
I maintainer di solito rispondono entro 2 giorni
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
rustyrazorblade/easy-db-lab#1017 ·
I maintainer di solito rispondono entro 1 giorno
-
enhancement impact:U1 later P3 reach:R2 size:S
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100