Embedding-based `semantic` chunking (the strategy behind ingestion's `NotImplementedError`) — deferred until a deterministic-snapshot mode is designed

Offen
  • Stimmt etwas nicht?
#925 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
5/5
Geschätzter Aufwand
Über eine Woche
Anfängerfreundlichkeit
8/100
Issue-Typ
Feature
Klarheit
Muss geklärt werden
Aktivitätsstatus
Aktiv
Tech-Stack
python
Bereich
backend

Rechercherichtung

The stub is _chunk_semantic in forgelm/ingestion.py, which raises NotImplementedError; its message points at docs/roadmap/risks-and-decisions.md rather than this issue. The feature is explicitly deferred until a deterministic-snapshot mode is designed, so there is no implementation to start. A newcomer can only usefully link this issue from the error message (via #712) and wait for the condition to hold.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

area: ingestion deferred severity: low source: roadmap wave: 4

Deferred work · F-PR29-A6-05 · severity Low · cost the feature: not estimated · recorded in docs/roadmap/risks-and-decisions.md:98

Summary

forgelm/ingestion.py reserves a semantic chunking strategy that raises NotImplementedError, and its message points users at a roadmap table row instead of an issue. The work was deferred at Phase 12 because a runtime embedding-model dependency and non-deterministic chunk counts conflict with air-gapped, reproducible Annex IV evidence. This issue is the tracker the error message should link to.

Condition or budget

Only viable once a deterministic-snapshot mode is designed (Phase 12's won't-do list in completed-phases.md).

State today (re-checked 2026-10-09)

_chunk_semantic raises NotImplementedError; no issue tracked the feature before this one.

Related

#712 asks for a real issue URL in the NotImplementedError message — this issue; #837 suggested pointing at the roadmap row instead.

Acceptance criteria

  • The NotImplementedError message links this issue (through #712).
  • When the condition holds: a deterministic, air-gap-safe semantic chunker with a pinned embedding snapshot and reproducible chunk counts.
  • The row is removed from the deferral table and the CHANGELOG names its ID (the table's removal contract).

Recorded in docs/roadmap/risks-and-decisions.md (read at f94595f) and re-checked against the code on 2026-10-09. The row leaves that table only when the fix is on main, the CHANGELOG names its ID and any promised guard exists.

Vorherrschende Sprache
Python
Sterne
9
Forks
1
Ø Merge
8 Std. 20 Min.
Gemergte PRs (30 T.)
59

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus HodeTech/ForgeLM

Alle Issues in HodeTech/ForgeLM

Ähnliche Issues

Weitere Issues zu Python

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.