Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Retained sampled suffix can carry logprobs under a different causal prefix

Aperta
  • Qualcosa non va?
#883 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 3 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
25/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python
Ambito
machine-learning

Direzione di ricerca

Inizia con _retained_output_suffix e _tokenize_exact_projected_chat_history nel percorso di tokenizzazione, quindi esegui la regressione sintetica in tests.unit.trajectories.test_tokenize usando _chat_exchange. Esamina come i suffissi mantenuti ricevono i logprobs e i flag originali e conferma che i suffissi con contesto modificato non vengano selezionati come posizioni campionate esatte, a meno che il loro intero prefisso causale catturato non venga preservato; l’implementazione rimane in attesa della decisione sull’ambito.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Status reconciliation — September 11, 2026 (Schulman)

Default sampled-lineage attribution remains open; PR #885 is explicitly HELD by Brad. Do not merge it, add a blanket manual-history rejection rule or treat private experiment constraints as the shared API contract.

The independent prompt-aware capture-deduplication fix #886 is already merged at 66f644db05954ae950c13554ce0cc9f4df020af8. Earlier comments calling it unmerged are historical. #886 does not include #885 and does not resolve this separate default-attribution question.

Planned lane: Schulman and subagents, parked pending Brad's scope decision; no new implementation or launch is initiated by this status update. Manually assigned/edited histories remain supported.

Historical report (preserved):

Native multi-history tokenization can mark a retained response suffix
SAMPLED|EXACT and select it for training after earlier sampled tokens have been
removed from its conditioning context. Its stored logprob still belongs to the
original longer prefix. This is independent of the terminal renderer-tail work
in #882 and does not require the experimental raw-capture adapter.

Minimal synthetic capture:

Exchange Captured prompt Captured output
First [1] [2, 3]
Second [1, 3, 4] [5]

The second request retains the first assistant's message text, but its prompt
tokens omit token 2. With multi_history=True and
reconcile_text_equivalent_tokenizations=False, ART emits a second history
[1, 3, 4, 5], marks token 3 as SAMPLED|EXACT|ASSISTANT|OUTPUT (23), and gives
it the first response's logprob -0.3. first_occurrence_masks(where=SAMPLED)
selects it: prefix [1, 3] differs from the actual sampled prefix [1, 2, 3].
The probability was observed for 3 | [1, 2], not 3 | [1].

A CPU regression using the repository's existing synthetic exchange fixture:

import art
from art.trajectories import TokenFlag
from tests.unit.trajectories.test_tokenize import _chat_exchange

def test_retained_suffix_requires_its_original_causal_prefix():
    trajectory = art.Trajectory(
        exchanges=art.trajectories.TrajectoryExchanges(
            chat_completions=[
                _chat_exchange([1], [2, 3]),
                _chat_exchange([1, 3, 4], [5], offset=1),
            ]
        )
    )
    tokenized = trajectory.tokenize(
        multi_history=True,
        reconcile_text_equivalent_tokenizations=False,
    )
    masks = tokenized.tensorize().first_occurrence_masks(where=TokenFlag.SAMPLED)
    captured_prefixes = {(1, 2), (1, 2, 3), (1, 3, 4, 5)}
    for history, mask in zip(tokenized.histories, masks, strict=True):
        for index, selected in enumerate(mask.tolist()):
            if selected:
                assert tuple(history.tokens[:index + 1]) in captured_prefixes

This reproduces on ART 1ce5d7213e2d9cea2dd8a04a608966c24f4629a0 and current
077abf6fa7a3a51f9ff255c37f8886c6c8256082 (both _tokenize.py SHA256
cf8745b44edf3fb22a377246ccb81fc476153f4d96d5680e9c953180c4cff134), and the
reviewed terminal-boundary overlay 7bcd560cc117e593bbc8fad17a856ffc8dcbf667
(9922a80bafeccc89d1439b4c04bc08f63daccf047b37084e01d819e6d2ec0dd2).
The existing first-occurrence trie behaves as designed: it keys model-visible
prefixes, so it cannot repair an earlier attribution error.

The source is _retained_output_suffix, which searches every output[start:]
against the later prompt. _tokenize_exact_projected_chat_history then attaches
the suffix's original logprobs and sampled/exact flags at len(prompt).
Its complete-output requirement only applies to earlier finish_reason=length
sources; a non-length source can therefore lose its original causal conditioning.

Independent offline analysis also found this shape in two real captured
rollouts: 321 and 896 positions respectively remained first-occurrence-selected
after two leading sampled tokens were omitted. Raw content is private; the
synthetic regression above contains no research payload and reproduces the
same native behavior. This establishes attribution risk, not a measured GPU
training-quality effect.

Expected behavior: every sampled position selected for importance weighting
must retain the complete conditioning under which its stored logprob was
captured. A changed-context suffix can remain ordinary conditioning, but must
not reuse the old probability as an exact sampled probability under the new
prefix. Preserve the full original sampled source in another history or refuse
when exact attribution cannot be established; avoid silently dropping observed
samples. The experiment-local adapter is being tightened separately while the
appropriate shared history/attribution behavior is reviewed.

Lingua principale
Python
Stelle
10.8k
Fork
989
Merge medio
11h 38m
PR unite (30g)
104

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di OpenPipe/ART

Tutte le issue di OpenPipe/ART

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.