Round 3 control (c): a window-4 observer does not lose 0.05 BA on converged on the calibration data (#1758 R3.0-I)

Abierto
  • ¿Algo no está bien?
#2,327 1 comentario 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
8/100
Tipo de issue
Error
Claridad
Necesita aclaración
Estado de actividad
Activo
Stack tecnológico
python
Área
testing-qa

Línea de trabajo

Start with tools/observer_oracle/judge.py on branch r30i and the docs/PREDICATES.md section "How Round 3 is judged", then read control_c_hcal.log and judge_calibration.log. Row f01_0 at step 10 shows the window-4 build certifying convergence earlier than INC, which is the mechanism behind the lost BA margin. Work cannot start until a maintainer rules on #1758 whether control (c) is restated or kept as registered; done means that ruling is recorded as a Deviation and control (c) is re-run and posted on #1758.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

area:observer kind:decision

The pre-registration's positive control (c) on #1758 says a planted build with OBSERVER_WINDOW_N = 4 must lose at least 0.05 BA on converged against INC. Measured with the R3.0-I instrument (tools/observer_oracle/judge.py, branch r30i), the window-4 build gains BA instead:

  • on H-cal, BA(INC) − BA(W4) = −0.0591 [−0.1354, +0.0074];
  • on corpus.json ∪ H-cal, −0.0305 [−0.0847, +0.0173];
  • on corpus.json alone, +0.0081 [−0.0600, +0.0722].

The instrument reads correctly. A direct run of H-cal row f01_0 @10 prints CK 10 false on INC and CK 10 true on W4. The label is true: x_10 is 5e-5 from the limit, eps is 5.7e-3, and INC's 11-value window still holds x_0 = 8.7. A shorter window certifies a settled geometric tail sooner, so it trades FPR for TPR: converged FPR is 0.0217 against 0 on H-cal. Rules 2 and 3 still FAIL W4 on H-cal, but the 0.05 BA margin named by (c) does not hold.

The logs are control_c_hcal.log and judge_calibration.log in the R3.0-I outputs.

Done when

  • A ruling on #1758 decides whether (c) is re-stated, for example as an FPR or decision-table criterion, or kept as registered and judged on H1 only.
  • The ruling is recorded as a Deviation in docs/PREDICATES.md "How Round 3 is judged" before the reveal, if it changes the text.
  • R3.0-I's control (c) is re-run under the ruling and its output is posted on #1758.

— Claude (builder, Claude Code)

Lenguaje dominante
C
Estrellas
3
Forks
7
Merge medio
6 h 32 min
PR fusionados (30 d)
285

Preparar el entorno

Abrir en Codespaces

Inicia el contenedor de desarrollo del proyecto en tu navegador, con tu propia cuenta de GitHub.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de InauguralSystems/EigenScript

Todos los issues de InauguralSystems/EigenScript

Issues similares

Más issues de C

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.