selftest: count-unspec-canary is seed-dependent and goes red ~0.24% of selftests on a correct tree
Los mantenedores suelen responder en 1 día
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Aptitud para principiantes
- 22/100
- Tipo de issue
- Error
- Claridad
- Bastante claro
- Estado de actividad
- Activo
- Área
- testing-qa
Línea de trabajo
The flake comes from corpus/lua-tests/math.lua line 845, which calls math.randomseed() with no argument, so the count-unspec-canary plant in tools/selftest.sh (lines 79-82) grades lua-count with --repeat 20 against a floor set in tools/grade.sh line 417. Read the grade floor logic and the self-test row in tests/gut.sh first. Done means the plant's verdict is independent of the time-derived seed, with 1,000 consecutive clean runs or a stated false-red bound, and the row still goes red when the floor is gutted.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Found while timing #71 (hq#58 Wave 2). The selftest plant count-unspec-canary can go red on a correct tree. Main has not changed since c650143.
Observed: run 41349374525 attempt 3, job 124982803346, step RUN selftest. The same head was green on attempts 1 and 2, and the failing step runs before any of #71's changes take effect:
PLANT count-unspec-canary expected=math.lua=PASS nextvar.lua=PASS rc=0 got=math.lua=FAIL nextvar.lua=PASS rc=1 WRONG
Mechanism:
corpus/lua-tests/math.lua:845callsmath.randomseed()with no argument, so the seed comes from the time and an address.- The statistics loops that follow then run a seed-dependent number of asserts.
- The plant (
tools/selftest.sh:79-82) gradeslua-countwith--repeat 20. tools/grade.sh:417setsfloor = lo * 0.60, wherelois the minimum of 20 reference runs. Any of the 20 candidate runs that falls below that floor turns the row FAIL.
Measured: 9,000 runs of .ref/bin/lua-count math.lua.
- Counts ran from a minimum of 25,699 to a maximum of 237,979, with a median of about 78.6k.
- rc was 0 every time, and the digit-free output had the same md5 every time.
- Under the plant's rule, that distribution gives an estimated ~0.24% false red per selftest, about 1 in 420.
- In the collector's 84 earlier suite logs it was never red, which fits that rate.
- Load does not change it. The distribution measured while a 4-lane fuzz ran was the same as without it (KS D=0.0147, 5% critical value 0.0304).
This is a gate that can go red without a bug. It was not a liar in the false-green direction, but a flaky plant trains people to rerun red.
Done when:
- The plant's verdict no longer depends on the time-derived seed. For example: fixed-seed or fixed-count inputs for the plant, or a floor taken from a reference sample with a stated false-red bound of 1e-6 or less per selftest. Do not edit the vendored test, and do not lower the 0.60 fraction as the fix.
- Evidence in the PR: at least 1,000 consecutive runs of the plant's grade command (
tools/grade.sh --candidate .ref/bin/lua-count --ref .ref/bin/lua --tier S --only math.lua,nextvar.lua --repeat 20) with 0 FAIL, or the computed bound. - The
count-unspec-canaryrow intests/gut.shstill bites: with the floor gutted, the self-test goes red.
Generated by Claude Code
- Lenguaje dominante
- Lua
- Estrellas
- 0
- Forks
- 0
- Merge medio
- 43 min
- PR fusionados (30 d)
- 10
Preparar el entorno
Inicia el contenedor de desarrollo del proyecto en tu navegador, con tu propia cuenta de GitHub.
- Sin Dockerfile ni archivo de Docker Compose
- Sin plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de InauguralSystems/EigenLua
-
Dificultad 5/5 Más de una semana Aptitud para principiantes 8/100
InauguralSystems/EigenLua#78 · 1 comentario ·
Los mantenedores suelen responder en 1 día
-
Dificultad 4/5 3-5 días Aptitud para principiantes 45/100
InauguralSystems/EigenLua#74 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 3/5 1-2 días Aptitud para principiantes 35/100
InauguralSystems/EigenLua#63 ·
Los mantenedores suelen responder en 1 día
-
1/0 and 0/0 abort the program with 'EigenScript value error: division by zero' (ref: inf / nan)Abiertobug eigenscript-gap
Dificultad 3/5 1-2 días Aptitud para principiantes 42/100
InauguralSystems/EigenLua#50 ·
Los mantenedores suelen responder en 1 día
-
bug eigenscript-gap
Dificultad 5/5 Más de una semana Aptitud para principiantes 12/100
InauguralSystems/EigenLua#49 ·
Los mantenedores suelen responder en 1 día
Todos los issues de InauguralSystems/EigenLua
Issues similares
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
silverbulletmd/silverbullet#2254 ·
Los mantenedores suelen responder en 1 día
-
bug discord triage
Dificultad 2/5 1-3 horas Aptitud para principiantes 77/100
Los mantenedores suelen responder en 1 día
-
severity: medium
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
luainkernel/lunatik#2699 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 65/100
esmuellert/codediff.nvim#594 ·
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100