`PRISM_DEPTH_MAXIMUM` assumes ~6 MB of C stack; Ruby threads (1 MB) overflow at ~2000 levels

Abierto
  • ¿Algo no está bien?
#4,305 1 comentario 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
45/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
c, ruby
Área
compilers

Línea de trabajo

The depth check uses PRISM_DEPTH_MAXIMUM in parse.y; start by reading where nesting depth is incremented and validated. Review pm_options_t and the Ruby/Thread reproduction to see the 1 MB overflow. Done means parses on small stacks hit a safe 'nesting too deep' error or use a real stack check instead of segfaulting.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Follow-up to #4252. With #4258 and fb2fea4cb every nesting I tried now stops at "nesting too deep" on the main thread, including a pure C harness at 64000 levels. Thanks for the quick fix.

The limit only holds on a main-thread-sized stack though. Each level costs roughly 500–600 bytes of C stack (nested arrays are the worst case I measured; other constructs cost 150–450 bytes per depth unit), so reaching PRISM_DEPTH_MAXIMUM (10000) needs close to 6 MB. A Ruby thread gets 1 MB, and there the check never fires:

require "prism"

src = "x = " + "[" * 2000 + "]" * 2000
Prism.parse(src).errors                 # => []
Thread.new { Prism.parse(src) }.value   # => SystemStackError

Measured on main (cde8d1183, Ruby 3.4.9, arm64 macOS):

context stack nested [ nested [ pattern
main thread 8 MB "nesting too deep" at 10000 "nesting too deep" at 10000
Thread.new 1 MB overflows between 1500 and 2000 overflows between 2000 and 4000
C harness, ulimit -s 1024 1 MB SIGSEGV between 1600 and 2000 SIGSEGV between 2000 and 3000

Outside of CRuby that is a segfault rather than an exception, and Rust and the JVM default to 1–2 MB thread stacks as well; RUBY_THREAD_MACHINE_STACK_SIZE only helps whoever starts the CRuby process. I think the root is that #3413 matched the limit to Bison's YYMAXDEPTH, but Bison's stack is on the heap and Prism's is C frames (parse.y parses 9000 nested [ under ulimit -s 2048), so the same number does not give the same safety.

Options, in increasing order of effort:

  1. Lower the default to something that fits 1 MB. Simple, but reverts #3413 and rejects code that parses today; real view templates already went past 1000 in #1774.
  2. Keep the default and let the host set the limit through pm_options_t. No behaviour change; CRuby could pass a smaller value on non-main threads, which would reject some code that parses today (#1774 shows real templates past depth 1000) but still beats a segfault.
  3. Check actual stack use instead of a count, like CRuby's stack_check and CPython 3.14. Most robust, but needs a platform-specific way to find the stack bounds, with the counter as fallback.

Happy to open a PR for whichever you prefer. I have option 2 implemented (yuki-300/prism#1, if it helps to see the shape of it); option 3 I would want to discuss the API for first.

C harness used for the `ulimit` row 
#include <prism.h>
#include <stdio.h>
#include <stdlib.h>

int main(int argc, char **argv) {
    FILE *f = fopen(argv[1], "rb");
    fseek(f, 0, SEEK_END); long n = ftell(f); fseek(f, 0, SEEK_SET);
    uint8_t *buf = malloc(n); fread(buf, 1, n, f); fclose(f);

    pm_arena_t *arena = pm_arena_new();
    pm_parser_t *parser = pm_parser_new(arena, buf, n, NULL);
    pm_node_t *root = pm_parse(parser);
    printf("parsed %p\n", (void *) root);

    pm_parser_free(parser);
    pm_arena_free(arena);
    return 0;
}

$ cc -Iinclude harness.c build/libprism.a -o harness
$ ruby -e 'File.write("n.rb", "x = " + "[" * 2000 + "]" * 2000)'
$ (ulimit -s 1024; ./harness n.rb); echo $?
139
Lenguaje dominante
C
Estrellas
1k
Forks
199
Merge medio
8 h 7 min
PR fusionados (30 d)
72

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de ruby/prism

Todos los issues de ruby/prism

Issues similares

Más issues de C

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.