`PRISM_DEPTH_MAXIMUM` assumes ~6 MB of C stack; Ruby threads (1 MB) overflow at ~2000 levels

Aperta
  • Qualcosa non va?
#4,305 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
45/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
c, ruby
Ambito
compilers

Direzione di ricerca

The depth check uses PRISM_DEPTH_MAXIMUM in parse.y; start by reading where nesting depth is incremented and validated. Review pm_options_t and the Ruby/Thread reproduction to see the 1 MB overflow. Done means parses on small stacks hit a safe 'nesting too deep' error or use a real stack check instead of segfaulting.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Follow-up to #4252. With #4258 and fb2fea4cb every nesting I tried now stops at "nesting too deep" on the main thread, including a pure C harness at 64000 levels. Thanks for the quick fix.

The limit only holds on a main-thread-sized stack though. Each level costs roughly 500–600 bytes of C stack (nested arrays are the worst case I measured; other constructs cost 150–450 bytes per depth unit), so reaching PRISM_DEPTH_MAXIMUM (10000) needs close to 6 MB. A Ruby thread gets 1 MB, and there the check never fires:

require "prism"

src = "x = " + "[" * 2000 + "]" * 2000
Prism.parse(src).errors                 # => []
Thread.new { Prism.parse(src) }.value   # => SystemStackError

Measured on main (cde8d1183, Ruby 3.4.9, arm64 macOS):

context stack nested [ nested [ pattern
main thread 8 MB "nesting too deep" at 10000 "nesting too deep" at 10000
Thread.new 1 MB overflows between 1500 and 2000 overflows between 2000 and 4000
C harness, ulimit -s 1024 1 MB SIGSEGV between 1600 and 2000 SIGSEGV between 2000 and 3000

Outside of CRuby that is a segfault rather than an exception, and Rust and the JVM default to 1–2 MB thread stacks as well; RUBY_THREAD_MACHINE_STACK_SIZE only helps whoever starts the CRuby process. I think the root is that #3413 matched the limit to Bison's YYMAXDEPTH, but Bison's stack is on the heap and Prism's is C frames (parse.y parses 9000 nested [ under ulimit -s 2048), so the same number does not give the same safety.

Options, in increasing order of effort:

  1. Lower the default to something that fits 1 MB. Simple, but reverts #3413 and rejects code that parses today; real view templates already went past 1000 in #1774.
  2. Keep the default and let the host set the limit through pm_options_t. No behaviour change; CRuby could pass a smaller value on non-main threads, which would reject some code that parses today (#1774 shows real templates past depth 1000) but still beats a segfault.
  3. Check actual stack use instead of a count, like CRuby's stack_check and CPython 3.14. Most robust, but needs a platform-specific way to find the stack bounds, with the counter as fallback.

Happy to open a PR for whichever you prefer. I have option 2 implemented (yuki-300/prism#1, if it helps to see the shape of it); option 3 I would want to discuss the API for first.

C harness used for the `ulimit` row 
#include <prism.h>
#include <stdio.h>
#include <stdlib.h>

int main(int argc, char **argv) {
    FILE *f = fopen(argv[1], "rb");
    fseek(f, 0, SEEK_END); long n = ftell(f); fseek(f, 0, SEEK_SET);
    uint8_t *buf = malloc(n); fread(buf, 1, n, f); fclose(f);

    pm_arena_t *arena = pm_arena_new();
    pm_parser_t *parser = pm_parser_new(arena, buf, n, NULL);
    pm_node_t *root = pm_parse(parser);
    printf("parsed %p\n", (void *) root);

    pm_parser_free(parser);
    pm_arena_free(arena);
    return 0;
}

$ cc -Iinclude harness.c build/libprism.a -o harness
$ ruby -e 'File.write("n.rb", "x = " + "[" * 2000 + "]" * 2000)'
$ (ulimit -s 1024; ./harness n.rb); echo $?
139
Lingua principale
C
Stelle
1k
Fork
199
Merge medio
8h 7m
PR unite (30g)
72

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di ruby/prism

Tutte le issue di ruby/prism

Issue simili

Altre issue su C

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.