`PRISM_DEPTH_MAXIMUM` assumes ~6 MB of C stack; Ruby threads (1 MB) overflow at ~2000 levels

Offen
  • Stimmt etwas nicht?
#4,305 1 Kommentar 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Anfängerfreundlichkeit
45/100
Issue-Typ
Bug
Klarheit
Größtenteils klar
Aktivitätsstatus
Aktiv
Tech-Stack
c, ruby
Bereich
compilers

Rechercherichtung

The depth check uses PRISM_DEPTH_MAXIMUM in parse.y; start by reading where nesting depth is incremented and validated. Review pm_options_t and the Ruby/Thread reproduction to see the 1 MB overflow. Done means parses on small stacks hit a safe 'nesting too deep' error or use a real stack check instead of segfaulting.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

Follow-up to #4252. With #4258 and fb2fea4cb every nesting I tried now stops at "nesting too deep" on the main thread, including a pure C harness at 64000 levels. Thanks for the quick fix.

The limit only holds on a main-thread-sized stack though. Each level costs roughly 500–600 bytes of C stack (nested arrays are the worst case I measured; other constructs cost 150–450 bytes per depth unit), so reaching PRISM_DEPTH_MAXIMUM (10000) needs close to 6 MB. A Ruby thread gets 1 MB, and there the check never fires:

require "prism"

src = "x = " + "[" * 2000 + "]" * 2000
Prism.parse(src).errors                 # => []
Thread.new { Prism.parse(src) }.value   # => SystemStackError

Measured on main (cde8d1183, Ruby 3.4.9, arm64 macOS):

context stack nested [ nested [ pattern
main thread 8 MB "nesting too deep" at 10000 "nesting too deep" at 10000
Thread.new 1 MB overflows between 1500 and 2000 overflows between 2000 and 4000
C harness, ulimit -s 1024 1 MB SIGSEGV between 1600 and 2000 SIGSEGV between 2000 and 3000

Outside of CRuby that is a segfault rather than an exception, and Rust and the JVM default to 1–2 MB thread stacks as well; RUBY_THREAD_MACHINE_STACK_SIZE only helps whoever starts the CRuby process. I think the root is that #3413 matched the limit to Bison's YYMAXDEPTH, but Bison's stack is on the heap and Prism's is C frames (parse.y parses 9000 nested [ under ulimit -s 2048), so the same number does not give the same safety.

Options, in increasing order of effort:

  1. Lower the default to something that fits 1 MB. Simple, but reverts #3413 and rejects code that parses today; real view templates already went past 1000 in #1774.
  2. Keep the default and let the host set the limit through pm_options_t. No behaviour change; CRuby could pass a smaller value on non-main threads, which would reject some code that parses today (#1774 shows real templates past depth 1000) but still beats a segfault.
  3. Check actual stack use instead of a count, like CRuby's stack_check and CPython 3.14. Most robust, but needs a platform-specific way to find the stack bounds, with the counter as fallback.

Happy to open a PR for whichever you prefer. I have option 2 implemented (yuki-300/prism#1, if it helps to see the shape of it); option 3 I would want to discuss the API for first.

C harness used for the `ulimit` row 
#include <prism.h>
#include <stdio.h>
#include <stdlib.h>

int main(int argc, char **argv) {
    FILE *f = fopen(argv[1], "rb");
    fseek(f, 0, SEEK_END); long n = ftell(f); fseek(f, 0, SEEK_SET);
    uint8_t *buf = malloc(n); fread(buf, 1, n, f); fclose(f);

    pm_arena_t *arena = pm_arena_new();
    pm_parser_t *parser = pm_parser_new(arena, buf, n, NULL);
    pm_node_t *root = pm_parse(parser);
    printf("parsed %p\n", (void *) root);

    pm_parser_free(parser);
    pm_arena_free(arena);
    return 0;
}

$ cc -Iinclude harness.c build/libprism.a -o harness
$ ruby -e 'File.write("n.rb", "x = " + "[" * 2000 + "]" * 2000)'
$ (ulimit -s 1024; ./harness n.rb); echo $?
139
Vorherrschende Sprache
C
Sterne
1k
Forks
199
Ø Merge
8 Std. 7 Min.
Gemergte PRs (30 T.)
72

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus ruby/prism

Alle Issues in ruby/prism

Ähnliche Issues

Weitere Issues zu C

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.