`PRISM_DEPTH_MAXIMUM` assumes ~6 MB of C stack; Ruby threads (1 MB) overflow at ~2000 levels

未关闭
  • 哪里不对？
#4,305 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
4/5
预计耗时
3-5 天
新手友好度
45/100
Issue 类型
缺陷
描述清晰度
基本清楚
活跃度
活跃
技术栈
c, ruby
领域
compilers

调研方向

The depth check uses PRISM_DEPTH_MAXIMUM in parse.y; start by reading where nesting depth is incremented and validated. Review pm_options_t and the Ruby/Thread reproduction to see the 1 MB overflow. Done means parses on small stacks hit a safe 'nesting too deep' error or use a real stack check instead of segfaulting.

由索引模型根据 Issue 内容生成。

描述

Follow-up to #4252. With #4258 and fb2fea4cb every nesting I tried now stops at "nesting too deep" on the main thread, including a pure C harness at 64000 levels. Thanks for the quick fix.

The limit only holds on a main-thread-sized stack though. Each level costs roughly 500–600 bytes of C stack (nested arrays are the worst case I measured; other constructs cost 150–450 bytes per depth unit), so reaching PRISM_DEPTH_MAXIMUM (10000) needs close to 6 MB. A Ruby thread gets 1 MB, and there the check never fires:

require "prism"

src = "x = " + "[" * 2000 + "]" * 2000
Prism.parse(src).errors                 # => []
Thread.new { Prism.parse(src) }.value   # => SystemStackError

Measured on main (cde8d1183, Ruby 3.4.9, arm64 macOS):

context stack nested [ nested [ pattern
main thread 8 MB "nesting too deep" at 10000 "nesting too deep" at 10000
Thread.new 1 MB overflows between 1500 and 2000 overflows between 2000 and 4000
C harness, ulimit -s 1024 1 MB SIGSEGV between 1600 and 2000 SIGSEGV between 2000 and 3000

Outside of CRuby that is a segfault rather than an exception, and Rust and the JVM default to 1–2 MB thread stacks as well; RUBY_THREAD_MACHINE_STACK_SIZE only helps whoever starts the CRuby process. I think the root is that #3413 matched the limit to Bison's YYMAXDEPTH, but Bison's stack is on the heap and Prism's is C frames (parse.y parses 9000 nested [ under ulimit -s 2048), so the same number does not give the same safety.

Options, in increasing order of effort:

  1. Lower the default to something that fits 1 MB. Simple, but reverts #3413 and rejects code that parses today; real view templates already went past 1000 in #1774.
  2. Keep the default and let the host set the limit through pm_options_t. No behaviour change; CRuby could pass a smaller value on non-main threads, which would reject some code that parses today (#1774 shows real templates past depth 1000) but still beats a segfault.
  3. Check actual stack use instead of a count, like CRuby's stack_check and CPython 3.14. Most robust, but needs a platform-specific way to find the stack bounds, with the counter as fallback.

Happy to open a PR for whichever you prefer. I have option 2 implemented (yuki-300/prism#1, if it helps to see the shape of it); option 3 I would want to discuss the API for first.

C harness used for the `ulimit` row 
#include <prism.h>
#include <stdio.h>
#include <stdlib.h>

int main(int argc, char **argv) {
    FILE *f = fopen(argv[1], "rb");
    fseek(f, 0, SEEK_END); long n = ftell(f); fseek(f, 0, SEEK_SET);
    uint8_t *buf = malloc(n); fread(buf, 1, n, f); fclose(f);

    pm_arena_t *arena = pm_arena_new();
    pm_parser_t *parser = pm_parser_new(arena, buf, n, NULL);
    pm_node_t *root = pm_parse(parser);
    printf("parsed %p\n", (void *) root);

    pm_parser_free(parser);
    pm_arena_free(arena);
    return 0;
}

$ cc -Iinclude harness.c build/libprism.a -o harness
$ ruby -e 'File.write("n.rb", "x = " + "[" * 2000 + "]" * 2000)'
$ (ulimit -s 1024; ./harness n.rb); echo $?
139
主要语言
C
星标
1k
派生
199
平均合并
8 小时 7 分钟
30 天内合并 PR
72

环境准备

  • 没有 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

ruby/prism 的其他 Issue

查看 ruby/prism 的全部 Issue

相似的 Issue

更多 C Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。