Unterminated =begin in heredoc interpolation causes trailing short-line read, lex mode leak, and location underflow

未关闭
  • 哪里不对？
#4,365 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
4/5
预计耗时
3-5 天
新手友好度
45/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
活跃
技术栈
c, ruby
领域
backend, compilers

调研方向

Start in src/prism.c and trace lex_embdoc, parser_lex EOF handling, and the four named node-location helpers: pm_statements_node_body_update, pm_interpolated_regular_expression_node_append, pm_multi_target_node_targets_append, and pm_parameters_node_location_set. Reproduce with the supplied 21-byte input and inspect comment boundaries, lexer modes, and node locations. Done when the trailing "if" is consumed as embdoc, EOF recovery leaves no stale modes or rewound position, and node ends remain valid.

由索引模型根据 Issue 内容生成。

描述

Fuzzer find on bdbd482dc.

An input such as "if A\n<<0\n#{\n=begin\nif" — where an unterminated =begin embedded documentation block occurs inside a heredoc interpolation (#{...) and ends with a trailing line shorter than 4 bytes ("if") — exposes three bugs in src/prism.c:

  1. lex_embdoc stops scanning when fewer than 4 bytes remain: lex_embdoc loops while (parser->current.end + 4 <= parser->end), leaving a trailing line shorter than 4 bytes ("if") unconsumed at the end of an unterminated =begin block so that it is lexed as Ruby code after =begin.
  2. Unterminated =begin or __END__ at EOF inside a non-default lex_mode does not pop parser->lex_modes: When =begin or __END__ reaches EOF inside PM_LEX_EMBEXPR / PM_LEX_HEREDOC, parser_lex emits PM_TOKEN_EOF immediately without popping parser->lex_modes. This strands PM_LEX_EMBEXPR and PM_LEX_HEREDOC on the mode stack; a subsequent parser_lex call during outer error recovery then pops PM_LEX_HEREDOC and rewinds parser->current back to line 1 after parse_heredoc has already returned.
  3. PM_NODE_START is updated before reading PM_NODE_END in 4 node-location helpers: pm_statements_node_body_update, pm_interpolated_regular_expression_node_append, pm_multi_target_node_targets_append, and pm_parameters_node_location_set modify node->location.start before evaluating PM_NODE_END(node) (node->location.start + node->location.length), which shifts PM_NODE_END(node) backwards whenever an appended child has a smaller start offset.

Reproduction

1. C Reproducer (pm_parse + pm_parser_comments_first + pm_visit_node)

(Interactive simulation in Prism Loupe)

#include "prism.h"
#include <inttypes.h>
#include <stdio.h>
#include <string.h>

static pm_visit_result_t
check_nodes(const pm_node_t *node, void *data) {
    size_t source_len = *(const size_t *) data;
    uint64_t end = (uint64_t) node->location.start + (uint64_t) node->location.length;

    if (node->type == PM_IF_NODE || end > source_len) {
        printf(
            "  %-18s start=%2" PRIu32 " length=%" PRIu32 " (0x%08" PRIx32 ") end=%" PRIu64 "%s\n",
            pm_node_type(node->type),
            node->location.start,
            node->location.length,
            node->location.length,
            end,
            end > source_len ? " [OUT OF BOUNDS]" : ""
        );
    }

    return PM_VISIT_DESCEND;
}

int main(void) {
    const char *src = "if A\n<<0\n#{\n=begin\nif";
    size_t len = strlen(src);

    pm_arena_t *arena = pm_arena_new();
    pm_parser_t *parser = pm_parser_new(arena, (const uint8_t *) src, len, NULL);
    pm_node_t *root = pm_parse(parser);

    const pm_comment_t *comment = pm_parser_comments_first(parser);
    pm_location_t loc = pm_comment_location(comment);

    printf("source_len = %zu\n", len);
    printf(
        "embdoc comment: start=%" PRIu32 " length=%" PRIu32 " (ends at byte %" PRIu32 ", leaving \"%s\" unconsumed)\n",
        loc.start,
        loc.length,
        loc.start + loc.length,
        src + loc.start + loc.length
    );
    printf("IfNodes / out-of-bounds nodes:\n");
    pm_visit_node(root, check_nodes, &len);

    pm_parser_free(parser);
    pm_arena_free(arena);
    return 0;
}

Building and running against libprism.a:

cc -Iinclude check.c build/libprism.a -o check && ./check

Output on the 21-byte input (0..21):

source_len = 21
embdoc comment: start=12 length=7 (ends at byte 19, leaving "if" unconsumed)
IfNodes / out-of-bounds nodes:
  PM_IF_NODE         start= 0 length=9 (0x00000009) end=9
  PM_IF_NODE         start=19 length=4294967286 (0xfffffff6) end=4294967305 [OUT OF BOUNDS]
主要语言
C
星标
1k
派生
202
平均合并
5 小时 55 分钟
30 天内合并 PR
85

环境准备

  • 没有 Dockerfile 或 Docker Compose 文件
  • 没有 Pull Request 模板
  • 阅读贡献指南

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

ruby/prism 的其他 Issue

查看 ruby/prism 的全部 Issue

相似的 Issue

更多 C Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。