Unterminated =begin in heredoc interpolation causes trailing short-line read, lex mode leak, and location underflow

Offen
  • Stimmt etwas nicht?
#4,365 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Anfängerfreundlichkeit
45/100
Issue-Typ
Bug
Klarheit
Klar beschrieben
Aktivitätsstatus
Aktiv
Tech-Stack
c, ruby
Bereich
backend, compilers

Rechercherichtung

Start in src/prism.c and trace lex_embdoc, parser_lex EOF handling, and the four named node-location helpers: pm_statements_node_body_update, pm_interpolated_regular_expression_node_append, pm_multi_target_node_targets_append, and pm_parameters_node_location_set. Reproduce with the supplied 21-byte input and inspect comment boundaries, lexer modes, and node locations. Done when the trailing "if" is consumed as embdoc, EOF recovery leaves no stale modes or rewound position, and node ends remain valid.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

Fuzzer find on bdbd482dc.

An input such as "if A\n<<0\n#{\n=begin\nif" — where an unterminated =begin embedded documentation block occurs inside a heredoc interpolation (#{...) and ends with a trailing line shorter than 4 bytes ("if") — exposes three bugs in src/prism.c:

  1. lex_embdoc stops scanning when fewer than 4 bytes remain: lex_embdoc loops while (parser->current.end + 4 <= parser->end), leaving a trailing line shorter than 4 bytes ("if") unconsumed at the end of an unterminated =begin block so that it is lexed as Ruby code after =begin.
  2. Unterminated =begin or __END__ at EOF inside a non-default lex_mode does not pop parser->lex_modes: When =begin or __END__ reaches EOF inside PM_LEX_EMBEXPR / PM_LEX_HEREDOC, parser_lex emits PM_TOKEN_EOF immediately without popping parser->lex_modes. This strands PM_LEX_EMBEXPR and PM_LEX_HEREDOC on the mode stack; a subsequent parser_lex call during outer error recovery then pops PM_LEX_HEREDOC and rewinds parser->current back to line 1 after parse_heredoc has already returned.
  3. PM_NODE_START is updated before reading PM_NODE_END in 4 node-location helpers: pm_statements_node_body_update, pm_interpolated_regular_expression_node_append, pm_multi_target_node_targets_append, and pm_parameters_node_location_set modify node->location.start before evaluating PM_NODE_END(node) (node->location.start + node->location.length), which shifts PM_NODE_END(node) backwards whenever an appended child has a smaller start offset.

Reproduction

1. C Reproducer (pm_parse + pm_parser_comments_first + pm_visit_node)

(Interactive simulation in Prism Loupe)

#include "prism.h"
#include <inttypes.h>
#include <stdio.h>
#include <string.h>

static pm_visit_result_t
check_nodes(const pm_node_t *node, void *data) {
    size_t source_len = *(const size_t *) data;
    uint64_t end = (uint64_t) node->location.start + (uint64_t) node->location.length;

    if (node->type == PM_IF_NODE || end > source_len) {
        printf(
            "  %-18s start=%2" PRIu32 " length=%" PRIu32 " (0x%08" PRIx32 ") end=%" PRIu64 "%s\n",
            pm_node_type(node->type),
            node->location.start,
            node->location.length,
            node->location.length,
            end,
            end > source_len ? " [OUT OF BOUNDS]" : ""
        );
    }

    return PM_VISIT_DESCEND;
}

int main(void) {
    const char *src = "if A\n<<0\n#{\n=begin\nif";
    size_t len = strlen(src);

    pm_arena_t *arena = pm_arena_new();
    pm_parser_t *parser = pm_parser_new(arena, (const uint8_t *) src, len, NULL);
    pm_node_t *root = pm_parse(parser);

    const pm_comment_t *comment = pm_parser_comments_first(parser);
    pm_location_t loc = pm_comment_location(comment);

    printf("source_len = %zu\n", len);
    printf(
        "embdoc comment: start=%" PRIu32 " length=%" PRIu32 " (ends at byte %" PRIu32 ", leaving \"%s\" unconsumed)\n",
        loc.start,
        loc.length,
        loc.start + loc.length,
        src + loc.start + loc.length
    );
    printf("IfNodes / out-of-bounds nodes:\n");
    pm_visit_node(root, check_nodes, &len);

    pm_parser_free(parser);
    pm_arena_free(arena);
    return 0;
}

Building and running against libprism.a:

cc -Iinclude check.c build/libprism.a -o check && ./check

Output on the 21-byte input (0..21):

source_len = 21
embdoc comment: start=12 length=7 (ends at byte 19, leaving "if" unconsumed)
IfNodes / out-of-bounds nodes:
  PM_IF_NODE         start= 0 length=9 (0x00000009) end=9
  PM_IF_NODE         start=19 length=4294967286 (0xfffffff6) end=4294967305 [OUT OF BOUNDS]
Vorherrschende Sprache
C
Sterne
1k
Forks
202
Ø Merge
5 Std. 55 Min.
Gemergte PRs (30 T.)
85

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus ruby/prism

Alle Issues in ruby/prism

Ähnliche Issues

Weitere Issues zu C

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.