Unterminated =begin in heredoc interpolation causes trailing short-line read, lex mode leak, and location underflow

Abierto
  • ¿Algo no está bien?
#4,365 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
45/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
c, ruby
Área
backend, compilers

Línea de trabajo

Start in src/prism.c and trace lex_embdoc, parser_lex EOF handling, and the four named node-location helpers: pm_statements_node_body_update, pm_interpolated_regular_expression_node_append, pm_multi_target_node_targets_append, and pm_parameters_node_location_set. Reproduce with the supplied 21-byte input and inspect comment boundaries, lexer modes, and node locations. Done when the trailing "if" is consumed as embdoc, EOF recovery leaves no stale modes or rewound position, and node ends remain valid.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Fuzzer find on bdbd482dc.

An input such as "if A\n<<0\n#{\n=begin\nif" — where an unterminated =begin embedded documentation block occurs inside a heredoc interpolation (#{...) and ends with a trailing line shorter than 4 bytes ("if") — exposes three bugs in src/prism.c:

  1. lex_embdoc stops scanning when fewer than 4 bytes remain: lex_embdoc loops while (parser->current.end + 4 <= parser->end), leaving a trailing line shorter than 4 bytes ("if") unconsumed at the end of an unterminated =begin block so that it is lexed as Ruby code after =begin.
  2. Unterminated =begin or __END__ at EOF inside a non-default lex_mode does not pop parser->lex_modes: When =begin or __END__ reaches EOF inside PM_LEX_EMBEXPR / PM_LEX_HEREDOC, parser_lex emits PM_TOKEN_EOF immediately without popping parser->lex_modes. This strands PM_LEX_EMBEXPR and PM_LEX_HEREDOC on the mode stack; a subsequent parser_lex call during outer error recovery then pops PM_LEX_HEREDOC and rewinds parser->current back to line 1 after parse_heredoc has already returned.
  3. PM_NODE_START is updated before reading PM_NODE_END in 4 node-location helpers: pm_statements_node_body_update, pm_interpolated_regular_expression_node_append, pm_multi_target_node_targets_append, and pm_parameters_node_location_set modify node->location.start before evaluating PM_NODE_END(node) (node->location.start + node->location.length), which shifts PM_NODE_END(node) backwards whenever an appended child has a smaller start offset.

Reproduction

1. C Reproducer (pm_parse + pm_parser_comments_first + pm_visit_node)

(Interactive simulation in Prism Loupe)

#include "prism.h"
#include <inttypes.h>
#include <stdio.h>
#include <string.h>

static pm_visit_result_t
check_nodes(const pm_node_t *node, void *data) {
    size_t source_len = *(const size_t *) data;
    uint64_t end = (uint64_t) node->location.start + (uint64_t) node->location.length;

    if (node->type == PM_IF_NODE || end > source_len) {
        printf(
            "  %-18s start=%2" PRIu32 " length=%" PRIu32 " (0x%08" PRIx32 ") end=%" PRIu64 "%s\n",
            pm_node_type(node->type),
            node->location.start,
            node->location.length,
            node->location.length,
            end,
            end > source_len ? " [OUT OF BOUNDS]" : ""
        );
    }

    return PM_VISIT_DESCEND;
}

int main(void) {
    const char *src = "if A\n<<0\n#{\n=begin\nif";
    size_t len = strlen(src);

    pm_arena_t *arena = pm_arena_new();
    pm_parser_t *parser = pm_parser_new(arena, (const uint8_t *) src, len, NULL);
    pm_node_t *root = pm_parse(parser);

    const pm_comment_t *comment = pm_parser_comments_first(parser);
    pm_location_t loc = pm_comment_location(comment);

    printf("source_len = %zu\n", len);
    printf(
        "embdoc comment: start=%" PRIu32 " length=%" PRIu32 " (ends at byte %" PRIu32 ", leaving \"%s\" unconsumed)\n",
        loc.start,
        loc.length,
        loc.start + loc.length,
        src + loc.start + loc.length
    );
    printf("IfNodes / out-of-bounds nodes:\n");
    pm_visit_node(root, check_nodes, &len);

    pm_parser_free(parser);
    pm_arena_free(arena);
    return 0;
}

Building and running against libprism.a:

cc -Iinclude check.c build/libprism.a -o check && ./check

Output on the 21-byte input (0..21):

source_len = 21
embdoc comment: start=12 length=7 (ends at byte 19, leaving "if" unconsumed)
IfNodes / out-of-bounds nodes:
  PM_IF_NODE         start= 0 length=9 (0x00000009) end=9
  PM_IF_NODE         start=19 length=4294967286 (0xfffffff6) end=4294967305 [OUT OF BOUNDS]
Lenguaje dominante
C
Estrellas
1k
Forks
202
Merge medio
5 h 55 min
PR fusionados (30 d)
85

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de ruby/prism

Todos los issues de ruby/prism

Issues similares

Más issues de C

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.