Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

ArrowArrayConcatenator throws on a zero-length string/binary array with an empty offsets buffer

Abierto Apto para principiantes
  • ¿Algo no está bien?
#462 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
2/5
Tiempo estimado
1-3 horas
Aptitud para principiantes
90/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
csharp
Área
data-engineering

Línea de trabajo

Empieza por src/Apache.Arrow/Arrays/ArrayDataConcatenator.cs y lee ConcatenateVariableBinaryValueBuffer() y ConcatenateLargeVariableBinaryValueBuffer() junto con las rutas cercanas que omiten las entradas de longitud cero. Comprueba que concatenar una cadena vacía o un array binario vacío con un búfer de offsets vacío ya no provoca una excepción y que se conservan los valores de la otra entrada.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Summary

ArrowArrayConcatenator.Concatenate throws ArgumentOutOfRangeException when one of the inputs is a zero-length string, binary, large_string or large_binary array whose offsets buffer is empty (0 bytes).

That input is valid, and it arrives through IPC. For a length-0 variable-size array, Arrow C++ accepts an absent or empty offsets buffer (ARROW-544, ValidateOffsetsAndSizes in cpp/src/arrow/array/validate.cc: "For length 0, an empty offsets buffer is accepted"). Its IPC writer emits that buffer as it is (GetZeroBasedValueOffsets returns array.value_offsets() unchanged when length() == 0). nanoarrow's IPC writer does the same. Apache.Arrow's ArrowStreamReader reads such a batch fine. Only concatenating it fails.

Seen on Apache.Arrow 23.0.0 (.NET 10, Linux arm64). The code is unchanged on main as of 117da35.

Cause

ConcatenateVariableBinaryValueBuffer() (src/Apache.Arrow/Arrays/ArrayDataConcatenator.cs:972) and ConcatenateLargeVariableBinaryValueBuffer() (:869) slice Length + 1 offsets from every input:

var offsets = arrayData.Buffers[1].Span.CastTo<int>().Slice(arrayData.Offset, arrayData.Length + 1);

For a length-0 input with an empty offsets buffer, that slices one element out of an empty span and throws. The neighbouring list paths (:758) and the offset-buffer concatenators ConcatenateOffsetBuffer() (:995) and ConcatenateLargeOffsetBuffer() already skip such inputs with if (arrayData.Length == 0) continue;. Those two functions are missing the same guard. A length-0 input contributes no value bytes, so skipping it is correct.

Repro (pure C#)

using Apache.Arrow;
using Apache.Arrow.Types;

// A zero-length string array with an empty (0-byte) offsets buffer, as read from IPC
var empty = new StringArray(new ArrayData(StringType.Default, length: 0, nullCount: 0, offset: 0,
    new[] { ArrowBuffer.Empty, ArrowBuffer.Empty, ArrowBuffer.Empty }));
var other = new StringArray.Builder().Append("a").Build();

Console.WriteLine(ArrowArrayConcatenator.Concatenate(new IArrowArray[] { empty, other }).Length);

Unhandled exception. System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values.
   at Apache.Arrow.ArrayDataConcatenator.ArrayDataConcatenationVisitor.ConcatenateVariableBinaryValueBuffer()
   at Apache.Arrow.ArrayDataConcatenator.ArrayDataConcatenationVisitor.ConcatenateVariableBinaryArrayData(IArrowType type)
   at Apache.Arrow.ArrayDataConcatenator.ArrayDataConcatenationVisitor.Visit(StringType type)

Repro (IPC from pyarrow)

This writes a stream whose string, large_string and list<int32> columns have 0-byte offsets buffers. pyarrow 26.0.0 validates it with validate(full=True) before writing and after reading it back:

import pyarrow as pa, pyarrow.ipc as ipc

cols = {
    "s": pa.Array.from_buffers(pa.string(), 0, [None, None, pa.py_buffer(b"")]),
    "ls": pa.Array.from_buffers(pa.large_string(), 0, [None, None, pa.py_buffer(b"")]),
    "l": pa.Array.from_buffers(pa.list_(pa.int32()), 0, [None, None],
                               children=[pa.array([], pa.int32())]),
}
batch = pa.record_batch(list(cols.values()), names=list(cols))
with ipc.new_stream("empty_offsets.arrows", batch.schema) as w:
    w.write_batch(batch)

Reading it with ArrowStreamReader and calling ArrowArrayConcatenator.Concatenate(new[] { col, col }) on each column gives:

column 0 utf8 length=0
  ArgumentOutOfRangeException at ...ConcatenateVariableBinaryValueBuffer()
column 1 large_utf8 length=0
  ArgumentOutOfRangeException at ...ConcatenateLargeVariableBinaryValueBuffer()
column 2 list length=0
  Concatenate([col, col]) length=0      <- lists already skip empty inputs

Expected: the empty input contributes nothing, and the concatenation succeeds.

Suggested fix: add if (arrayData.Length == 0) continue; to the loops in ConcatenateVariableBinaryValueBuffer() and ConcatenateLargeVariableBinaryValueBuffer(), as the other offset-reading paths in the file already do.

Context

I hit this when a DuckDB extension started writing Arrow IPC with nanoarrow, and a .NET worker using Apache.Arrow concatenated the incoming batches. The value array of an empty MAP arrived as a zero-length string with an empty offsets buffer.

Lenguaje dominante
C#
Estrellas
42
Forks
31
Merge medio
1 d 23 h
PR fusionados (30 d)
12

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de apache/arrow-dotnet

Todos los issues de apache/arrow-dotnet

Issues similares

Más issues de C#

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.