Hacktoberfest 2026：メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

IPC writer silently writes a corrupt stream for a Table from another copy of apache-arrow

オープン
  • 何か問題がありますか？
#496 コメント 1 件 リアクション 1 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

@kentkwu がすでに取り組んでいます。

2026年10月6日 から。

  • #500 @kentkwu による — オープン

評価

難易度
3/5
見積もり時間
1〜2日
初心者へのやさしさ
68/100
issue の種類
バグ
明瞭さ
明確に書かれている
活発さ
活発
技術スタック
node.js, typescript
領域
backend, data

調査の方向性

Start in src/visitor.ts at getVisitFn, where the issue identifies instanceof DataType checks as the cause; compare with DataType.isDataType() in src/type.ts. Add or update tests for cross-copy types and run the relevant visitor and IPC tests. Done means tables from separate CommonJS and ES module builds serialize with valid types and buffers and round-trip successfully.

索引モデルが issue の本文から書いたものです。

説明

[!NOTE]
I discovered this issue and wrote it up with help from Claude Opus 5.5.

Describe the bug, including details regarding any error messages, version, and platform.

Since #377 (released in 21.2.0), instanceof Table accepts a Table created by another copy of apache-arrow. tableToIPC then serializes such a table without any error, but the Arrow IPC stream it writes is unusable: field types are missing or wrong, and the dictionary and record batch messages have empty bodies. None of the data is written.

An easy way to get two copies is the package's own CommonJS and ES module builds. Node loads them separately when, for example, an ES module application passes a table to a CommonJS library. Bundlers can do the same through the import and require conditions.

Reproduction

apache-arrow 21.2.0, Node.js 24.8.0, macOS:

// repro.mjs
import { createRequire } from "node:module";
import * as esm from "apache-arrow";
const cjs = createRequire(import.meta.url)("apache-arrow");

const table = esm.tableFromArrays({ id: Int32Array.from([1, 2]), text: ["a", "b"] });
console.log(esm.Table === cjs.Table, table instanceof cjs.Table);  // false true

const crossBuild = cjs.tableToIPC(table, "stream");  // No error
const sameBuild = esm.tableToIPC(table, "stream");
console.log(crossBuild.length, sameBuild.length);  // 440 648

esm.tableFromIPC(crossBuild);
// TypeError: Cannot destructure property 'length' of '(intermediate value)(intermediate value)(intermediate value)' as it is undefined.
//     at .../apache-arrow/visitor/vectorloader.mjs:154

PyArrow 25.0.1 rejects the same crossBuild bytes with OSError: Invalid flatbuffers message. The sameBuild stream reads correctly in both.

Single-column tables fail in different ways. In each case the cross-build stream has empty message bodies:

Column type Same-build bodies Cross-build bodies PyArrow reading the cross-build stream
Int32 batch: 8 bytes batch: 0 bytes ArrowNotImplementedError: Integers with less than 8 bits not implemented
Float64 batch: 16 bytes batch: 0 bytes OSError: buffer_index out of range.
Utf8 batch: 24 bytes batch: 0 bytes OSError: buffer_index out of range.
Dictionary<Utf8, Int32> dictionary: 24, batch: 16 bytes dictionary: 0, batch: 0 bytes OSError: Unexpected null field DictionaryEncoding.indexType in flatbuffer-encoded metadata
Cause

#377 added DataType.isDataType() for checks across copies, and deliberately did not give DataType a Symbol.hasInstance. But getVisitFn still dispatches on node instanceof DataType and node.type instanceof DataType. A type from another copy fails both checks and falls through to Type.NONE. As a result, TypeAssembler writes no type, and no indexType for a dictionary, and the writer emits no buffers. Nothing reports an error.

Suggested fix

Use the cross-copy check in getVisitFn:

-    if (node && (node instanceof DataType)) {
+    if (node && DataType.isDataType(node)) {
         return getVisitFnByTypeId(visitor, inferDType(node), throwIfNotFound);
     }
-    if (node?.type && (node.type instanceof DataType)) {
+    if (node?.type && DataType.isDataType(node.type)) {
         return getVisitFnByTypeId(visitor, inferDType(node.type), throwIfNotFound);
     }

I applied the equivalent change to visitor.js in the CommonJS build of 21.2.0. With it, the cross-build write in the reproduction round-trips, and its output is byte-identical to a same-build write. I also checked a table with an Int32 column and a Dictionary<Utf8, Int16> column: identical bytes, and PyArrow reads the result.

It may also be worth making the writer throw when a type can't be visited, rather than writing an empty body.

Related issues

#61, #377, #56

主要言語
TypeScript
スター
112
フォーク
25
平均マージ
10時間 5分
マージ済み PR（30日）
14

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

apache/arrow-js のほかの issue

apache/arrow-js の issue をすべて見る

似ている issue

TypeScript の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。