Timestamp get() throws for valid data outside Number.MAX_SAFE_INTEGER range
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 35/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- javascript, typescript
Research direction
Start in visitor/get.mjs and trace the timestamp visitors alongside bigIntToNumber(), divideBigInts(), and the existing Int64/Uint64 visitors. Resolve the proposed get() return contract and breaking-change approach before editing; done means valid extreme and high-precision timestamps no longer throw or lose their raw integer value.
Written by the indexing model from the issue text.
Description
Summary
Vector.get(i) on Timestamp columns throws TypeError for valid timestamp values whose microsecond representation exceeds Number.MAX_SAFE_INTEGER. This makes large portions of the valid timestamp range inaccessible through Arrow JS's public API.
Root cause
The timestamp get visitors in visitor/get.mjs convert BigInt64Array values to Number via bigIntToNumber() / divideBigInts():
const getTimestampSecond = ({ values }, index) => 1000 * bigIntToNumber(values[index]);
const getTimestampMillisecond = ({ values }, index) => bigIntToNumber(values[index]);
const getTimestampMicrosecond = ({ values }, index) => divideBigInts(values[index], BigInt(1000));
const getTimestampNanosecond = ({ values }, index) => divideBigInts(values[index], BigInt(1000000));
bigIntToNumber() throws if the value is outside [-MAX_SAFE_INTEGER, MAX_SAFE_INTEGER]:
export function bigIntToNumber(number) {
if (typeof number === 'bigint' && (number < Number.MIN_SAFE_INTEGER || number > Number.MAX_SAFE_INTEGER)) {
throw new TypeError(`${number} is not safe to convert to a number.`);
}
return Number(number);
}
Impact
For Timestamp<MICROSECOND> (DuckDB's default, and common in Parquet files):
- Accessible range: 1970-04-26 to 2255-06-05 (~285 years)
- Valid DuckDB range: 290,309 BC to 294,247 AD (~584,000 years)
- Infinity sentinels (
INT64_MAX,INT64_MIN+1): completely inaccessible
This means get() throws — not returns null, not returns a sentinel, but throws an exception — for valid data that was correctly serialized and deserialized through IPC.
For Timestamp<NANOSECOND>, the accessible range is even narrower.
The conversion is also lossy within range. divideBigInts(value, 1000n) divides BigInt microseconds into a float, losing sub-millisecond precision. A microsecond timestamp of 1652397825000001n returns 1652397825000.001 — correct here, but precision degrades as values grow.
Comparison with other Arrow implementations
- Arrow C++: Returns
int64_t. Caller interprets. - Arrow Python: Returns raw int via
.as_py()on scalar, or numpy int64 in arrays.to_pylist()preserves int64. - Arrow Rust: Returns
i64. - Arrow JS: Converts to
Number, throws if out of range.
Arrow JS is the only implementation where get() can throw on valid data.
Note that visitInt64 and visitUint64 already return raw BigInts:
const getBigInts = ({ values }, index) => values[index];
The timestamp visitors are inconsistent with this — they convert instead of returning the raw value.
Proposal
Return raw BigInt from timestamp visitors, matching Int64/Uint64 behavior:
const getTimestampSecond = ({ values }, index) => values[index];
const getTimestampMillisecond = ({ values }, index) => values[index];
const getTimestampMicrosecond = ({ values }, index) => values[index];
const getTimestampNanosecond = ({ values }, index) => values[index];
The raw BigInt preserves full precision and never throws. Callers that want a JS Date or millisecond Number can convert explicitly.
For convenience, a toDate(index) method or utility function could provide the current behavior for callers in the common case (recent dates, millisecond precision is fine):
function timestampToDate(vector, index) {
const raw = vector.get(index);
if (raw === null) return null;
const ms = Number(raw / 1000n); // microseconds to ms, lossy but fine for Date
return new Date(ms);
}
Similarly, getDateDay should return raw int32 days rather than converting to epoch milliseconds, since the millisecond conversion loses precision for dates far from epoch.
Breaking change
This is a breaking change for code that assumes get() on a timestamp column returns a Number. The alternatives are:
- Breaking change in a major version — cleanest, aligns with C++/Python/Rust
- Opt-in flag (e.g.,
{ useBigInt: true }on reader options) — backwards-compatible but adds API surface - Return BigInt only when the value is outside safe range — surprising behavior difference based on data values
I'd advocate for option 1. Returning BigInt is consistent with how Int64/Uint64 already behave, and the current behavior (throwing) is arguably already broken — callers can't rely on get() succeeding, which defeats the purpose of a typed API.
Context
We hit this building a DuckDB WASM shell that displays test_all_types() output. Lists containing timestamps with infinity sentinels, extreme-range timestamps, and nanosecond-precision timestamps all trigger this. Our workaround reads raw BigInt64Array values from column.data[0].values, bypassing Arrow's getter entirely — which works but defeats the purpose of the Vector abstraction.
Related issues: #91 (timestamp types all return same values), #83 (pre-epoch timestamp offset errors), #77 (decimal conversion errors). These are all symptoms of the same underlying design: eagerly converting int64 → Number in get visitors.
- Dominant language
- TypeScript
- Stars
- 112
- Forks
- 23
- Avg merge
- 17h 55m
- Merged PRs (30d)
- 10
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/arrow-js
-
Difficulty 3/5 1-2 days Newbie friendliness 68/100
-
Difficulty 3/5 1-2 days Newbie friendliness 55/100
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
-
Difficulty 3/5 1-2 days Newbie friendliness 50/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
mksglu/context-mode#1200 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
jaegertracing/jaeger-ui#4506 ·
-
area:desktop area:ui bug platform:macos
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
anthropics/claude-code#96687 ·
-
good first issue
Difficulty 1/5 Under an hour Newbie friendliness 95/100
AOSSIE-Org/DebateAI#582 · 2 comments ·