[JS] Regression in roundtripping JS Date objects from apache-arrow@15 to apache-arrow@16
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 25/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- javascript, typescript
- Domain
- data
Research direction
Start with the “Create a Table from JavaScript arrays” documentation example and reproduce it under apache-arrow@15.0.2 and @16.0.0 on Node 20. Review PR apache/arrow#40892 and the reported duckdb-wasm issue to understand the mapping change. Done means the intended Date roundtrip behavior is decided and either addressed or documented.
Written by the indexing model from the issue text.
Description
Describe the bug, including details regarding any error messages, version, and platform.
The Create a Table from JavaScript arrays example from the documentation is sufficient:
import { tableFromArrays } from 'apache-arrow';
const LENGTH = 2000;
const rainAmounts = Float32Array.from(
{ length: LENGTH },
() => Number((Math.random() * 20).toFixed(1)));
const rainDates = Array.from(
{ length: LENGTH },
(_, i) => new Date(Date.now() - 1000 * 60 * 60 * 24 * i));
const rainfall = tableFromArrays({
precipitation: rainAmounts,
date: rainDates
});
console.table([...rainfall]);
When running this code with apache-arrow@15.0.2 installed, the date column in the resulting table is a column of JavaScript Date objects. With apache-arrow@16.0.0 installed, it is a column of JavaScript numbers representing epoch timestamps. This is a surprising result, I think; while there's no reason why Arrow would be required to map Date objects to something which maps back to Date objects, it is an unusual choice which is not documented and which has clearly been changed recently.
Tested under Node 20.15.0 on Apple Silicon, but I have no reason to believe this is platform-specific.
PR apache/arrow#40892 appears to have gone through some iteration after I tested it in https://github.com/duckdb/duckdb-wasm/issues/1231, and these follow-up changes caused this regression. Perhaps it's desirable—but I'm not quite convinced this was the correct outcome? Feel free to close as by design (but I think it would be worth documenting all of the. type mappings going from JS to Arrow and back, at the very least).
Component(s)
JavaScript
- Dominant language
- TypeScript
- Stars
- 112
- Forks
- 23
- Avg merge
- 17h 55m
- Merged PRs (30d)
- 10
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/arrow-js
-
Difficulty 3/5 1-2 days Newbie friendliness 68/100
-
Difficulty 3/5 1-2 days Newbie friendliness 55/100
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
-
Difficulty 3/5 1-2 days Newbie friendliness 50/100
Similar issues
-
blocklist removal
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
MetaMask/eth-phishing-detect#296544 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
pastelsky/bundlephobia#1122 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
-
category/development priority/P2 scope/file-operations scope/testing type/enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Enatega Customer and Rider app: Add-ons price is not visible to customer after order is placed. Open
Difficulty 2/5 1-3 hours Newbie friendliness 75/100