[Bug] Mosaic accepts TIMESTAMP(0) columns that its native writer cannot write
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 76/100
Research direction
Read MosaicFileFormat.validateDataFields and the existing checks for unsupported types, then trace how ArrowFieldTypeConversion maps timestamp precision. Add coverage for TIMESTAMP(0) and TIMESTAMP(0) WITH LOCAL TIME ZONE in the relevant tests, and confirm table creation rejects these types with a clear message before a write is attempted.
Written by the indexing model from the issue text.
Description
Search before asking
- I searched in the issues and found nothing similar.
Paimon version
master @ 33dbcfcb17
Compute Engine
Engine-agnostic (Mosaic file format).
Minimal reproduce step
Create a table with 'file.format' = 'mosaic' and a TIMESTAMP(0) or TIMESTAMP(0) WITH LOCAL TIME ZONE column, then insert a row. CREATE TABLE succeeds, but every write fails when the file is opened:
java.lang.RuntimeException: writer open failed: unsupported Timestamp unit: Second
ArrowFieldTypeConversion maps precision 0 to the Arrow SECOND unit, and the Mosaic native writer only accepts millisecond, microsecond and nanosecond timestamps (validate_data_type in apache/paimon-mosaic core/src/types.rs). Precisions 1 to 9 write fine.
MySQL CDC maps DATETIME without a precision to TIMESTAMP(0), so a CDC job into a Mosaic table hits this on its first write.
What doesn't meet your expectations?
MosaicFileFormat.validateDataFields should reject TIMESTAMP(0) the way it already rejects VARIANT, BLOB, VECTOR, MULTISET and ROW, so the table definition fails with a message that names the type, instead of every write failing later with a native error that names neither the column nor the format.
Anything else?
Storing precision 0 as milliseconds would make the type writable, but it needs matching changes on the write and read sides of the Arrow conversion. Rejecting it is the smaller fix.
Are you willing to submit a PR?
- I'm willing to submit a PR!
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 365
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/paimon
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
apache/paimon#10332 · 1 comment ·
Maintainers usually reply within 1 day
-
[Bug] Hive query on Paimon table fails with 'Proxy user is not supported' when Kerberos proxy user is usedPossibly taken @sohurdc claimed this 88 days ago. Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
apache/paimon#9006 · 1 comment ·
Maintainers usually reply within 1 day
-
[Bug] FileIndexFormat.Reader leaks input stream when construction fails with a runtime exceptionMay be free again A pull request for this issue was closed without being merged. Open
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Maintainers usually reply within 1 day
-
[Bug] Bitmap benchmarks do not close seekable input streamsMay be free again A pull request for this issue was closed without being merged. Open
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Maintainers usually reply within 1 day
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
floci-io/floci#8791 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
Maintainers usually reply within 1 day
-
affects/8.10 affects/8.8 affects/8.9 area/reliability component/zeebe kind/bug likelihood/low severity/high
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
geonetwork/geonetwork#276 ·
Maintainers usually reply within 1 day