Timestamp handling is broken for BCE dates

Open
#200 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
java
Domain
databases

Research direction

Start by tracing the DuckDB JDBC driver's internal uses of java.sql.Timestamp and java.sql.Date for parsing, formatting, and conversion to and from java.time.LocalDateTime. Reproduce the BCE example and verify that client-provided and returned Timestamp values use UNIX millisecond offsets while BCE dates round-trip correctly.

Written by the indexing model from the issue text.

Description

Noticed some discrepancies when using negative year values in tests and tried to look deeper:

  • historically Java has 2 different date-time APIs - java.util.Date + java.util.Calendar from Java 1.1 and java.time from Java 8
  • JDBC API mandates the usage of java.sql.Timestamp (that is thin wrapper over java.util.Date) for passing dates to DB and reading dates from DB
  • in DuckDB JDBC java.sql.Timestamp is currently used for parsing dates, formatting dates and converting it to-from java.time.LocalDateTime
  • apparently java.sql.Timestamp is completely broken for BCE dates and cannot be used neither for parsing/formatting, nor for converting to/from java.time.LocalDateTime, the following behaviour cannot be correct:
java.time.LocalDateTime ldt = java.time.LocalDateTime.of(-490, 9, 12, 9, 0, 0);
System.out.println("Original LocalDateTime: " + ldt);
// convert to timestamp field-by-field
java.sql.Timestamp ts1 = java.sql.Timestamp.valueOf(ldt);
System.out.println("Timestamp converted field-by-field: " + ts1);
// convert to timestamp using UNIX time offset in milliseconds
java.sql.Timestamp ts2 = new java.sql.Timestamp(ldt.toInstant(ZoneOffset.UTC).toEpochMilli());
System.out.println("Timestamp converted using UNIX time offset: " + ts2);
// convert back to LocalDateTime - year value is in CE now
java.time.LocalDateTime ldt1 = ts1.toLocalDateTime();
System.out.println("LocalDateTime converted back: " + ldt1);
System.out.println(ldt.getYear() == ldt1.getYear());
Original LocalDateTime: -0490-09-12T09:00
Timestamp converted field-by-field: 0491-09-12 09:00:00.0
Timestamp converted using UNIX time offset: 0491-09-17 09:00:00.0
LocalDateTime converted back: 0491-09-12T09:00
false

Apparently nobody uses java.sql.Timestamp in practice (except JDBC drivers) and this problem is a known bug in JDK.

This may be a low priority (CE dates work correctly), but users can realistically load some datasets with historic dates.

To fix this in DuckDB driver, all internal usage of java.sql.Timestamp (and java.sql.Date) functionality needs to be replaced with java.time usage. And java.sql.Timestamp values (coming from client app, or returned to client app) need to be treated only as UNIX time milliseconds offset holders.

Dominant language
C++
Stars
127
Forks
80
Avg merge
13h 41m
Merged PRs (30d)
44

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from duckdb/duckdb-java

All issues in duckdb/duckdb-java

Similar issues

More C++ issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.