ANSI NaN and infinity use the wrong exception for cast to timestamp
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 68/100
Research direction
The issue names the error class InvalidInputInCastToDatetime, which already maps to Spark's SparkDateTimeException. Start by finding where the NaN and infinity cast-to-timestamp path under ANSI mode picks SparkNumberFormatException instead, and reuse the existing mapping. The payload names no files, so search for InvalidInputInCastToDatetime and for ANSI cast tests. Done when a NaN or infinite double cast to TIMESTAMP under ANSI throws SparkDateTimeException with Spark's message format.
Written by the indexing model from the issue text.
Description
With ANSI on, NaN and infinities raise SparkNumberFormatException (CastInvalidValue) with text like The value 'inf' of the type "STRING". Spark throws SparkDateTimeException: The value Infinity of the type "DOUBLE" cannot be cast to "TIMESTAMP". The error class and SQLSTATE match. InvalidInputInCastToDatetime already maps to Spark's exception. The source type, the spelling (inf versus Infinity), and the quoting differ.
This predates and is unchanged by #6969; found in review there.
- Dominant language
- Scala
- Stars
- 1.3k
- Forks
- 391
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 421
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/datafusion-comet
-
Extract shared list-array construction from the split_* functions in split.rsPossibly taken @arminazimi claimed this 1 day ago. Openarea:expressions enhancement good first issue help wanted requires-triage rust tech debt
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
apache/datafusion-comet#6983 ·
Maintainers usually reply within 1 day
-
Deduplicate spark_ceil and spark_floor and port floor's decimal boundary tests to ceilPossibly taken @med9110 claimed this 1 day ago. Openarea:expressions enhancement help wanted requires-triage rust tech debt
Difficulty 2/5 Half a day Newbie friendliness 68/100
apache/datafusion-comet#6974 · 1 comment ·
Maintainers usually reply within 1 day
-
Iceberg data tables whose names end with a metadata-table name fall back to SparkMay be free again @aryan9948 claimed this 10 days ago, and no pull request is open. Openrequires-triage
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
apache/datafusion-comet#6719 · 1 comment ·
Maintainers usually reply within 1 day
-
ANSI integral SUM overflow reports "integer overflow" without Spark's try_add suggestionPossibly taken @0lai0 claimed this 10 days ago. Openarea:aggregation bug priority:medium
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
apache/datafusion-comet#6661 · 1 comment · 1 assignee ·
Maintainers usually reply within 1 day
-
Native from_json with a schema that repeats a field name fails with "ArrowArray struct has 2 children (expected 1)"Possibly taken @Samearth17 claimed this 11 days ago. Openarea:expressions area:ffi bug good first issue priority:medium
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
apache/datafusion-comet#6592 ·
Maintainers usually reply within 1 day
All issues in apache/datafusion-comet
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
scylladb/scylla-migrator#413 ·
-
Difficulty 2/5 Under an hour Newbie friendliness 70/100
-
dependencies
Difficulty 1/5 Under an hour Newbie friendliness 78/100
apache/pekko-http#1354 ·
Maintainers usually reply within 1 day
-
[Bug] HiveDialect shares its colspecs dict with SQLAlchemy's DefaultDialect (python/pyhive)Possibly taken @rusackas claimed this 1 day ago. Open
Difficulty 1/5 Under an hour Newbie friendliness 80/100
apache/kyuubi#7786 · 1 comment ·
Maintainers usually reply within 2 days
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
Maintainers usually reply within 2 days