[C++] Reader cannot resolve Java fixed-offset writer timezone IDs such as GMT-00:00
Valutazione
- Difficoltà
- 3/5
- Tempo stimato
- 1-2 giorni
- Idoneità per principianti
- 68/100
- Tipo di issue
- Bug
- Chiarezza
- Abbastanza chiara
- Stato di attività
- Attiva
- Ambito
- data-engineering, databases
Direzione di ricerca
Start with c++/src/Timezone.cc, especially getTimezoneByName and getTimezoneByFilename, then run orc-contents against the supplied gmt-0000.orc fixture. Confirm that the reader accepts the GMT-00:00 writerTimezone and still handles normal zoneinfo names correctly.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
I was closing out some old Arrow issues and this one (apache/arrow#21471, from 2019) looks like it is potentially an ORC bug rather than an Arrow one. I don't understand this domain well myself, so everything below the line was produced by Claude working through the ORC and Arrow source with me. I have checked that the reprex fails against current main (4dbfe288) as described, but I have not independently verified the code reading, so please treat it accordingly. I have tried to keep it concise.
Cause
When the JVM's default zone is a fixed offset, the Java writer records a custom ID such as GMT-00:00 or GMT+05:30 as the stripe writerTimezone (WriterImpl.java:563 uses TimeZone.getDefault().getID()). The Java reader accepts these because TimeZone.getTimeZone parses GMT±hh:mm (TreeReaderFactory.java:1251).
The C++ reader resolves a writer timezone only by opening $TZDIR/<name> (getTimezoneByName in c++/src/Timezone.cc), plus the US/* alias map from ORC-2011. No such file exists for GMT-00:00, so any stripe with a timestamp column fails:
$ orc-contents gmt-0000.orc
Caught exception in gmt-0000.orc: Time zone file /usr/share/zoneinfo/GMT-00:00 does not exist. Please install IANA time zone database and set TZDIR env.
Adding a GMT-00:00 symlink to GMT under TZDIR makes the same file read fine, so the timezone name is the only problem with it.
Reprex
gmt-0000.orc (250 bytes) has one stripe, one timestamp column, uncompressed, with writerTimezone set to GMT-00:00. It was written with the C++ writer via pyorc (with TZDIR pointed at a directory containing a GMT-00:00 zone file) to reproduce the footer field that Hive produced in the original report.
gmt-0000.orc as base64 (decode with base64 -d)
T1JDCgYSBAgBUAAKHgoEAAAAABIWCAFKEBjA2u/vslogwNrv77JaMAFQAHYAD9kwyEAAAAoGCAYQABgICgYIBhABGCAKBggBEAEYBgoGCAUQARgDEgQIABAAEgQIAhAAGglHTVQtMDA6MDAKHgoECAFQAAoWCAFKEBjA2u/vslogwNrv77JaMAFQAAgDEGgaCggDECgYCSA3KAEiDwgMEgEBGgJ0cyAAKAAwACIICAkgACgAMAAwAToECAFQADoWCAFKEBjA2u/vslogwNrv77JaMAFQAECQTkgBYgUyLjEuNAhXEAAYgIAEIgIADCggMAaC9AMDT1JDFw==
Suggested fix (untested)
Recognise names matching GMT[+-]hh(:mm)? in getTimezoneByFilename and return a fixed-offset Timezone instead of loading a file. The Etc/GMT±n zoneinfo files are not a substitute since they only cover whole hours and use the inverted POSIX sign.
- Lingua principale
- Java
- Stelle
- 769
- Fork
- 515
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Preparare l'ambiente
- Nessun Dockerfile né file Docker Compose
- Ha un modello di pull request
- Nessuna guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di apache/orc
-
estimateRgEndOffset slop calculation is insufficient for incompressible dataForse già presa @thexiay l’ha presa 153 giorni fa. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
-
cross compileAperta
Difficoltà 4/5 3-5 giorni Idoneità per principianti 35/100
-
Difficoltà 5/5 Più di una settimana Idoneità per principianti 20/100
Issue simili
-
enhancement
Difficoltà 2/5 1-3 ore Idoneità per principianti 85/100
I maintainer di solito rispondono entro 1 giorno
-
waiting-for-triage
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 72/100
spring-cloud/spring-cloud-openfeign#1443 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 1/5 1-3 ore Idoneità per principianti 84/100
ADORSYS-GIS/keycloak-oid4vp-plugin#221 ·
I maintainer di solito rispondono entro 2 giorni
-
status: team-only type: dependency-upgrade
Difficoltà 2/5 1-3 ore Idoneità per principianti 65/100
spring-projects/spring-boot#52099 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 67/100
tchiotludo/akhq#3307 · 1 reazione ·
I maintainer di solito rispondono entro 1 giorno