[Bug]: AMS Spark type rendering loses Iceberg timestamp-without-zone semantics
Maintainers usually reply within 2 days
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 35/100
Research direction
Start with IcebergTypeToSparkType and the TestIcebergTypeToSparkType tests named in the issue. Reproduce the direct and nested timestamp-without-zone cases, then check that the zone-adjusted case still renders as timestamp. Done means the regressions pass with the expected timestamp_ntz and timestamp catalog strings; an open linked pull request is already addressing this issue.
Written by the indexing model from the issue text.
Description
What happened?
IcebergTypeToSparkType maps both Iceberg timestamp variants to Spark timestamp. The AMS schema display therefore loses the distinction between timestamp without a zone and a zone-adjusted timestamp.
Expected behavior: Map shouldAdjustToUTC=false to Spark TimestampNTZType and keep the zone-adjusted variant as TimestampType. Add direct and nested type rendering regressions.
Affects Versions
master at 061eb724e93592fe87cf44b61af34c1d799f5db6
Table formats / engines
Iceberg; AMS / Spark
How to reproduce
Evaluate TypeUtil.visit(Types.TimestampType.withoutZone(), new IcebergTypeToSparkType()).catalogString(). Expected timestamp_ntz; actual timestamp. The same mismatch occurs inside nested structs. Types.TimestampType.withZone() should remain timestamp.
Relevant log output
JDK 17, default Spark 3.5.8 profile: TestIcebergTypeToSparkType, 3 tests passed; 2 failed against the original implementation. Root ./mvnw -q -Pskip-dashboard-build validate passed. Spark 3.3/3.4 profile tests were not executed.
Are you willing to submit a PR?
- Yes
Code of Conduct
- I agree to follow the Apache Code of Conduct.
- Dominant language
- Java
- Stars
- 1.2k
- Forks
- 397
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 18
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/amoro
-
[CI] Core CI doesn't run for changes to amoro-format-lance (and amoro-web)Possibly taken @MarkAlex1234 claimed this 6 days ago. Open
Difficulty 1/5 Under an hour Newbie friendliness 88/100
Maintainers usually reply within 2 days
-
fix: low-contrast secondary text color in global theme tokenMay be free again A pull request for this issue was closed without being merged. Openstale
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
apache/amoro#4157 · 1 comment ·
Maintainers usually reply within 2 days
-
fix: mock api server enabled in production build masks real error/loading uxMay be free again A pull request for this issue was closed without being merged. Openstale
Difficulty 1/5 Under an hour Newbie friendliness 78/100
apache/amoro#4155 · 1 comment ·
Maintainers usually reply within 2 days
-
type:bug
Difficulty 4/5 3-5 days Newbie friendliness 45/100
Maintainers usually reply within 2 days
-
[Bug]: After 0.9, disabling self-optimizing leaves process RUNNING instead of CLOSEDPossibly taken @zhang-arvin claimed this 3 days ago. Opentype:bug
Difficulty 4/5 3-5 days Newbie friendliness 48/100
apache/amoro#4431 · 1 comment ·
Maintainers usually reply within 2 days
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
modelcontextprotocol/java-sdk#1174 · 1 reaction ·
Maintainers usually reply within 2 days
-
module: core
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
bigbluebutton/bigbluebutton#26028 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
MetricsHub/metricshub-community#1390 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
apache/rocketmq-dashboard#6331 ·
Maintainers usually reply within 5 days
-
status: team-only type: dependency-upgrade
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
spring-projects/spring-boot#52143 · 1 comment ·
Maintainers usually reply within 1 day