Breaking change in Avro 1.12.1 affects parquet-avro users: logical types now deserialize to different Java types
还没有人认领这个 Issue。
评估
- 难度
- 5/5
- 预计耗时
- 一周以上
- 新手友好度
- 25/100
- Issue 类型
- 缺陷
- 描述清晰度
- 需要澄清
- 活跃度
- 冷清
- 技术栈
- java
调研方向
从 issue 的 AVRO-3989 链接以及此处描述的 ReflectData、SpecificData 和 GenericRecord 行为开始;没有指定 parquet-java 文件或测试。比较 Avro 1.12.0 和 1.12.1 对所列逻辑类型的处理方式,然后与 maintainers 确定预期的更改或文档,再定义完成的标准。
由索引模型根据 Issue 内容生成。
描述
Summary
Avro 1.12.1 introduces a breaking change that affects parquet-avro users. Multiple logical types are now deserialized to their proper Java types instead of their underlying primitive types when using ReflectData, causing ClassCastException in existing code.
Background
- Upstream issue: https://issues.apache.org/jira/browse/AVRO-3989
- Pull request: https://github.com/apache/avro/pull/3354
- Avro version affected: 1.12.1 (released as a patch version)
Problem
The fix in AVRO-3989 changed the deserialization behavior for ALL logical types. When using within parquet-avro ReflectData or SpecificData classes, these fields now return their logical Java type objects instead of their underlying primitive types from GenericRecord.
Affected logical types and conversions:
| Logical Type | Old Type (≤1.12.0) | New Type (1.12.1) |
|---|---|---|
timestamp-millis |
Long |
Instant |
timestamp-micros |
Long |
Instant |
date |
Integer |
LocalDate |
time-millis |
Integer |
LocalTime |
time-micros |
Long |
LocalTime |
local-timestamp-millis |
Long |
LocalDateTime |
local-timestamp-micros |
Long |
LocalDateTime |
uuid |
String |
UUID |
decimal |
ByteBuffer |
BigDecimal |
This breaks existing code that expects primitive values:
// This used to work in Avro 1.12.0
Long timestamp = (Long) genericRecord.get("timestamp_field");
Integer date = (Integer) genericRecord.get("date_field");
String uuid = (String) genericRecord.get("uuid_field");
// In Avro 1.12.1, all of these throw ClassCastException
// because values are now Instant, LocalDate, UUID, etc.
Impact
- Widespread breakage: Affects anyone using parquet-avro with ANY logical types
- Code that has worked for years suddenly breaks after a patch version update
- The breaking nature wasn't clearly documented in Avro's changelog
- This is a significant behavior change that affects the entire logical type system
Heads up for future avro updates
I apologize if this isn't the right place to report this, please feel free to redirect me if there's a better venue for this discussion. My main goal is to ensure that when parquet-java does consider updating Avro, the team and users are aware of this significant behavioral change.
For Current Users
If you're using parquet-avro and explicitly upgrading Avro to 1.12.1+ in your project, be aware of this breaking change. You'll need to update code that reads logical type fields from GenericRecord.
- 主要语言
- Java
- 星标
- 3.1k
- 派生
- 1.6k
- 平均合并
- 4 天 12 小时
- 30 天内合并 PR
- 28
贡献指南
这个仓库没有索引到贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
apache/parquet-java 的其他 Issue
-
Type: bug
难度 2/5 1-3 小时 新手友好度 68/100
apache/parquet-java#3792 ·
-
难度 2/5 1-3 小时 新手友好度 82/100
apache/parquet-java#3767 ·
-
难度 2/5 1-3 小时 新手友好度 72/100
apache/parquet-java#3695 · 1 条评论 ·
-
难度 2/5 1-3 小时 新手友好度 78/100
apache/parquet-java#3667 ·
-
Type: bug
难度 2/5 1-3 小时 新手友好度 76/100
apache/parquet-java#3587 ·
查看 apache/parquet-java 的全部 Issue
相似的 Issue
-
awaiting triage bug Causes friction Hop Gui P1 P2 Transforms
难度 2/5 1-3 小时 新手友好度 75/100
-
难度 2/5 1-3 小时 新手友好度 75/100
apache/flink-agents#1152 ·
-
难度 2/5 1-3 小时 新手友好度 75/100
-
难度 2/5 1-3 小时 新手友好度 70/100
jenkinsci/blueocean-plugin#5417 ·
-
难度 2/5 1-3 小时 新手友好度 75/100
objectionary/eo-graphs#75 ·