Writer: integer cascade choices differ from Rust's on real columns (run-end, nullable bit-packing), 1.2–1.8x larger
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
调研方向
The payload gives six columns with Rust and Java sizes and encoding trees, and each one is a reproducer. Start with one column, such as RefererRegionID, and use the encoding-tree interop tests, which take a column shape and compare trees, to see where the Java cascade chooses run-end or bit-packing differently. Done means the Java tree matches Rust's for that shape and the output size is close to Rust's. Confirm first whether the gap comes from #649 or from chunk count.
由索引模型根据 Issue 内容生成。
描述
vortex-java's Parquet import vs the vortex-jni-written file, data bytes; the trees are each side's two most common chunk encodings:
| dataset | column | type | Rust | Java | Rust tree | Java tree |
|---|---|---|---|---|---|---|
| clickbench-hits | RefererRegionID | I32 | 68.1 MB | 124.7 MB (1.83x) | 171x bitpacked(patches), 12x runend |
274x bitpacked(patches), 39x runend |
| clickbench-hits | FUniqID | I64 | 255.4 MB | 358.3 MB (1.40x) | 289x bitpacked, 176x primitive |
365x runend(for(bitpacked), primitive) |
| clickbench-hits | UserID | I64 | 274.5 MB | 366.4 MB (1.33x) | 433x primitive, 116x runend(bitpacked, primitive) |
367x runend(for(bitpacked), primitive) |
| bi-generico | Hora_Pagina | I32? | 207.3 MB | 269.1 MB (1.30x) | 158x primitive, 109x bitpacked |
371x bitpacked(bool) |
| bi-generico | Disco | I16? | 92.8 MB | 119.9 MB (1.29x) | 159x runend(for(bitpacked), bitpacked) |
183x bitpacked(bool) |
| bi-generico | Codigo | I32? | 249.8 MB | 289.4 MB (1.16x) | 272x primitive |
263x bitpacked(patches) |
Java picks run-end where Rust keeps bit-packed/primitive (and vice versa), and its nullable chunks pay a vortex.bool validity on every bit-packed chunk. Likely several sampling/estimate differences in the cascade; each column above is a reproducer to bisect (the encoding-tree interop tests take a column shape and compare trees). Some of RefererRegionID may be #649 or chunk count (Java writes more chunks of these columns).
- 主要语言
- Java
- 星标
- 3
- 派生
- 2
- 平均合并
- 9 小时 55 分钟
- 30 天内合并 PR
- 126
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
dfa1/vortex-java 的其他 Issue
-
bug
难度 2/5 1-3 小时 新手友好度 66/100
dfa1/vortex-java#636 ·
维护者通常 1 天内回复
-
enhancement
难度 4/5 3-5 天 新手友好度 38/100
dfa1/vortex-java#660 ·
维护者通常 1 天内回复
-
bug
难度 4/5 3-5 天 新手友好度 35/100
dfa1/vortex-java#657 · 1 条评论 ·
维护者通常 1 天内回复
-
bug
难度 3/5 1-2 天 新手友好度 70/100
dfa1/vortex-java#655 ·
维护者通常 1 天内回复
-
Writer: string columns written as per-chunk dicts where Rust uses its dict layout (up to 2x larger)未关闭perf
难度 4/5 3-5 天 新手友好度 30/100
dfa1/vortex-java#650 ·
维护者通常 1 天内回复
相似的 Issue
-
Type: Bug
难度 2/5 1-3 小时 新手友好度 76/100
维护者通常 1 天内回复
-
MCP add-on: JSON responses with non-ASCII characters are truncated (wrong Content-Length)可能已有人在做 @dhanushbs10 今天认领。 未关闭add-on bug in:mcp
难度 2/5 1-3 小时 新手友好度 78/100
维护者通常 1 天内回复
-
vaadin-map
难度 2/5 1-3 小时 新手友好度 70/100
vaadin/flow-components#10586 ·
维护者通常 1 天内回复
-
WeReadInstaller writes the launcher database on the main thread at handover (Activity pause timeout)未关闭bug good first issue
难度 2/5 1-3 小时 新手友好度 70/100
-
难度 2/5 1-3 小时 新手友好度 63/100
wpilibsuite/SystemcoreTesting#446 ·
维护者通常 1 天内回复