perf(writer, core): optional Vector API candidates on the write path and in shared helpers
メンテナーはふだん 1 日以内に返信
まだ誰も着手していません。
評価
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 初心者へのやさしさ
- 22/100
- issue の種類
- リファクタリング
- 明瞭さ
- おおむね明確
- 活発さ
- 活発
- 技術スタック
- java
- 領域
- data, performance
調査の方向性
Start with PrimitiveArraysBenchmark and the JMH baselines the issue asks for, before any Vector API code. The one plain-Java item that could ship alone is hoisting the per-element switch in PTypeIO.set out of the fromLongs narrowing loop, with one loop per width. Done means the baselines are recorded on both machines, the narrowing loops are specialized, and the output is byte-identical to the scalar path.
索引モデルが issue の本文から書いたものです。
説明
Companion to the reader-side issue (fused filter-aggregate, Delta undelta). Same ground rules: opt-in, scalar default, hardwood-style link-failure fallback, no DecodeContext/WriteOptions flag, and ADR 0005's bar (C2 can't do it, >= 10% on two architectures, hot in a real profile). Nothing here is measured yet. This issue lists where to measure first.
Prior evidence we must respect
BitpackedEncodingEncoder already records that a fused scalar pass beat a separately vectorized min/max pass ("the loop is memory-bound ... measured, #475"). So any candidate that merely re-adds a standalone min/max/histogram reduction starts with negative evidence. Candidates below are ranked by compute density per byte loaded, which is what the Vector API can actually win on.
Writer candidates
W1. FastLanes bit-pack (BitpackedEncodingEncoder, the block / lane / row nest around line 210)
Per block, per lane, the loop walks typeBits rows, shifting each value into an accumulator and flushing a word when filled >= typeBits:
for (lane...) { acc = 0; shift = 0; word = 0;
for (row...) { value = values[blockStart + rowOffset[row] + lane] & widthMask;
acc |= value << shift;
if (filled >= typeBits) { words[word*lanes+lane] = acc & typeMask; ... carry ... } } }
Why it fits: idx = blockStart + rowOffset[row] + lane, so adjacent lanes are contiguous. The branch filled >= typeBits depends only on row (and bitWidth), not on lane, so it is uniform across the vector and becomes a scalar branch hoisted out of the lane dimension. With lane innermost, acc/shift-by-row become per-row scalars and acc becomes a LongVector (LongVector.fromArray(values, idx), .and(widthMask), .lanewise(LSHL, shift), .or). C2 can't do this today because lane is the outer loop and the acc dependency runs along the inner row loop.
Do first (plain Java): interchange to row-outer, lane-inner with long[] acc = new long[lanes]. The inner loop is then uniform and contiguous, and C2 superword may take it. Only if it doesn't does a LongVector version earn its keep.
Caveat: the fullBlock || idx < n tail guard is the only non-uniform part. Keep the full-block fast path and leave the partial tail block scalar (the existing branch-split idiom).
Why this is the best writer target: unlike min/max, bit-packing does shifts, ORs and a store per element, so it's compute-bound rather than purely memory-bound, and it sits in the write path for every integer column the cascade tries bitpacking on (write was the 0.44 to 0.88 ops/s gap vs vortex-jni, see the cascade-write stack).
W2. ALP encode (AlpEncodingEncoder#computeF64 / computeF32, the per-value encodeF64 + verify)
Per value: multiply by two table constants, add and subtract the sweet-spot magic number, cast to long, decode again, compare raw bits for the exception check.
long encoded = (long) ((v * F10[e] * IF10[f] + SWEET) - SWEET);
if (doubleToRawLongBits(encoded * F10[f] * IF10[e]) != doubleToRawLongBits(v)) { patch... }
Why it fits: straight-line FP math with a single rare exception. The exception path is the only non-uniform part: compute the mismatch mask for a vector, and if (!mask.anyTrue()) store; else scalar fallback for those lanes is the standard pattern, and patches are rare by construction (ALP is chosen only when it mostly round-trips). DoubleVector.convert(D2L, 0) and reinterpretAsLongs() cover the cast and the bit compare.
Parity risk (prime directive): results must be bit-identical to Rust's encode_single / decode_single. The two-step multiply order (* F10[f] * IF10[e], never a pre-multiplied scale) is already protected in the reader's javadoc. The vector path must do the same two multiplies, in the same order, with no FMA contraction. Java doesn't fuse implicitly, but DoubleVector.fma must not be used here. Add a differential test over seeded-random doubles plus edge cases (-0.0, NaN, +-Inf, subnormals, Double.MAX_VALUE, values at exactly the sweet-spot boundary).
The exponent search (for e..., for f... over the 1024-value sample) is ~e^2/2 passes over a tiny array and is a cheaper first target for the same kernel: it is dominated by exactly this encode-and-verify arithmetic.
W3 (low priority, measure only). FrameOfReferenceEncodingEncoder#computeRef and ZoneMapStats.ofDecimal(long[])
Standalone min scans (computeRef) and a min+max scan (ofDecimal) are textbook reduceLanes(MIN/MAX) shapes, but see the #475 evidence above. Worth a single JMH row to confirm or refute, not a commitment. The 128/256-bit decimal ofDecimal(MemorySegment, byteWidth) is a multi-word compare and is not a candidate.
Shared helpers (core.compute.PrimitiveArrays, PTypeIO)
The existing PrimitiveArraysBenchmark (widen, narrow) is the harness. Before writing any Vector API, measure what C2 already does.
| Helper | Shape | Note |
|---|---|---|
toLongs(Object, ...) |
heap array to long[], sign or zero extension, one specialized loop per ptype |
Likely already auto-vectorized (simple widening of a heap array). Measure with -prof perfasm or PrintAssembly or JFR before touching. convertShape(B2L / ZERO_EXTEND_B2L) would be the Vector API form |
toLongsInto(MemorySegment, ...) |
LE MemorySegment to long[], same shape |
Segment accessors with int loop var and long offset: check C2 vectorizes it (long-offset index math can defeat superword). If not, LongVector/IntVector.fromMemorySegment(..., LE) plus convert is a direct fit. This helper is called per FastLanes chunk by the Delta decoder, so it sits on a real read path |
fromLongs(long[], ptype, arena) (narrowing branch) |
per-element PTypeIO.set(seg, off, ptype, v) |
Not a Vector API problem. PTypeIO.set is a MethodHandle[] lookup + invokeExact per element with the ptype switch inside the loop, which is exactly what the hot-loop rule forbids (SETTERS[ptype.ordinal()] is variable-target). Fix: hoist the switch and write one specialized loop per width (as toLongsInto already does). Used by DeltaEncodingEncoder for bases and deltas. Plain-Java win, no flag, no ADR gate |
That last row is the cheapest, safest item in this whole issue and could ship independently of any Vector API decision.
Packaging note (applies to W1/W2 and helpers)
Don't scatter jdk.incubator.vector imports across encoders. Mirror hardwood's seam: one small SimdKernels-style interface per module boundary (writer kernels in writer, shared widening in core), scalar implementation always present, a vector implementation selected by a guarded static initializer that catches Throwable (NoClassDefFoundError/UnsupportedOperationException when the module is absent, or SPECIES_PREFERRED too narrow). Writer must not depend on reader, so there will be two small seams rather than one shared class, or the seam lives in core if both sides need it.
Acceptance
- JMH baselines (
-f 3,ClassName.methodName): bit-pack, ALP encode (incl. exponent search),toLongsInto/fromLongs, on M5 and one x86 machine -
fromLongsnarrowing hoisted to per-width loops (independent, plain Java) - Bit-pack loop interchange tried before any Vector API
- Vector kernel for a candidate only if >= 10% over the best scalar on both machines
- Differential tests vs the scalar path, byte-identical output, including ALP edge values
- ADR 0005 and
docs/reference.mdupdated in the same commit
- 主要言語
- Java
- スター
- 3
- フォーク
- 2
- 平均マージ
- 9時間 51分
- マージ済み PR(30日)
- 127
環境構築
- Dockerfile・Docker Compose ファイルなし
- プルリクエストのテンプレートなし
- コントリビューションガイドを読む
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
dfa1/vortex-java のほかの issue
-
bug
難易度 2/5 1〜3時間 初心者へのやさしさ 66/100
dfa1/vortex-java#636 ·
メンテナーはふだん 1 日以内に返信
-
bug
難易度 4/5 3〜5日 初心者へのやさしさ 35/100
dfa1/vortex-java#657 ·
メンテナーはふだん 1 日以内に返信
-
bug
難易度 3/5 1〜2日 初心者へのやさしさ 70/100
dfa1/vortex-java#655 ·
メンテナーはふだん 1 日以内に返信
-
perf
難易度 4/5 3〜5日 初心者へのやさしさ 30/100
dfa1/vortex-java#651 ·
メンテナーはふだん 1 日以内に返信
-
Writer: string columns written as per-chunk dicts where Rust uses its dict layout (up to 2x larger)オープンperf
難易度 4/5 3〜5日 初心者へのやさしさ 30/100
dfa1/vortex-java#650 ·
メンテナーはふだん 1 日以内に返信
dfa1/vortex-java の issue をすべて見る
似ている issue
-
bug
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
apache/maven-enforcer#1074 ·
メンテナーはふだん 1 日以内に返信
-
Gradle liquibaseDiffChangelog drops jhi_date_time_wrapper対応中かも @mraible が今日担当しました。 オープンarea: triage theme: undefined
難易度 2/5 1〜3時間 初心者へのやさしさ 88/100
jhipster/generator-jhipster#35998 ·
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 78/100
openai/openai-java#1337 ·
メンテナーはふだん 1 日以内に返信
-
bug
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 75/100
checkstyle/checkstyle#22447 ·
メンテナーはふだん 1 日以内に返信