perf(writer, core): optional Vector API candidates on the write path and in shared helpers
Maintainer antworten meist innerhalb von 1 Tag
Dieses Issue hat noch niemand übernommen.
Bewertung
- Schwierigkeit
- 4/5
- Geschätzter Aufwand
- 3-5 Tage
- Anfängerfreundlichkeit
- 22/100
- Issue-Typ
- Refactoring
- Klarheit
- Größtenteils klar
- Aktivitätsstatus
- Aktiv
- Tech-Stack
- java
- Bereich
- data, performance
Rechercherichtung
Start with PrimitiveArraysBenchmark and the JMH baselines the issue asks for, before any Vector API code. The one plain-Java item that could ship alone is hoisting the per-element switch in PTypeIO.set out of the fromLongs narrowing loop, with one loop per width. Done means the baselines are recorded on both machines, the narrowing loops are specialized, and the output is byte-identical to the scalar path.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Beschreibung
Companion to the reader-side issue (fused filter-aggregate, Delta undelta). Same ground rules: opt-in, scalar default, hardwood-style link-failure fallback, no DecodeContext/WriteOptions flag, and ADR 0005's bar (C2 can't do it, >= 10% on two architectures, hot in a real profile). Nothing here is measured yet. This issue lists where to measure first.
Prior evidence we must respect
BitpackedEncodingEncoder already records that a fused scalar pass beat a separately vectorized min/max pass ("the loop is memory-bound ... measured, #475"). So any candidate that merely re-adds a standalone min/max/histogram reduction starts with negative evidence. Candidates below are ranked by compute density per byte loaded, which is what the Vector API can actually win on.
Writer candidates
W1. FastLanes bit-pack (BitpackedEncodingEncoder, the block / lane / row nest around line 210)
Per block, per lane, the loop walks typeBits rows, shifting each value into an accumulator and flushing a word when filled >= typeBits:
for (lane...) { acc = 0; shift = 0; word = 0;
for (row...) { value = values[blockStart + rowOffset[row] + lane] & widthMask;
acc |= value << shift;
if (filled >= typeBits) { words[word*lanes+lane] = acc & typeMask; ... carry ... } } }
Why it fits: idx = blockStart + rowOffset[row] + lane, so adjacent lanes are contiguous. The branch filled >= typeBits depends only on row (and bitWidth), not on lane, so it is uniform across the vector and becomes a scalar branch hoisted out of the lane dimension. With lane innermost, acc/shift-by-row become per-row scalars and acc becomes a LongVector (LongVector.fromArray(values, idx), .and(widthMask), .lanewise(LSHL, shift), .or). C2 can't do this today because lane is the outer loop and the acc dependency runs along the inner row loop.
Do first (plain Java): interchange to row-outer, lane-inner with long[] acc = new long[lanes]. The inner loop is then uniform and contiguous, and C2 superword may take it. Only if it doesn't does a LongVector version earn its keep.
Caveat: the fullBlock || idx < n tail guard is the only non-uniform part. Keep the full-block fast path and leave the partial tail block scalar (the existing branch-split idiom).
Why this is the best writer target: unlike min/max, bit-packing does shifts, ORs and a store per element, so it's compute-bound rather than purely memory-bound, and it sits in the write path for every integer column the cascade tries bitpacking on (write was the 0.44 to 0.88 ops/s gap vs vortex-jni, see the cascade-write stack).
W2. ALP encode (AlpEncodingEncoder#computeF64 / computeF32, the per-value encodeF64 + verify)
Per value: multiply by two table constants, add and subtract the sweet-spot magic number, cast to long, decode again, compare raw bits for the exception check.
long encoded = (long) ((v * F10[e] * IF10[f] + SWEET) - SWEET);
if (doubleToRawLongBits(encoded * F10[f] * IF10[e]) != doubleToRawLongBits(v)) { patch... }
Why it fits: straight-line FP math with a single rare exception. The exception path is the only non-uniform part: compute the mismatch mask for a vector, and if (!mask.anyTrue()) store; else scalar fallback for those lanes is the standard pattern, and patches are rare by construction (ALP is chosen only when it mostly round-trips). DoubleVector.convert(D2L, 0) and reinterpretAsLongs() cover the cast and the bit compare.
Parity risk (prime directive): results must be bit-identical to Rust's encode_single / decode_single. The two-step multiply order (* F10[f] * IF10[e], never a pre-multiplied scale) is already protected in the reader's javadoc. The vector path must do the same two multiplies, in the same order, with no FMA contraction. Java doesn't fuse implicitly, but DoubleVector.fma must not be used here. Add a differential test over seeded-random doubles plus edge cases (-0.0, NaN, +-Inf, subnormals, Double.MAX_VALUE, values at exactly the sweet-spot boundary).
The exponent search (for e..., for f... over the 1024-value sample) is ~e^2/2 passes over a tiny array and is a cheaper first target for the same kernel: it is dominated by exactly this encode-and-verify arithmetic.
W3 (low priority, measure only). FrameOfReferenceEncodingEncoder#computeRef and ZoneMapStats.ofDecimal(long[])
Standalone min scans (computeRef) and a min+max scan (ofDecimal) are textbook reduceLanes(MIN/MAX) shapes, but see the #475 evidence above. Worth a single JMH row to confirm or refute, not a commitment. The 128/256-bit decimal ofDecimal(MemorySegment, byteWidth) is a multi-word compare and is not a candidate.
Shared helpers (core.compute.PrimitiveArrays, PTypeIO)
The existing PrimitiveArraysBenchmark (widen, narrow) is the harness. Before writing any Vector API, measure what C2 already does.
| Helper | Shape | Note |
|---|---|---|
toLongs(Object, ...) |
heap array to long[], sign or zero extension, one specialized loop per ptype |
Likely already auto-vectorized (simple widening of a heap array). Measure with -prof perfasm or PrintAssembly or JFR before touching. convertShape(B2L / ZERO_EXTEND_B2L) would be the Vector API form |
toLongsInto(MemorySegment, ...) |
LE MemorySegment to long[], same shape |
Segment accessors with int loop var and long offset: check C2 vectorizes it (long-offset index math can defeat superword). If not, LongVector/IntVector.fromMemorySegment(..., LE) plus convert is a direct fit. This helper is called per FastLanes chunk by the Delta decoder, so it sits on a real read path |
fromLongs(long[], ptype, arena) (narrowing branch) |
per-element PTypeIO.set(seg, off, ptype, v) |
Not a Vector API problem. PTypeIO.set is a MethodHandle[] lookup + invokeExact per element with the ptype switch inside the loop, which is exactly what the hot-loop rule forbids (SETTERS[ptype.ordinal()] is variable-target). Fix: hoist the switch and write one specialized loop per width (as toLongsInto already does). Used by DeltaEncodingEncoder for bases and deltas. Plain-Java win, no flag, no ADR gate |
That last row is the cheapest, safest item in this whole issue and could ship independently of any Vector API decision.
Packaging note (applies to W1/W2 and helpers)
Don't scatter jdk.incubator.vector imports across encoders. Mirror hardwood's seam: one small SimdKernels-style interface per module boundary (writer kernels in writer, shared widening in core), scalar implementation always present, a vector implementation selected by a guarded static initializer that catches Throwable (NoClassDefFoundError/UnsupportedOperationException when the module is absent, or SPECIES_PREFERRED too narrow). Writer must not depend on reader, so there will be two small seams rather than one shared class, or the seam lives in core if both sides need it.
Acceptance
- JMH baselines (
-f 3,ClassName.methodName): bit-pack, ALP encode (incl. exponent search),toLongsInto/fromLongs, on M5 and one x86 machine -
fromLongsnarrowing hoisted to per-width loops (independent, plain Java) - Bit-pack loop interchange tried before any Vector API
- Vector kernel for a candidate only if >= 10% over the best scalar on both machines
- Differential tests vs the scalar path, byte-identical output, including ALP edge values
- ADR 0005 and
docs/reference.mdupdated in the same commit
- Vorherrschende Sprache
- Java
- Sterne
- 3
- Forks
- 2
- Ø Merge
- 9 Std. 47 Min.
- Gemergte PRs (30 T.)
- 128
Entwicklungsumgebung
- Kein Dockerfile und keine Docker-Compose-Datei
- Keine Pull-Request-Vorlage
- Beitragsleitfaden lesen
Erste Schritte
- Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
- Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
- Forken Sie das Repository und arbeiten Sie in einem Branch.
- Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.
Mehr aus dfa1/vortex-java
-
bug
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 66/100
dfa1/vortex-java#636 ·
Maintainer antworten meist innerhalb von 1 Tag
-
bug
Schwierigkeit 3/5 1-2 Tage Anfängerfreundlichkeit 70/100
dfa1/vortex-java#655 ·
Maintainer antworten meist innerhalb von 1 Tag
-
perf
Schwierigkeit 4/5 3-5 Tage Anfängerfreundlichkeit 30/100
dfa1/vortex-java#651 ·
Maintainer antworten meist innerhalb von 1 Tag
-
Writer: string columns written as per-chunk dicts where Rust uses its dict layout (up to 2x larger)Offenperf
Schwierigkeit 4/5 3-5 Tage Anfängerfreundlichkeit 30/100
dfa1/vortex-java#650 ·
Maintainer antworten meist innerhalb von 1 Tag
-
perf
Schwierigkeit 4/5 3-5 Tage Anfängerfreundlichkeit 28/100
dfa1/vortex-java#649 ·
Maintainer antworten meist innerhalb von 1 Tag
Alle Issues in dfa1/vortex-java
Ähnliche Issues
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 66/100
a2aproject/a2a-java#1275 ·
Maintainer antworten meist innerhalb von 1 Tag
-
module/ftp Type/Bug
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 72/100
ballerina-platform/ballerina-library#9476 ·
Maintainer antworten meist innerhalb von 1 Tag
-
enhancement
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 72/100
anwilli5/coin-collection-android-US#515 ·
Maintainer antworten meist innerhalb von 1 Tag
-
status: waiting-for-triage type: bug
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 72/100
spring-projects/spring-integration#11654 ·
Maintainer antworten meist innerhalb von 1 Tag
-
Schwierigkeit 2/5 Unter einer Stunde Anfängerfreundlichkeit 70/100
Vakore/ZappierGames#248 ·