Benchmarks: make jmh.gradle run cleanly on JDK 17 (module opens, configurable heap, failOnError)
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 72/100
- Issue type
- Feature
- Clarity
- Clearly specified
- Activity status
- Quiet
- Tech stack
- java
- Domain
- build-system, performance
Research direction
Start with the one named file, jmh.gradle, and run the provided Gradle JMH command on JDK 17 to reproduce the ZoneInfo IllegalAccessException. Verify that the JDK 17 module opens are applied, -PjmhHeap and -PjmhFailOnError override their documented defaults, and benchmark results remain available when a failure occurs.
Written by the indexing model from the issue text.
Description
Problem
The JMH harness fails on a clean JDK 17 checkout:
- Spark 4.x reflects into
sun.util.calendar; without--add-opensthe read-path benchmarks throw
IllegalAccessExceptionat warm-up (stack trace below). - Heap is hard-coded to
-Xmx32g; the full-scalePlanningBenchmark...WithStatsOOMs at that size. failOnError = trueaborts the whole suite (and discards completed results) on one failure.
Reproduce (JDK 17, Corretto 17.0.19)
./gradlew -PsparkVersions=3.5 -PscalaVersion=2.12 \
:iceberg-spark:iceberg-spark-extensions-3.5_2.12:jmh \
-PjmhIncludeRegex=IcebergSourceParquetEqDeleteBenchmark \
-PjmhOutputPath=build/reports/jmh/human.txt \
-PjmhJsonOutputPath=build/reports/jmh/results.json
=> IllegalAccessException: sun.util.calendar.ZoneInfo
Example failure (from earlier EC2 run):
org.apache.spark.SparkException: Job aborted due to stage failure:
Task 0 in stage 0.0 failed 1 times ...
java.lang.IllegalAccessException: symbolic reference class is not accessible:
class sun.util.calendar.ZoneInfo, from interface
org.apache.spark.sql.catalyst.util.SparkDateTimeUtils (unnamed module)
at java.base/java.lang.invoke.MemberName.makeAccessException(MemberName.java:955)
at org.apache.spark.sql.catalyst.util.SparkDateTimeUtils.toJavaDate(SparkDateTimeUtils.scala:304)
at org.apache.iceberg.spark.source.IcebergSourceBenchmark.appendAsFile(IcebergSourceBenchmark.java:136)
Proposal
Add the JDK 17 --add-opens set to JMH jvmArgs, make heap configurable via -PjmhHeap (default 32g),
and make failOnError configurable via -PjmhFailOnError (default true). One file: jmh.gradle.
Evidence
- Failure: read-path benchmarks crash at warm-up without
sun.util.calendaropen - Success: same suites complete on Corretto 17.0.19 / JMH 1.37 on EC2
r7i.4xlarge/r7i.8xlargewhen the opens set and a larger heap (-PjmhHeap=220gfor full-scale planning) are applied - Defaults preserved: heap stays
32gandfailOnErrorstaystrueunless-Pflags are passed
Sharing PR with fix
- Dominant language
- Java
- Stars
- 9.3k
- Forks
- 3.5k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 143
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/iceberg
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
improvement
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
infinispan/infinispan#18150 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
untriaged
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
opensearch-project/k-NN#3597 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100