Benchmarks: make jmh.gradle run cleanly on JDK 17 (module opens, configurable heap, failOnError)

Open Beginner friendly
#17,330 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
72/100
Issue type
Feature
Clarity
Clearly specified
Activity status
Quiet
Tech stack
java

Research direction

Start with the one named file, jmh.gradle, and run the provided Gradle JMH command on JDK 17 to reproduce the ZoneInfo IllegalAccessException. Verify that the JDK 17 module opens are applied, -PjmhHeap and -PjmhFailOnError override their documented defaults, and benchmark results remain available when a failure occurs.

Written by the indexing model from the issue text.

Description

Problem

The JMH harness fails on a clean JDK 17 checkout:

  1. Spark 4.x reflects into sun.util.calendar; without --add-opens the read-path benchmarks throw
    IllegalAccessException at warm-up (stack trace below).
  2. Heap is hard-coded to -Xmx32g; the full-scale PlanningBenchmark ...WithStats OOMs at that size.
  3. failOnError = true aborts the whole suite (and discards completed results) on one failure.
Reproduce (JDK 17, Corretto 17.0.19)
./gradlew -PsparkVersions=3.5 -PscalaVersion=2.12 \
  :iceberg-spark:iceberg-spark-extensions-3.5_2.12:jmh \
  -PjmhIncludeRegex=IcebergSourceParquetEqDeleteBenchmark \
  -PjmhOutputPath=build/reports/jmh/human.txt \
  -PjmhJsonOutputPath=build/reports/jmh/results.json

=> IllegalAccessException: sun.util.calendar.ZoneInfo

Example failure (from earlier EC2 run):

org.apache.spark.SparkException: Job aborted due to stage failure:
  Task 0 in stage 0.0 failed 1 times ...
java.lang.IllegalAccessException: symbolic reference class is not accessible:
  class sun.util.calendar.ZoneInfo, from interface
  org.apache.spark.sql.catalyst.util.SparkDateTimeUtils (unnamed module)
    at java.base/java.lang.invoke.MemberName.makeAccessException(MemberName.java:955)
    at org.apache.spark.sql.catalyst.util.SparkDateTimeUtils.toJavaDate(SparkDateTimeUtils.scala:304)
    at org.apache.iceberg.spark.source.IcebergSourceBenchmark.appendAsFile(IcebergSourceBenchmark.java:136)
Proposal

Add the JDK 17 --add-opens set to JMH jvmArgs, make heap configurable via -PjmhHeap (default 32g),
and make failOnError configurable via -PjmhFailOnError (default true). One file: jmh.gradle.

Evidence
  • Failure: read-path benchmarks crash at warm-up without sun.util.calendar open
  • Success: same suites complete on Corretto 17.0.19 / JMH 1.37 on EC2 r7i.4xlarge / r7i.8xlarge when the opens set and a larger heap (-PjmhHeap=220g for full-scale planning) are applied
  • Defaults preserved: heap stays 32g and failOnError stays true unless -P flags are passed

Sharing PR with fix

Dominant language
Java
Stars
9.3k
Forks
3.5k
Avg merge
2d 11h
Merged PRs (30d)
143

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/iceberg

All issues in apache/iceberg

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.