[BUILD] `iceberg` and `iceberg-test` profiles collide on one build-helper execution id, silently dropping `src-iceberg/test` sources
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 86/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Domain
- build-system, testing
Research direction
Start in the root pom.xml at the iceberg and iceberg-test build-helper-maven-plugin executions around lines 1833 and 1884. Give the two executions distinct ids, then run the reported ClickHouse test-compile command with both profiles and inspect the effective POM to confirm both source lists remain; done means the command succeeds and both named classes are emitted.
Written by the indexing model from the issue text.
Description
What happens
Both profiles in the root pom declare a build-helper-maven-plugin execution with the id add-iceberg-test-sources: pom.xml:1833 under iceberg, pom.xml:1884 under iceberg-test. Maven merges plugin executions by id, so when both profiles are active only one <sources> list survives.
help:effective-pom with -Pbackends-clickhouse -Pspark-3.5 -Pspark-ut -Pdelta -Piceberg -Piceberg-test resolves that execution to:
add-iceberg-test-sources
src-iceberg-spark35/test/scala
src-iceberg-spark35/test/java
The iceberg profile's four entries (src-iceberg/test/{scala,java} and src-iceberg${iceberg.binary.version}/test/{scala,java}) are gone. No warning is printed; the sources are simply not compiled.
Effect on ClickHouse: the combination cannot build
ClickHouseIcebergHiveTableSupport lives in backends-clickhouse/src-iceberg/test/scala, and both Java tests under src-iceberg-spark35/test/java instantiate it. The class therefore leaves the source path at exactly the moment the tests join it:
$ ./build/mvn -Pbackends-clickhouse -Pspark-3.5 -Pscala-2.13 -Pspark-ut -Pdelta \
-Piceberg -Piceberg-test -DskipTests test-compile -pl backends-clickhouse
[ERROR] .../src-iceberg-spark35/test/java/.../TestPositionDeletesTableGluten.java:[41,18] cannot find symbol
[ERROR] symbol: class ClickHouseIcebergHiveTableSupport
Giving the iceberg-test execution a distinct id (add-iceberg-spark-test-sources) and changing nothing else makes the same command exit 0 and emit both ClickHouseIcebergHiveTableSupport.class and TestPositionDeletesTableGluten.class.
Effect on Velox: harmless today, which is why nobody noticed
Nothing in backends-velox needs both directories at once. src-iceberg/test holds VeloxIcebergSuite and VeloxTPCHIcebergSuite, which run in the jobs that pass -Piceberg without -Piceberg-test (velox_backend_x86.yml:941 and :1013, whose wildcardSuites includes org.apache.gluten), plus a @EnhancedFeaturesTest variant that velox_backend_enhanced.yml covers, a JUnit test surefire picks up, and one helper object. The 52 files under src-iceberg-spark34/test run in the three groups at :817, :821, :825 that pass both profiles.
Suggested fix
Rename the iceberg-test profile's execution so the two lists no longer overwrite each other. Both profiles are always passed together in every in-repo invocation, so the merged list is what the plugin was meant to produce.
Two things a reviewer will want to know about the blast radius. The three x86 groups and the 4.0 test-compile at :1277 would newly compile src-iceberg/test/**, so those suites run twice across the matrix; expect longer jobs rather than new failures. And src-iceberg${iceberg.binary.version}/test is empty in every module, so nothing else appears on the source path.
Only backends-velox and backends-clickhouse hold files under src-iceberg*/test, so the change is confined to those two modules.
This blocks the ClickHouse Iceberg Java test work in #12934.
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 661
- Avg merge
- 4d 9h
- Merged PRs (30d)
- 74
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/gluten
-
[VL] madvise(WILLNEED) call fails in MmapFileStream because of wrong calculation of fetching length Openbug triage
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
bug triage
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
ergoplatform/ergodocs#614 ·
-
area:ci enhancement requires-triage
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
apache/datafusion-comet#6078 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
chipsalliance/rocket-chip#3831 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100