[build] Shaded fat-JARs (fluss-client, fluss-flink, etc.) leak shaded module-info.class, breaking JPMS modular consumption

Open Beginner friendly
#4,045 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
88/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
java
Domain
build-system

Research direction

Open the root pom.xml and inspect the global maven-shade-plugin filter used for shaded JARs. Build a representative artifact such as fluss-client, then run jar --describe-module --file=... --release 21 to verify that dependency module-info.class files are no longer embedded and the Fluss packages are available through an automatic module.

Written by the indexing model from the issue text.

Description

Search before asking
  • I searched in the issues and found nothing similar.
Fluss version

main (development)

Please describe the bug 🐞

Problem Summary

All shaded uber-JARs produced by Fluss (e.g. fluss-client, fluss-flink-1.20, fluss-flink-1.19, fluss-server) currently embed META-INF/versions/9/module-info.class originating from shaded dependencies (such as commons-lang3:3.18.0 and fluss-shaded-jackson).

As a result, on Java 9+, the Java Platform Module System (JPMS) treats these shaded JARs as the named module org.apache.commons.lang3 rather than deriving an automatic module (e.g. fluss.client, fluss.flink).

How to Reproduce

Run jar --describe-module on any built shaded JAR (e.g., fluss-client or fluss-flink-1.20):

jar --describe-module --file=fluss-client-1.0-SNAPSHOT.jar --release 21

Actual Output:

releases: 9

org.apache.commons.lang3@3.18.0 jar:file:///.../fluss-client-1.0-SNAPSHOT.jar!/META-INF/versions/9/module-info.class
exports org.apache.commons.lang3
exports org.apache.commons.lang3.arch
...
requires java.base mandated
requires java.desktop

Notice that:

  1. The entire Fluss JAR is identified as module org.apache.commons.lang3.
  2. None of the org.apache.fluss.* packages are exported.

Impact on Downstream Projects

Downstream projects using the Java Module System (module-info.java) cannot access any Fluss APIs (org.apache.fluss.client.*, org.apache.fluss.flink.*, org.apache.fluss.config.*) without manually configuring awkward compiler flags such as:
--add-exports org.apache.commons.lang3/org.apache.fluss.client=<target-module>

Solution

Solution Details

In the root pom.xml, configure the global maven-shade-plugin filter (<artifact>*</artifact>) to exclude all module-info.class and multi-release module-info.class files:

<filter>
    <artifact>*</artifact>
    <excludes>
        ...
        <!-- Do not copy module-info.class from shaded dependencies -->
        <exclude>module-info.class</exclude>
        <exclude>META-INF/versions/**/module-info.class</exclude>
        ...
    </excludes>
</filter>

This is the standard pattern used across Apache projects (Flink, Spark) when bundling shaded dependencies into uber-JARs.

Local Validation

I have tested this fix across multiple modules:

  • fluss-client now correctly derives fluss.client@1.0-SNAPSHOT automatic with all packages exported.
  • fluss-flink-1.20 correctly derives fluss.flink@1.20-1.0-SNAPSHOT automatic.
  • fluss-server correctly derives fluss.server@1.0-SNAPSHOT automatic.
  • Spotless formatting (mvn spotless:check) and Checkstyle pass cleanly.

I have this fix ready and tested locally, and would be very happy to submit a Pull Request if the maintainers approve this approach.

Are you willing to submit a PR?
  • I'm willing to submit a PR!
Dominant language
Java
Stars
2.2k
Forks
628
Avg merge
1d 19h
Merged PRs (30d)
138

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/fluss

All issues in apache/fluss

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.