[Bug]: Maven fat/runtime jars: missing ThreeTen-Extra attribution and stray third-party LICENSE/NOTICE files

Aperta
  • Qualcosa non va?
#4,481 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 2 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
48/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
java
Ambito
build-system, devtools

Direzione di ricerca

Start with the runtime jar configuration and Maven shade filters, then read tools/maven/fat-jar-with-licenses.xml and compare its output with the staged jars described in the issue. Check how the runtime jars and jar-with-dependencies collect LICENSE, NOTICE, and META-INF/licenses files, including the ThreeTen-Extra attribution and conflicting root files. Done means the bundled files preserve required attribution and no longer expose misleading duplicate root-level notices.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

What happened?

While reviewing the licensing of the Maven artefacts staged for 0.9.0-incubating RC14 (orgapacheamoro-1093; tag v0.9.0-rc14, commit 18d12b99d5297dbd0bb4eb72517c4855d75210f6), I found the problems below in the jars that bundle third-party code:

  • the six runtime jars: amoro-format-mixed-flink-runtime-1.{18,19,20} and amoro-format-mixed-spark-runtime-3.{3,4,5}_2.12
  • the two jar-with-dependencies jars: amoro-optimizer-flink and amoro-metrics-prometheus

Overall the staged artefacts are in good shape. All 150 files are correctly signed. Every jar, including the sources, tests and javadoc jars, carries META-INF/LICENSE, NOTICE and DISCLAIMER. The fat jars document their non-Apache components in detail.

1. ThreeTen-Extra (BSD-3-Clause) classes are bundled without attribution in the runtime jars

All six runtime jars contain:

org/apache/amoro/shade/org/threeten/extra/chrono/HybridChronology.class
org/apache/amoro/shade/org/threeten/extra/chrono/HybridChronology$1.class
org/apache/amoro/shade/org/threeten/extra/chrono/HybridDate.class
org/apache/amoro/shade/org/threeten/extra/chrono/HybridDate$1.class

These come in via Apache ORC (orc-core), which copies them from ThreeTen-Extra, BSD-3-Clause; upstream ORC credits it in its own LICENSE. No licence or readme file in the runtime jars mentions ThreeTen: not META-INF/LICENSE, META-INF/README-LICENSES or META-INF/NOTICE, and nothing under META-INF/licenses/. BSD-3-Clause requires its copyright notice and licence text to accompany binary redistribution.

For comparison, the jar-with-dependencies jars bundle the full org.threeten:threeten-extra:1.7.1 and do list it correctly, pointing to META-INF/licenses/org.threeten.threeten-extra-1.7.1.txt.

Suggested fix: add a ThreeTen-Extra (BSD-3-Clause) entry to the runtime jars' LICENSE, noting it arrives via ORC, and include the licence text under META-INF/licenses/. It's also worth checking LICENSE-binary in the dist tarballs, which bundle orc-core too.

2. Third-party LICENSE.* / NOTICE.* files leak into the root of META-INF/ in all fat jars

Besides Amoro's own META-INF/LICENSE and META-INF/NOTICE, all eight fat jars contain extra root-level files that come from whichever dependency the shade or assembly step processed last:

File Content
META-INF/LICENSE.txt An Apache-2.0 text from some dependency
META-INF/LICENSE.md The Eclipse Public License 2.0 text
META-INF/NOTICE.txt "Apache Commons Pool", "Apache Commons Collections" or "Apache Commons BeanUtils", depending on the jar
META-INF/NOTICE.md "Notices for Jakarta Annotations"
META-INF/NOTICE-binary (jar-with-dependencies only) "Apache Hadoop Third-party Libs"

They don't replace Amoro's files, but they are misleading at the top level (an EPL-2.0 LICENSE.md next to the Apache LICENSE, for example). Because they are last-one-wins, the same-named files from other dependencies were silently overwritten in the process.

Suggested fix: exclude these paths in the maven-shade-plugin filters and the assembly descriptor (tools/maven/fat-jar-with-licenses.xml), after confirming their content is already covered by Amoro's META-INF/LICENSE / NOTICE / licenses/. Alternatively, merge NOTICE content with the shade plugin's ApacheNoticeResourceTransformer.

3. Fat-jar LICENSE files don't name the bundled Apache-2.0 components (consistency)
  • The jar-with-dependencies LICENSE files list only non-Apache components. About 100 bundled Apache-2.0 dependencies are not named, including Guava, the commons-* jars, Jetty, DataNucleus, Dropwizard metrics, hppc, bonecp, failsafe and xnio.
  • The runtime jars' LICENSE is assembled by concatenating upstream LICENSE files (it contains 22 copies of the Apache-2.0 text). It does not name the bundled components themselves, such as ZooKeeper, Guava, Thrift, Caffeine, httpclient5, Netty, ORC and Iceberg.

ASF policy does not strictly require Apache-2.0 components to be listed, so this is a consistency point rather than a compliance problem. The dist binary LICENSE (LICENSE-binary) does list its Apache-2.0 components, so the conventions differ between the two kinds of convenience binary. It would be good to settle on one approach, as with #4478 for the source LICENSE.

Related: com.github.jsqlparser:jsqlparser (dual LGPL-2.1 / Apache-2.0) is bundled in amoro-optimizer-flink-*-jar-with-dependencies.jar with no statement of which licence is chosen. The dist LICENSE lists it under Apache-2.0; the fat jar should say the same.

Affects Versions

0.9.0-incubating (RC14)

How to reproduce
R=https://repository.apache.org/content/repositories/orgapacheamoro-1093/org/apache/amoro
curl -fO $R/amoro-format-mixed-spark-runtime-3.5_2.12/0.9.0-incubating/amoro-format-mixed-spark-runtime-3.5_2.12-0.9.0-incubating.jar
J=amoro-format-mixed-spark-runtime-3.5_2.12-0.9.0-incubating.jar

# 1. threeten classes bundled, no attribution
unzip -Z1 $J | grep threeten/extra/chrono/.*class
for f in META-INF/LICENSE META-INF/NOTICE META-INF/README-LICENSES; do unzip -p $J $f | grep -ci threeten; done   # 0 0 0

# 2. stray root-level licence/notice files
unzip -Z1 $J | grep -E '^META-INF/(LICENSE|NOTICE)\.(txt|md)$'
unzip -p $J META-INF/LICENSE.md | head -1    # Eclipse Public License - v 2.0
unzip -p $J META-INF/NOTICE.txt | head -1    # Apache Commons Pool
Lingua principale
Java
Stelle
1.2k
Fork
398
Merge medio
3g 7h
PR unite (30g)
15

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di apache/amoro

Tutte le issue di apache/amoro

Issue simili

Altre issue su Java

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.