[Bug]: Maven fat/runtime jars: missing ThreeTen-Extra attribution and stray third-party LICENSE/NOTICE files
メンテナーはふだん 2 日以内に返信
まだ誰も着手していません。
評価
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 初心者へのやさしさ
- 48/100
- issue の種類
- バグ
- 明瞭さ
- おおむね明確
- 活発さ
- 活発
- 技術スタック
- java
- 領域
- build-system, devtools
調査の方向性
Start with the runtime jar configuration and Maven shade filters, then read tools/maven/fat-jar-with-licenses.xml and compare its output with the staged jars described in the issue. Check how the runtime jars and jar-with-dependencies collect LICENSE, NOTICE, and META-INF/licenses files, including the ThreeTen-Extra attribution and conflicting root files. Done means the bundled files preserve required attribution and no longer expose misleading duplicate root-level notices.
索引モデルが issue の本文から書いたものです。
説明
What happened?
While reviewing the licensing of the Maven artefacts staged for 0.9.0-incubating RC14 (orgapacheamoro-1093; tag v0.9.0-rc14, commit 18d12b99d5297dbd0bb4eb72517c4855d75210f6), I found the problems below in the jars that bundle third-party code:
- the six runtime jars:
amoro-format-mixed-flink-runtime-1.{18,19,20}andamoro-format-mixed-spark-runtime-3.{3,4,5}_2.12 - the two
jar-with-dependenciesjars:amoro-optimizer-flinkandamoro-metrics-prometheus
Overall the staged artefacts are in good shape. All 150 files are correctly signed. Every jar, including the sources, tests and javadoc jars, carries META-INF/LICENSE, NOTICE and DISCLAIMER. The fat jars document their non-Apache components in detail.
1. ThreeTen-Extra (BSD-3-Clause) classes are bundled without attribution in the runtime jars
All six runtime jars contain:
org/apache/amoro/shade/org/threeten/extra/chrono/HybridChronology.class
org/apache/amoro/shade/org/threeten/extra/chrono/HybridChronology$1.class
org/apache/amoro/shade/org/threeten/extra/chrono/HybridDate.class
org/apache/amoro/shade/org/threeten/extra/chrono/HybridDate$1.class
These come in via Apache ORC (orc-core), which copies them from ThreeTen-Extra, BSD-3-Clause; upstream ORC credits it in its own LICENSE. No licence or readme file in the runtime jars mentions ThreeTen: not META-INF/LICENSE, META-INF/README-LICENSES or META-INF/NOTICE, and nothing under META-INF/licenses/. BSD-3-Clause requires its copyright notice and licence text to accompany binary redistribution.
For comparison, the jar-with-dependencies jars bundle the full org.threeten:threeten-extra:1.7.1 and do list it correctly, pointing to META-INF/licenses/org.threeten.threeten-extra-1.7.1.txt.
Suggested fix: add a ThreeTen-Extra (BSD-3-Clause) entry to the runtime jars' LICENSE, noting it arrives via ORC, and include the licence text under META-INF/licenses/. It's also worth checking LICENSE-binary in the dist tarballs, which bundle orc-core too.
2. Third-party LICENSE.* / NOTICE.* files leak into the root of META-INF/ in all fat jars
Besides Amoro's own META-INF/LICENSE and META-INF/NOTICE, all eight fat jars contain extra root-level files that come from whichever dependency the shade or assembly step processed last:
| File | Content |
|---|---|
META-INF/LICENSE.txt |
An Apache-2.0 text from some dependency |
META-INF/LICENSE.md |
The Eclipse Public License 2.0 text |
META-INF/NOTICE.txt |
"Apache Commons Pool", "Apache Commons Collections" or "Apache Commons BeanUtils", depending on the jar |
META-INF/NOTICE.md |
"Notices for Jakarta Annotations" |
META-INF/NOTICE-binary (jar-with-dependencies only) |
"Apache Hadoop Third-party Libs" |
They don't replace Amoro's files, but they are misleading at the top level (an EPL-2.0 LICENSE.md next to the Apache LICENSE, for example). Because they are last-one-wins, the same-named files from other dependencies were silently overwritten in the process.
Suggested fix: exclude these paths in the maven-shade-plugin filters and the assembly descriptor (tools/maven/fat-jar-with-licenses.xml), after confirming their content is already covered by Amoro's META-INF/LICENSE / NOTICE / licenses/. Alternatively, merge NOTICE content with the shade plugin's ApacheNoticeResourceTransformer.
3. Fat-jar LICENSE files don't name the bundled Apache-2.0 components (consistency)
- The
jar-with-dependenciesLICENSE files list only non-Apache components. About 100 bundled Apache-2.0 dependencies are not named, including Guava, thecommons-*jars, Jetty, DataNucleus, Dropwizard metrics, hppc, bonecp, failsafe and xnio. - The runtime jars' LICENSE is assembled by concatenating upstream LICENSE files (it contains 22 copies of the Apache-2.0 text). It does not name the bundled components themselves, such as ZooKeeper, Guava, Thrift, Caffeine, httpclient5, Netty, ORC and Iceberg.
ASF policy does not strictly require Apache-2.0 components to be listed, so this is a consistency point rather than a compliance problem. The dist binary LICENSE (LICENSE-binary) does list its Apache-2.0 components, so the conventions differ between the two kinds of convenience binary. It would be good to settle on one approach, as with #4478 for the source LICENSE.
Related: com.github.jsqlparser:jsqlparser (dual LGPL-2.1 / Apache-2.0) is bundled in amoro-optimizer-flink-*-jar-with-dependencies.jar with no statement of which licence is chosen. The dist LICENSE lists it under Apache-2.0; the fat jar should say the same.
Affects Versions
0.9.0-incubating (RC14)
How to reproduce
R=https://repository.apache.org/content/repositories/orgapacheamoro-1093/org/apache/amoro
curl -fO $R/amoro-format-mixed-spark-runtime-3.5_2.12/0.9.0-incubating/amoro-format-mixed-spark-runtime-3.5_2.12-0.9.0-incubating.jar
J=amoro-format-mixed-spark-runtime-3.5_2.12-0.9.0-incubating.jar
# 1. threeten classes bundled, no attribution
unzip -Z1 $J | grep threeten/extra/chrono/.*class
for f in META-INF/LICENSE META-INF/NOTICE META-INF/README-LICENSES; do unzip -p $J $f | grep -ci threeten; done # 0 0 0
# 2. stray root-level licence/notice files
unzip -Z1 $J | grep -E '^META-INF/(LICENSE|NOTICE)\.(txt|md)$'
unzip -p $J META-INF/LICENSE.md | head -1 # Eclipse Public License - v 2.0
unzip -p $J META-INF/NOTICE.txt | head -1 # Apache Commons Pool
- 主要言語
- Java
- スター
- 1.2k
- フォーク
- 398
- 平均マージ
- 3日 13時間
- マージ済み PR(30日)
- 14
環境構築
- Dockerfile・Docker Compose ファイルなし
- プルリクエストのテンプレートあり
- コントリビューションガイドを読む
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
apache/amoro のほかの issue
-
難易度 2/5 1〜3時間 初心者へのやさしさ 77/100
メンテナーはふだん 2 日以内に返信
-
難易度 1/5 1時間未満 初心者へのやさしさ 82/100
メンテナーはふだん 2 日以内に返信
-
[Doc]: Clarify that catalog-level table properties should be added via the "Table properties" interface, and the "table." prefix is only required when using the raw "Properties" interface対応中かも @Akeron-Zhu が 8 日前に担当しました。 オープンtype:improvement
難易度 1/5 1時間未満 初心者へのやさしさ 85/100
メンテナーはふだん 2 日以内に返信
-
[CI] Core CI doesn't run for changes to amoro-format-lance (and amoro-web)再び着手できるかも このイシューのプルリクエストはマージされずにクローズされました。 オープン
難易度 1/5 1時間未満 初心者へのやさしさ 88/100
メンテナーはふだん 2 日以内に返信
-
難易度 4/5 3〜5日 初心者へのやさしさ 57/100
apache/amoro#4485 · コメント 1 件 ·
メンテナーはふだん 2 日以内に返信
似ている issue
-
難易度 2/5 1〜3時間 初心者へのやさしさ 88/100
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
jakartaee/expression-language#501 ·
メンテナーはふだん 7 日以内に返信
-
enhancement triage
難易度 2/5 1〜3時間 初心者へのやさしさ 82/100
ShaftHQ/SHAFT_ENGINE#7902 ·
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 76/100
-
Cohort sample with age "not between" includes people from other cohorts and ignores the gender filter対応中かも @DG47 が今日担当しました。 オープン
難易度 2/5 1〜3時間 初心者へのやさしさ 82/100
メンテナーはふだん 1 日以内に返信