arm64 and x86_64 linux: TF java full native builds are failing to find the native headers
還沒有人認領這個 Issue。
評估
- 難度
- 4/5
- 預估耗時
- 3-5 天
- 新手友好度
- 35/100
- Issue 類型
- 缺陷
- 描述清晰度
- 需要釐清
- 活躍度
- 停滯
- 技術堆疊
- java
- 領域
- build-system
研究方向
從報告中列出的 tensorflow-core 原生檔案以及 v1.0.0-rc.1 原生建置的 Maven/Bazel 命令開始。在 Linux arm64 上重現缺少標頭檔的失敗,然後將建置行為與報告中的 x86_64 情況進行比較。當原生 Java 建置解析其相依性並成功完成時,即視為完成。
由索引模型根據 Issue 內容生成。
描述
Please make sure that this is a build/installation issue. As per our GitHub Policy, we only address code/doc bugs, performance issues, feature requests and build/installation issues on GitHub. tag:build_template
System information
- OS Platform and Distribution (e.g., Linux Ubuntu 16.04 x86_64): Linux Ubuntu 22.04, aarch64
- TensorFlow installed from (source or binary): Source
- TensorFlow version: TF version is v2.16.1, TF java version is tag: v1.0.0-rc.1
- Java version (i.e., the output of
java -version): openjdk 11.0.23 2024-04-16 - Java command line flags (e.g., GC parameters): mvn install -P native-build -Dbazel.build.flags='--verbose_failures -s --config=mkl_aarch64_threadpool' -X
- Installed from Maven Central?: No
- Bazel version (if compiling from source): bazel 6.5.0
- GCC/Compiler version (if compiling from source):gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
- CUDA/cuDNN version: Not applicable
- GPU model and memory: Not applicable
Describe the problem
TensorFlow java source builds are failing on aarch64 linux system with the missing native headers. please let me know how it's built for x86_64 linux platform.
based on my debugging so far it looks like the dependency comes from this commit which added C API extension for custom gradient functions, and introduced these headers and .cc which requires several third_party libraries from tensorflow native but none of those bazel workspaces are cloned.
tensorflow-core/tensorflow-core-native/src/main/native/org/tensorflow/internal/c_api$
tfj_gradients.h tfj_gradients_impl.cc tfj_graph.h tfj_graph_impl.cc tfj_scope.h tfj_scope_impl.cc
I tried to manually clone the missing workspaces into bazel cache, but the cycle is never ending, it's missing tsl, eigen, ml_dtype, absl, protobuf, and now compiled headers for protobuf....
Provide the exact sequence of commands / steps that you executed before running into the problem
sudo apt-get install pkg-config ccache clang ant python3-pip swig git file wget unzip tar bzip2 gzip patch autoconf-archive autogen automake make cmake libtool bison flex perl nasm curl gfortran libasound2-dev freeglut3-dev libgtk2.0-dev libusb-dev zlib1g libffi-dev libbz2-dev zlib1g-dev
sudo apt install maven default-jdk
cd $HOME
mkdir bazel
cd bazel
wget https://github.com/bazelbuild/bazel/releases/download/6.5.0/bazel-6.5.0-linux-arm64
mv bazel-6.5.0-linux-arm64 bazel
chmod a+x bazel
export PATH=/home/ubuntu/bazel/:$PATH
# Build and install javacpp-presets.
# Clone the following forked repo to exclude the libraries that are not supported and not required
git clone https://github.com/snadampal/javacpp-presets.git
cd javacpp-presets
git checkout tfjava_aarch64
mvn install -Djavacpp.platform=linux-arm64 -Dmaven.javadoc.skip=true -X -T 16
# Build and install tensorflow java bindings
git clone https://github.com/tensorflow/java.git
cd java
git checkout v1.0.0-rc.1
mvn install -P native-build -Dbazel.build.flags='--verbose_failures -s --config=mkl_aarch64_threadpool' -X
Any other info / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached.
- 主要語言
- Java
- 星號
- 928
- 分支
- 227
- PR 合併指標
- 30 天內沒有已合併 PR
貢獻指南
從這裡開始
- 先讀完整個 Issue,再讀專案的貢獻指南。
- 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
- Fork 儲存庫,在一個分支上完成修改。
- 送出 Pull Request,並在描述裡引用這個 Issue 編號。
tensorflow/java 的其他 Issue
-
難度 2/5 1-3 小時 新手友好度 65/100
tensorflow/java#653 · 1 則留言 · 4 個 reaction ·
-
難度 5/5 一週以上 新手友好度 25/100
tensorflow/java#621 · 4 則留言 ·
-
難度 5/5 一週以上 新手友好度 25/100
tensorflow/java#617 · 3 則留言 ·
-
難度 2/5 1-3 小時 新手友好度 55/100
tensorflow/java#615 · 1 則留言 ·
-
難度 5/5 一週以上 新手友好度 25/100
tensorflow/java#614 · 1 則留言 ·
相似的 Issue
-
難度 2/5 1-3 小時 新手友好度 82/100
infinispan/infinispan#18150 ·
-
area/frontend
難度 2/5 1-3 小時 新手友好度 65/100
-
難度 2/5 1-3 小時 新手友好度 84/100
-
untriaged
難度 2/5 1-3 小時 新手友好度 82/100
opensearch-project/k-NN#3597 ·
-
bug
難度 2/5 1-3 小時 新手友好度 88/100