[Bug] Internal execution failure due to container image error after Helm chart deployment
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 38/100
- Issue type
- Bug
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- docker, helm, kubernetes
- Domain
- build-system, devops, infrastructure
Research direction
Start by reproducing the failure with ghcr.io/microsoft/jrtc-apps/grc:latest, running volk_profile -v and the GRC flowgraph in the Helm deployment. Compare the image build configuration and CPU targets with a locally built image on the Intel Xeon Gold 6148, and check the provided gNB image similarly. Done means the container image avoids the segmentation fault and both GRC and gNB start successfully on the reported host.
Written by the indexing model from the issue text.
Description
Hi all,
First of all, since I have already successfully built and executed everything on a bare-metal environment, I am confident that there are no issues with the source code itself. However, I am concerned that the provided built image (especially on grc, and srsgnb) might have been compiled too strictly (e.g., heavily optimized for specific newer CPU architectures).
1. Environment
Image: ghcr.io/microsoft/jrtc-apps/grc:latest
GNU Radio Version: 3.10.1.1
Host OS: Ubuntu 22.04.5 LTS (Kernel 5.15.0-119-generic)
CPU: Intel(R) Xeon(R) Gold 6148 (Supports AVX-512)
2. The Core Issue
The GRC process hangs indefinitely after printing the following log:
[INFO] Starting flowgraph....
When running volk_profile -v inside the container, it immediately crashes:
command terminated with exit code 139 (Segmentation Fault)
**Our Suspicion: ** This indicates a failure within the VOLK library, likely during SIMD kernel selection/dispatching. We suspect a potential -march=native optimization issue during the image build process. If the image was compiled on a different CPU architecture (e.g., a newer generation Xeon), the specific instructions might be incompatible with our 1st Gen Xeon Gold processor, leading to this crash.
3. Related Binary Issues (gNB Case)
We encountered a similar compatibility issue with the gNB image provided in the Helm chart, where the original binary crashed upon startup.
We were only able to resolve the gNB issue by replacing the container's binary with one built locally on the same host.
This consistent failure across both gNB and GRC images strongly suggests a general binary incompatibility with our environment.
4. Verified Configurations & Troubleshooting
We have ruled out permission and networking issues by verifying the following:
-
Applied yaml file ( privileged: true, hostIPC: true, and seccomp: Unconfined to the pod. )
-
Verified ZMQ connectivity (E2E).
-
Subscriber successfully registered in Open5GS DB.
5. Our Situation and Concerns
We have transitioned to a Kubernetes-based setup following your recommendation to achieve stability. However, encountering these fundamental binary issues at the initial stage is quite unexpected.
While we could technically proceed by manually building and replacing all binaries locally, we are deeply concerned that such a workaround will lead to future compatibility risks and massive maintenance overhead. To ensure a stable and sustainable setup as intended by your project, we believe a fundamental fix at the image level is necessary.
6. Requests
-
Could you verify if these images were built with specific CPU optimizations (e.g., AVX-512 for newer architectures) that might be unstable on certain Xeon architectures?
-
Could you provide a more generic image (e.g., compiled with AVX2 or Generic targets) or advise on a definitive way to resolve these instruction set mismatches?
We really want to avoid relying on local manual builds for long-term maintenance. Looking forward to your thoughts on this.
- Dominant language
- Python
- Stars
- 14
- Forks
- 13
- PR merge metrics
- No merged PRs in 30d
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from microsoft/jrtc-apps
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
-
Add basic tests to the CIMay be free again @doctorlai-msrc claimed this 453 days ago, and no pull request is open. Open
All issues in microsoft/jrtc-apps
Similar issues
-
Difficulty 1/5 1-3 hours Newbie friendliness 88/100
gradio-app/gradio#13964 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 93/100
BSData/horus-heresy-3rd-edition#3232 · 1 comment ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
jbacalso24/mimry#123 ·
Maintainers usually reply within 1 day
-
p:3-mid pydanty:bug pydanty:is-working
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
pydantic/pydantic-ai#10153 · 1 comment ·
Maintainers usually reply within 1 day
-
good first issue priority: medium size: S testing
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
cubrid-lab/cubrid-cookbook-python#316 ·
Maintainers usually reply within 1 day