Native consumers of the libcuopt-mathopt wheel segfault: solver_settings_t is split across mathopt and client

Đang mở
  • Có gì không ổn?
#2,009 4 bình luận 0 reaction 1 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

@ramakrishnap-nv đang làm issue này rồi.

Từ ngày 30/9/2026.

Đánh giá

Issue này chưa được đánh giá.

Mô tả

Description

cuopt_cli as shipped in the libcuopt-mathopt wheel segfaults immediately on startup. The same binary built from the same commit into the libcuopt wheel works.

$ pip install libcuopt_mathopt_cu13-26.10.0-py3-none-manylinux_2_28_x86_64.whl
$ .../site-packages/libcuopt_mathopt/bin/cuopt_cli --help
Segmentation fault (core dumped)     # exit 139, zero bytes of output

$ .../site-packages/libcuopt/bin/cuopt_cli --help
Usage: cuopt_cli [--help] [--version] ...   # exit 0, 96 lines

Found while installing each component wheel on its own in a bare container, against the wheels from the latest green build.yaml run on main (run 36580941794, x86_64, cu13).

It is the binary, not the environment

With both wheels installed in the same container, and the mathopt binary forced onto the metapackage's libraries:

libcuopt/bin/cuopt_cli                   exit=0    lines=96
libcuopt_mathopt/bin/cuopt_cli           exit=139  lines=0
mathopt CLI + LD_LIBRARY_PATH=libcuopt/lib64   exit=139

The two are genuinely different compilations, not the same binary with different metadata:

build-id .text .rodata
libcuopt/bin/cuopt_cli b9c1fdb1... 3100a14cc7034fe1 cf1b26601c8b39d2
libcuopt_mathopt/bin/cuopt_cli 924272b9... 724578ddda193f99 b2b4a865cc6f3d2e

libcuopt_client.so has the same build-id in both wheels; libcuopt_mathopt.so and cuopt_cli differ.

Ruled out

  • Unresolved libraries. ldd reports nothing missing. Every entry resolves, including libcuopt_client.so from the sibling wheel via $ORIGIN/../../libcuopt_client/lib64.
  • Lazy binding. LD_BIND_NOW=1 still segfaults, so every symbol resolves; the crash is in execution, not loading.
  • A missing component. Installing libcuopt-routing alongside makes no difference.
  • Arguments. --help and no arguments both crash, before any output.
  • ABI mismatch between CLI and engine. A matched pair from the same build job crashes just as a mismatched pair does.
  • RUNPATH corruption from patchelf. Both binaries carry the same RUNPATH entries and near-identical LOAD segments.
  • Duplicate OpenMP runtimes. Both wheels vendor libgomp-34324e08.so.1.0.0 under one soname, so only one instance loads.

What is known about the crash

Thread 1 "cuopt_cli" received signal SIGSEGV
#0  __memcpy_avx_unaligned ()
#1  0x000062a0875c245d in ?? ()
#2  0x000062a0875c0c3c in ?? ()
#3  __libc_start_call_main (main=0x62a0875c0320, argc=2, ...)

Two frames inside cuopt_cli between main and a memcpy, with no output produced, so it is early in main. The shipped binary is stripped, so the frames have no names; a debug build is needed to go further.

Why nothing caught it

The component wheels are built and published but never installed or imported in CI. Noted previously in https://github.com/NVIDIA/cuopt/issues/1635#issuecomment-5892294816. A job that installs each wheel in a clean environment and runs cuopt_cli --help would catch this directly.

Impact

cuopt_cli is the reason libcuopt-mathopt declares a console entry point. As shipped, a standalone pip install libcuopt-mathopt provides a CLI that cannot start.

Ngôn ngữ chính
Cuda
Star
1.1k
Fork
255
Merge trung bình
1 ngày 11 giờ
Pull request đã merge (30 ngày)
123

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của NVIDIA/cuopt

Tất cả issue của NVIDIA/cuopt

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.