Hacktoberfest 2026：メンテナが10月に向けて印を付けた、オープンで初心者向けの issue。 Hacktoberfest の issue を見る

[BUG] CAGRA hang on h100 oldest-deps CI (cuvs test_cagra, then cuvs_bench test_run_command_creates_results)

オープン
  • 何か問題がありますか？
#2,772 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 2 日以内に返信

@bdice がすでに取り組んでいます。

2026年10月11日 から。

  • #2829 @bdice による — オープン

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
42/100
issue の種類
バグ
明瞭さ
おおむね明確
活発さ
活発
技術スタック
python
領域
machine-learning, testing-qa

調査の方向性

Start with tests/test_cagra.py, especially the suspected test_cagra_dataset_dtype_host_device area, and tests/test_cli.py::test_run_command_creates_results; inspect the failing CI logs and the ci/test_python.sh timeout setup. Reproduce on the specified H100, CUDA 12.2, oldest-deps matrix and capture where each process hangs. Done means the intermittent hangs are explained and addressed, with both test suites passing reliably; the separate symbol-lookup failure is out of scope.

索引モデルが issue の本文から書いたものです。

説明

tests/test_cli.py::test_run_command_creates_results in cuvs_bench failed in 4 pr.yaml runs since 2026-10-06, all on the 12.2.2, 3.11, amd64, rockylinux8, h100, earliest-driver, oldest-deps matrix entry. test_plot_command_creates_png_files fails in the same jobs because it reads the results the run test did not produce.

The Aborted! message is not a crash. It is click's response to SIGINT: ci/test_python.sh runs pytest under timeout 40m --signal=SIGINT, and the benchmark was hung until timeout fired.

In all 4 jobs:

  1. The pytest cuvs step hits its 40 minute timeout and is SIGKILLed. It stalls inside tests/test_cagra.py at the same point (after roughly 52 tests, likely in test_cagra_dataset_dtype_host_device).
  2. The pytest cuvs-bench step then starts, and CUVS_CAGRA_HNSWLIB_ANN_BENCH hangs about 3 seconds in, right after "We have sufficient memory to proceed with in memory build", until its own 40 minute timeout.

A passing run of the same job finishes the cuvs suite in 11 minutes and the bench suite in 72 seconds, with the same driver (580.178.04), CUDA 12.2 and rmm version, so the hang is intermittent. Two of the failing commits already contain #2729. The root cause is not yet identified; the logs do not show where the process is stuck.

Failing jobs:

PR Job
#2649 https://github.com/NVIDIA/cuvs/actions/runs/37562048604/job/113140366555
#2649 https://github.com/NVIDIA/cuvs/actions/runs/37526496924/job/112498097520
#2753 https://github.com/NVIDIA/cuvs/actions/runs/37513765252/job/112456033858
#2429 https://github.com/NVIDIA/cuvs/actions/runs/37426744344/job/112159622042

A fifth job, https://github.com/NVIDIA/cuvs/actions/runs/34719781952/job/104185737619 (PR #2600, CUDA 13.3.0, rtxpro6000), failed the same two tests in 5.8 seconds with symbol lookup error on the bench binaries. That is a separate build/ABI problem, not this hang.

Method: failed pr.yaml runs created on or after 2026-09-08 (230 of 555), then failed python-tests jobs, then FAILED lines in their logs. The count may be incomplete if some job listings could not be fetched.

主要言語
Cuda
スター
869
フォーク
243
平均マージ
3日 15時間
マージ済み PR（30日）
78

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

NVIDIA/cuvs のほかの issue

NVIDIA/cuvs の issue をすべて見る

似ている issue

Machine Learning の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。