Logger: stale config guard can reset a newer configuration, and resets sinks without the mutex

Aperta
  • Qualcosa non va?
#1,831 1 commento 0 reazioni 1 assegnatario Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@kaatish ci sta già lavorando.

Dal 31/8/2026.

Valutazione

Questa issue non è ancora stata valutata.

Descrizione

bug

logger_config_guard's destructor resets the logger unconditionally and without holding g_guard_mutex (cpp/src/utilities/logger.cpp:141-142, and reset_default_logger at :127-130):

struct logger_config_guard {
  ~logger_config_guard() { cuopt::reset_default_logger(); }
};

Two problems, both on main today.

A stale guard can reset a newer configuration. A guard's refcount reaching zero expires g_active_guard before the destructor body runs. In that window another thread calling init_logger_t sees no live configuration, applies its own and installs a new guard. The first destructor then runs and resets the logger out from under it, so that thread's messages go to the buffer sink and are dropped while it still holds a live handle.

Unsynchronized sink mutation. reset_default_logger does sinks().clear() / push_back() with no lock, while init_logger_t's configure path mutates the same vector under g_guard_mutex (:151, :160-168). Two threads can mutate it concurrently.

Reproducing

Needs the refcount to hit zero exactly while another thread is inside the configure path. Under contention g_active_guard.lock() usually succeeds instead, so the window is rarely entered and a straightforward stress test does not reliably hit it. A TSAN build over concurrent solves that each construct an init_logger_t is the more likely way to surface the sink mutation.

Suggested fix

Give each configuration a generation number, record it in the guard, and have the destructor take g_guard_mutex and reset only if its own generation is still current. That closes the stale-reset window and puts sink mutation on the configure and reset paths under the same lock.

Exposure is limited in practice: configuration happens at solve boundaries rather than concurrently with a solve that is emitting. Split out of #1778, which found it but where the fix was out of scope.

Lingua principale
Cuda
Stelle
1.1k
Fork
255
Merge medio
1g 22h
PR unite (30g)
102

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di NVIDIA/cuopt

Tutte le issue di NVIDIA/cuopt

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.