No visibility on Juju controller when it gets detached from the dqlite cluster

Aperta
#155 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
45/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
python

Direzione di ricerca

Start by tracing how the charm determines and reports the controller unit's status, then find where it can check whether the Juju API is listening on port 17070. The issue gives no file or test paths, so first locate the status checks and their tests. Done means a detached controller is reported as blocked or in error so an administrator can investigate.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

A node in a Juju cluster can get detached from the cluster (the dqlite data may be corrupted or outdated) and this can go unnoticed since Juju status will still display the unit as active and jujud is still running.

The only symptom of the issue is the fact that the Juju API in the node cannot start and the unit is effectively not listening on port 17070 which causes API calls hitting this node to fail/timeout. The affected controller Juju logs would show something like:

2026-06-05 23:44:41 ERROR juju.worker.dependency engine.go:695 "lease-manager" manifold worker returned unexpected error: catacomb 0xc0019f3a48 is dying
2026-06-05 23:44:41 DEBUG juju.worker.dependency engine.go:618 "db-accessor" manifold worker stopped: create node: failed to set bind address "@dqlite-3297041220608546238": 1
2026-06-05 23:44:41 ERROR juju.worker.dependency engine.go:695 "db-accessor" manifold worker returned unexpected error: create node: failed to set bind address "@dqlite-3297041220608546238": 1

When the issue is outdated dqlite data (there may be other I'm not aware of), a cross-inspection of /var/lib/juju/dqlite/ shows that the affected controller's dqlite data is outdated.

I'd argue that we should have better visibility on this (and potentially other issues). The easiest I can think of to achieve this is to mark the unit as blocked or in error if it's not listening on the port. This should trigger an admin to investigate and potentially resolve this and other issues.

Lingua principale
Python
Stelle
3
Fork
21
Merge medio
2g 22h
PR unite (30g)
1

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di juju/juju-controller

Tutte le issue di juju/juju-controller

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.