KRaft: a new controller can bootstrap against itself and never join the quorum

Aberta
#1,062 0 comentários 0 reações 1 responsável Ver no GitHub

Mantenedores costumam responder em até 1 dia

@razvan já está trabalhando nisso.

Desde 13/11/2026.

  • #1063 de @razvan — aberto

Avaliação

Esta issue ainda não foi avaliada.

Descrição

type/bug
Description

A controller that starts without metadata (a new pod, e.g. after scaling up a controller role group) finds the quorum leader through controller.quorum.bootstrap.servers. Outside Kerberos, that list has one entry per controller role group: the role group's headless Service. That Service sets publishNotReadyAddresses: true, so the name also resolves to the starting pod's own IP.

Kafka treats each entry in controller.quorum.bootstrap.servers as one bootstrap node and resolves its name when it connects. If the starting pod's lookup returns its own IP, the connection succeeds, so Kafka never tries another address. The pod asks itself for the leader, gets no answer, and repeats this forever.

This is a race in the quorum management added in #1010.

Symptoms
  • The new controller stays in Raft state unattached (epoch 0) and never becomes an observer. The quorum's replication status doesn't list it at all.
  • Its quorum-manager sidecar logs Local Raft state is 'unattached', nothing to do indefinitely, so it never calls add-controller.
  • The kafka container never becomes ready, and the StatefulSet rollout stalls. Because the StatefulSet uses OrderedReady, later replicas are not created either.
  • The pod's only connections on the controller port go to its own IP.
  • Deleting the pod usually clears it: the replacement resolves another IP and joins normally.
Linguagem predominante
Rust
Estrelas
31
Forks
9
Merge médio
4d 5h
PRs com merge (30d)
14

Preparar o ambiente

  • Sem Dockerfile nem arquivo Docker Compose
  • Tem um modelo de pull request
  • Sem guia de contribuição

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de stackabletech/kafka-operator

Todas as issues de stackabletech/kafka-operator

Issues semelhantes

Mais issues de Rust

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.