Todo cluster descartavel que instala o chart deixa um PD de 50 GB para tras
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 62/100
Research direction
Start with templates/pvc-logs.yaml, where the 50 Gi PVC is created when logs.persistence.enabled defaults to true, and with the stack_up function in test/gcp/, which should pass --set logs.persistence.enabled=false. Check the "What a cluster delete does NOT remove" section of test/gcp/README.md. Done when the GCP test runners install the chart without a persistent log volume and the README notes that the chart creates one per install by default. The teardown.sh --leftovers change is out of scope and needs its own discussion.
Written by the indexing model from the issue text.
Description
Encontrado varrendo a conta GCP depois de um dia de experimentos: quatro
discos de 50 GB orfaos, zero usuarios, todos criados pelo nosso proprio chart.
NAME SIZE_GB STATUS USERS
pvc-4ecb46ac-... 50 READY 0
pvc-afd1b6b7-... 50 READY 0
pvc-bb907290-... 50 READY 0
pvc-f7955ee3-... 50 READY 0
A procedencia confirma de onde vieram:
{"kubernetes.io/created-for/pvc/name":"leoflow-logs",
"kubernetes.io/created-for/pvc/namespace":"leoflow-system",
"storage.gke.io/created-by":"pd.csi.storage.gke.io"}
200 GB a USD 0.10/GB/mes = USD 20 por mes se ficassem, o que e mais do que
gastei em clusters no dia inteiro. Apagados a mao.
Por que acontece
templates/pvc-logs.yaml cria um PVC de 50 Gi por install, com
logs.persistence.enabled: true por padrao. Isso esta certo para producao: sem
ele os logs de task vivem num emptyDir e somem no restart do pod (#227).
O problema e o ciclo de vida. Apagar o cluster GKE nao apaga o PD que o CSI
provisionou. O disco sobrevive, sem nenhum label do nosso purpose=leoflow-experiment,
entao teardown.sh --leftovers so consegue LISTAR, nunca reclamar: nao ha como
um script distinguir o nosso do disco de producao de outra pessoa.
Resultado: todo cluster descartavel que instala o chart deixa 50 GB para
tras, e o custo e invisivel porque nao aparece em nenhum lugar que a gente
olhe durante o experimento.
Escopo, sendo justo com o chart
Isto nao e bug numa instalacao normal. Um PVC que sobrevive ao cluster e o
comportamento correto de um volume de dados: e assim que se recupera log de
incidente depois de recriar o control plane. O default esta certo.
E um descasamento entre esse default e o uso descartavel, e ele mordeu tres
vezes hoje sem ninguem notar.
Opcoes
-
Os runners de
test/gcp/desligam a persistencia.stack_uppassa
--set logs.persistence.enabled=falsee o control plane usa emptyDir. Um
experimento que mede tempo de despacho nao precisa de log durando alem do
cluster. Corrige a causa no unico lugar onde ela e indesejada, sem mexer no
default de producao. Preferida. -
teardown.sh --leftoverspassa a reclamar PDs cujodescriptionaponta
para o namespaceleoflow-systemE que estejam sem usuarios. Mais util em
geral, e mais arriscado: um cluster de producao com o control plane escalado
a zero tem exatamente essa forma, e o custo de errar e o log de incidente de
outra pessoa. -
Documentar em
test/gcp/README.md. A secao "What a cluster delete does
NOT remove" ja descreve a classe e cita "leoflow's staging volumes are
exactly that shape", mas nao diz que o proprio chart cria um por install.
Vale de qualquer forma, junto com (1).
Sugiro (1) + (3). A (2) merece discussao propria: reclamar disco automaticamente
e o tipo de conveniencia que se paga caro uma vez.
- Dominant language
- Go
- Stars
- 46
- Forks
- 1
- Avg merge
- 13h 11m
- Merged PRs (30d)
- 684
Getting set up
- Ships a Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from dexadata/dexaflow
-
area/docs documentation priority/p3
Difficulty 1/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
area/ci enhancement priority/p3
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
area/api priority/p3
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 3-5 days Newbie friendliness 62/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
All issues in dexadata/dexaflow
Similar issues
-
automation doc-healer documentation
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
Maintainers usually reply within 1 day
-
Anthropic /v1/messages: only the last thinking block of an assistant message is keptPossibly taken @Shayan12456 claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
ollama/ollama#19415 · 1 comment ·
Maintainers usually reply within 1 day
-
area:recipes event:contribfest format:human-first source:agent status:needs-triage type:recipe
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
cadence-workflow/cadence-ai-samples#140 ·
Maintainers usually reply within 1 day
-
customer issue
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
dolthub/doltgresql#3716 ·
Maintainers usually reply within 1 day
-
ux
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
evcc-io/evcc#35605 · 1 comment ·
Maintainers usually reply within 1 day