netdata state container: runaway FD use
Personne n'a encore pris cette issue.
Évaluation
- Difficulté
- 4/5
- Temps estimé
- 3-5 jours
- Accessibilité débutants
- 30/100
- Type d'issue
- Bug
- Clarté
- À clarifier
- Activité
- À l'abandon
- Stack technique
- helm, kubernetes
- Domaine
- infrastructure
Piste de recherche
Commencez par reproduire le déploiement du state-container de Netdata et examinez comment le chart Helm configure ses chemins WAL et de base de données. Utilisez la sortie de lsof dans l’issue pour vérifier si des handles vers netdata-meta.db et context-meta.db sont ouverts de manière répétée ; le travail est terminé lorsque le state-container n’épuise plus les descripteurs de fichiers disponibles.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Description
Netdata needed to be removed due to consuming ALL available file descriptors (mildly entertaining that this isn't a monitored metric in netdata, that I could find).
From lsof it appears that it is just opening the WAL/db in a loop:
netdata 1781992 1782297 RRDCONTEX 201 15r CHR 1,3 0t0 6 /dev/null
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 16u sock 0,8 0t0 62179404 protocol: UNIX-STREAM
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 17ur REG 0,121 4587520 14044310 /var/cache/netdata/netdata-meta.db
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 18u REG 0,121 4906952 14044311 /var/cache/netdata/netdata-meta.db-wal
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 19ur REG 0,121 32768 14044312 /var/cache/netdata/netdata-meta.db-shm
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 20ur REG 0,121 4096 14044313 /var/cache/netdata/context-meta.db
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 21u REG 0,121 37112 14044314 /var/cache/netdata/context-meta.db-wal
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 22ur REG 0,121 32768 14044315 /var/cache/netdata/context-meta.db-shm
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 23ur REG 0,121 4587520 14044310 /var/cache/netdata/netdata-meta.db
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 24u REG 0,121 4906952 14044311 /var/cache/netdata/netdata-meta.db-wal
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 25u a_inode 0,14 0 12713 [eventpoll]
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 26r FIFO 0,13 0t0 62177496 pipe
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 27w FIFO 0,13 0t0 62177496 pipe
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 28u a_inode 0,14 0 12713 [eventfd]
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 29u a_inode 0,14 0 12713 [eventpoll]
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 30r FIFO 0,13 0t0 62171736 pipe
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 31w FIFO 0,13 0t0 62171736 pipe
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 32u a_inode 0,14 0 12713 [eventfd]
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 33r FIFO 0,13 0t0 62176463 pipe
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 34u sock 0,8 0t0 62161843 protocol: UDPv6
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 35w FIFO 0,13 0t0 62166944 pipe
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 36u sock 0,8 0t0 62161844 protocol: UDP
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 37u sock 0,8 0t0 62161847 protocol: TCPv6
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 38u sock 0,8 0t0 62161848 protocol: TCP
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 39u sock 0,8 0t0 62178506 protocol: UNIX-STREAM
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 40u a_inode 0,14 0 12713 [eventfd]
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 41u a_inode 0,14 0 12713 [eventpoll]
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 42r FIFO 0,13 0t0 62178505 pipe
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 43w FIFO 0,13 0t0 62178505 pipe
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 44r CHR 1,3 0t0 6 /dev/null
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 45r FIFO 0,13 0t0 62160878 pipe
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 46w FIFO 0,13 0t0 62160878 pipe
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 47r REG 0,273 0 62169795 /proc/1/task/209/stat
lsof: no pwd entry for UID 201
netdata 1781992 1782297 RRDCONTEX 201 48u sock 0,8 0t0 62194554 protocol: TCP
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 cwd DIR 0,121 4096 13901928 /etc/netdata
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 rtd DIR 0,121 4096 14034715 /
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 txt REG 0,121 6986240 13902420 /usr/sbin/netdata
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 mem-r REG 0,121 14044315 /var/cache/netdata/context-meta.db-shm (stat: No such file or directory)
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 mem-r REG 0,121 14044312 /var/cache/netdata/netdata-meta.db-shm (stat: No such file or directory)
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 mem REG 0,121 13896738 /usr/lib/libzstd.so.1.5.5 (stat: No such file or directory)
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 mem REG 0,121 13896436 /usr/lib/libgcc_s.so.1 (stat: No such file or directory)
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 mem REG 0,121 13896627 /usr/lib/libstdc++.so.6.0.30 (stat: No such file or directory)
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 mem REG 0,121 13896405 /usr/lib/libbson-1.0.so.0.0.0 (stat: No such file or directory)
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 mem REG 0,121 13896522 /usr/lib/libmongoc-1.0.so.0.0.0 (stat: No such file or directory)
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 mem REG 0,121 13896615 /usr/lib/libsnappy.so.1.1.10 (stat: No such file or directory)
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 mem REG 0,121 13896595 /usr/lib/libprotobuf.so.32.0.12 (stat: No such file or directory)
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 mem REG 0,121 13896491 /usr/lib/libjson-c.so.5.2.0 (stat: No such file or directory)
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 mem REG 0,121 13896512 /usr/lib/liblz4.so.1.9.4 (stat: No such file or directory)
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 mem REG 0,121 13896695 /usr/lib/libuv.so.1.0.0 (stat: No such file or directory)
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 mem REG 0,121 13895904 /lib/libuuid.so.1.3.0 (stat: No such file or directory)
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 mem REG 0,121 11672957 /lib/libssl.so.3 (stat: No such file or directory)
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 mem REG 0,121 11672956 /lib/libcrypto.so.3 (stat: No such file or directory)
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 mem REG 0,121 11672959 /lib/libz.so.1.2.13 (stat: No such file or directory)
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 mem REG 0,121 11672951 /lib/ld-musl-x86_64.so.1 (stat: No such file or directory)
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 0w CHR 1,3 0t0 6 /dev/null
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 1w FIFO 0,13 0t0 62178474 pipe
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 2w FIFO 0,13 0t0 62178474 pipe
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 3u sock 0,8 0t0 62173481 protocol: UNIX
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 4w FIFO 0,13 0t0 62178474 pipe
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 5w FIFO 0,13 0t0 62178474 pipe
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 6w REG 0,121 0 14044307 /var/log/netdata/health.log
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 7u sock 0,8 0t0 62173487 protocol: TCP
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 8u sock 0,8 0t0 62173488 protocol: TCPv6
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 9u a_inode 0,14 0 12713 [eventpoll]
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 10r FIFO 0,13 0t0 62179402 pipe
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 11w FIFO 0,13 0t0 62179402 pipe
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 12r FIFO 0,13 0t0 62179403 pipe
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 13w FIFO 0,13 0t0 62179403 pipe
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 14u a_inode 0,14 0 12713 [eventfd]
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 15r CHR 1,3 0t0 6 /dev/null
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 16u sock 0,8 0t0 62179404 protocol: UNIX-STREAM
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 17ur REG 0,121 4587520 14044310 /var/cache/netdata/netdata-meta.db
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 18u REG 0,121 4906952 14044311 /var/cache/netdata/netdata-meta.db-wal
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 19ur REG 0,121 32768 14044312 /var/cache/netdata/netdata-meta.db-shm
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 20ur REG 0,121 4096 14044313 /var/cache/netdata/context-meta.db
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 21u REG 0,121 37112 14044314 /var/cache/netdata/context-meta.db-wal
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 22ur REG 0,121 32768 14044315 /var/cache/netdata/context-meta.db-shm
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 23ur REG 0,121 4587520 14044310 /var/cache/netdata/netdata-meta.db
lsof: no pwd entry for UID 201
netdata 1781992 1782298 REPLAY[1] 201 24u REG 0,121 4906952 14044311 /var/cache/netdata/netdata-meta.db-wal
approximately ^ 100's of thousands of times until the node eventually becomes unavailable due to an inability to open any more file descriptors.
- Langage dominant
- Go Template
- Étoiles
- 177
- Forks
- 82
- Merge moyen
- 1 j 55 min
- PR mergées (30 j)
- 4
Préparer son environnement
- Aucun Dockerfile ni fichier Docker Compose
- Aucun modèle de pull request
- Lire le guide de contribution
Par où commencer
- Lisez l'issue en entier, puis le guide de contribution du projet.
- Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
- Forkez le dépôt et travaillez sur une branche.
- Ouvrez une pull request qui référence le numéro de l'issue.
Autres issues de netdata/helmchart
-
Difficulté 5/5 Plus d'une semaine Accessibilité débutants 25/100
-
Difficulté 2/5 1-3 heures Accessibilité débutants 35/100
-
Difficulté 4/5 3-5 jours Accessibilité débutants 35/100
-
Difficulté 3/5 1-2 jours Accessibilité débutants 42/100
Toutes les issues de netdata/helmchart
Issues similaires
-
Difficulté 2/5 1-3 heures Accessibilité débutants 72/100
-
Difficulté 2/5 1-3 heures Accessibilité débutants 65/100
greenelab/lab-website-template#357 · 1 commentaire ·
-
Difficulté 2/5 1-3 heures Accessibilité débutants 85/100
PolicyEngine/givecalc#95 ·
Les mainteneurs répondent en général sous 1 jour
-
Difficulté 2/5 1-3 heures Accessibilité débutants 82/100
PolicyEngine/crfb-tob-impacts#132 ·
-
Difficulté 2/5 1-3 heures Accessibilité débutants 80/100
PolicyEngine/cliff-watch#43 ·