Supervisor/supervisor

Bad file descriptor during cleanup_fds

Open

#1110 opened on Jun 13, 2018

View on GitHub
 (18 comments) (0 reactions) (0 assignees)Python (7,941 stars) (1,233 forks)batch import
help wanted

Description

I sometimes get (when quitting supervisor with CTRL+C) the following trace:

^C2018-06-13 22:32:21,219 WARN received SIGINT indicating exit request
2018-06-13 22:32:21,220 INFO waiting for web to die
2018-06-13 22:32:21,270 INFO stopped: web (exit status 0)
2018-06-13 22:32:21,270 INFO Closing socket tcp://0.0.0.0:8000
Exception ignored in: <bound method tail_f_producer.__del__ of <supervisor.http.tail_f_producer object at 0x7f599b85e2e8>>
Traceback (most recent call last):
  File "/home/florian/.local/lib/python3.6/site-packages/supervisor/http.py", line 649, in __del__
    self._close()
  File "/home/florian/.local/lib/python3.6/site-packages/supervisor/http.py", line 675, in _close
    self.file.close()
OSError: [Errno 9] Bad file descriptor

my supervisor config file:

[supervisord]
logfile = /tmp/supervisord.log

[inet_http_server]
port = 127.0.0.1:9001

[supervisorctl]
serverurl = http://127.0.0.1:9001

[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[fcgi-program:web]
# Patched gunicorn supporting fcgi-program: https://github.com/benoitc/gunicorn/pull/1809
command = python -m gunicorn.app.wsgiapp --reuse-port --workers 4 --bind fd://0 test:application
directory = %(here)s
socket = tcp://0.0.0.0:8000

I cannot yet reliably reproduce :/

Contributor guide

Tech stack
python
Domain
backend
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
stale
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
needs investigation
Prerequisites
knowledge of Pythonfile descriptors
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
45
Research direction
Investigate the cleanup code in supervisor/http.py, specifically the tail f producer. close method and its interaction with socket closing. The bug is intermittent and occurs during shutdown. Look for potential race conditions where the file descriptor is closed before the del method is called. Consider adding error handling to ignore EBADF errors or ensuring proper ordering of cleanup.