Bug: Redis connection pool exhaustion on /downloads/ endpoint under high concurrency
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 38/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Quiet
- Tech stack
- python, redis
- Domain
- backend, performance
Research direction
Start by reviewing gunicorn.conf, apps/downloads/views.py, and config/settings/cache.py, then inspect concurrency behavior using the application's Redis connection metrics and logs. Verify whether the downloads endpoint exhausts its pool and blocks on Redis or Celery work. Done means burst traffic no longer causes the reported exhaustion, latency, or connection errors, with cached statistics available under Redis pressure.
Written by the indexing model from the issue text.
Description
Describe the bug
Under high concurrency conditions, particularly during new Python release events, the /downloads/ endpoint may experience Redis connection pool exhaustion. The application is configured with gunicorn.conf setting workers = 2, which creates a bottleneck when combined with synchronous Redis operations in the downloads view.
This manifests as increased tail latency and potential redis.exceptions.ConnectionError entries in application logs during traffic spikes.
To Reproduce
- Access https://www.python.org/downloads/ during a high-traffic event (e.g., new Python release announcement)
- Monitor response times and Redis connection metrics via application monitoring
- Observe elevated error rates and latency under sustained concurrent requests
Expected behavior
- Connection pooling should gracefully handle burst traffic without exhaustion
- Stale-while-revalidate caching pattern should serve cached download statistics when Redis is under pressure
- Celery tasks triggered from download views should be truly asynchronous, not blocking Redis operations on the main request thread
URL to the issue
https://www.python.org/downloads/
Screenshots
N/A (requires production monitoring access: Datadog dashboards, Sentry error logs, or Redis connection pool metrics)
Browsers
Microsoft Edge
Operating System
Windows
Browser Version
N/A
Relevant log output
# Example of expected log entry during pool exhaustion (needs verification in production):
redis.exceptions.ConnectionError: Connection pool exhausted. Unable to acquire connection within timeout.
# Relevant files for investigation:
# - gunicorn.conf (workers = 2)
# - apps/downloads/views.py
# - config/settings/cache.py
Additional context
Recent PR #2764 added a sitetree cache purge command to the Makefile, indicating the team's awareness of caching best practices. Similar patterns should be applied to high-traffic endpoints like /downloads/. Django cache framework documentation recommends connection pooling for production deployments. The project's dependency modernization efforts (e.g., pygments bump in PR #2977) suggest the team is actively maintaining performance-critical components.
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 700
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 14
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from python/pythondotorg
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
python/pythondotorg#3137 · 2 comments ·
-
app/pages bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
python/pythondotorg#2677 · 3 comments ·
-
backend easy
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
python/pythondotorg#2453 · 8 comments ·
-
bug good-first-issue
Difficulty 4/5 3-5 days Newbie friendliness 30/100
python/pythondotorg#3125 · 1 comment ·
-
bug
Difficulty 3/5 1-2 days Newbie friendliness 74/100
python/pythondotorg#3094 · 5 comments ·
All issues in python/pythondotorg
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
anthropics/skills#1811 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
speaches-ai/speaches#678 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
datalayer/mcp-compose#42 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
conda-forge/spacy-feedstock#177 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
UKGovernmentBEIS/inspect_evals#2523 ·