Continuous memory increase in Sanic web server after calling `loop.start_tls(...)` in `connect_utils._create_ssl_connection`
@fantix is already working on this.
Since Feb 17, 2023.
Assessment
This issue has not been assessed yet.
Description
- asyncpg version:
0.2.7 - PostgreSQL version:
PostgreSQL 13.3 (Debian 13.3-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit - Do you use a PostgreSQL SaaS? If so, which? Can you reproduce
the issue with a local PostgreSQL install?: I use PostgreSQL installed in docker imagekartoza/postgis:pr-333-13-3.1 - Python version:
3.8.10 - Platform:
20.04.1-Ubuntu x86_64 - Do you use pgbouncer?: Not sure, I use it indirectly. (behind
sqlalchemy) - Did you install asyncpg with pip?: Yes
- If you built asyncpg locally, which version of Cython did you use?: /
- Can the issue be reproduced under both asyncio and
uvloop?: I see the code below is based on uvloop.
Background: I built a simple web server using sqlalchemy and sanic, then created a request handler function that can query my local PostgreSQL server running in docker. However, I saw continuous memory increase while benching my handler function.
After running tracemalloc for help, I got the result below:
Top 10 lines
#1: /home/tusimple/Workspace/projects/map-workdata-manager/venv/lib/python3.8/site-packages/asyncpg/connect_utils.py:700: 5670.1 KiB
new_tr = await loop.start_tls(
The code for creating the async_engine is:
from sqlalchemy.ext.asyncio import create_async_engine
engine = create_async_engine("postgresql+asyncpg://{}:{}@{}/{}".format(...))
At first, I tried adding ?ssl=disable after the previous connect string in order to prevent connect_utils._create_ssl_connection calls, and it works.
Then, I tried to make some stupid changes to the source code of asyncpg, because I think this SSL transport is not closed after creating new transport. (I'm not sure is that correct)
# line 697
if hasattr(loop, 'start_tls'):
if do_ssl_upgrade:
try:
new_tr = await loop.start_tls(
tr, pr, ssl_context, server_hostname=host) # <--- here
except (Exception, asyncio.CancelledError):
tr.close()
raise
else:
new_tr = tr
# line 706
Finally, I modified the code starts from line 702 to:
except (Exception, asyncio.CancelledError):
raise
finally:
tr.close()
After this modification, my web server keeps working fine, but I don't know whether this is a proper modification.
If you have any questions or ideas, please let me know, thank you so much!
- Dominant language
- Python
- Stars
- 8.1k
- Forks
- 469
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 9
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from MagicStack/asyncpg
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
MagicStack/asyncpg#1357 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
MagicStack/asyncpg#1354 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 72/100
MagicStack/asyncpg#1342 ·
-
Difficulty 3/5 1-2 days Newbie friendliness 56/100
MagicStack/asyncpg#1340 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 28/100
MagicStack/asyncpg#1337 ·
All issues in MagicStack/asyncpg
Similar issues
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
learningequality/ricecooker#747 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
BSData/horus-heresy-3rd-edition#3171 ·
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
run-llama/llama_index#23199 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
KhronosGroup/glTF-Blender-IO#2769 ·