register_vector leaks dumper classes on every call
还没有人认领这个 Issue。
评估
- 难度
- 3/5
- 预计耗时
- 1-2 天
- 新手友好度
- 55/100
调研方向
Start with register_vector_info and the dumper classes in pgvector/psycopg/vector.py, then compare their registration with psycopg’s hstore.py example linked in the issue. Use the supplied reproduction with psycopg’s C extension to check class counts before and after repeated calls; done means the dumper classes are no longer retained after contexts are deleted.
由索引模型根据 Issue 内容生成。
描述
Hi!
Investigating a memory leak in our API, we found out that register_vector (and register_vector_async) creates 8 classes on every call that are never garbage collected if psycopg's C extension is in use, even after the connection is closed.
This is because these classes are stored by psycopg in a dict shared by the whole python process, and never cleared.
pgvector python 0.4.2 (but the code has not been updated on master) and psycopg 3.3.2
To reproduce:
import gc
from types import SimpleNamespace
import psycopg
from pgvector.psycopg.vector import (
VectorBinaryDumper,
VectorDumper,
register_vector_info,
)
from psycopg.adapt import AdaptersMap
from psycopg.types import TypeInfo
def count() -> int:
gc.collect()
return sum(
1
for base in (VectorDumper, VectorBinaryDumper)
for c in base.__subclasses__()
if c.__name__ == ""
)
print(psycopg.__version__, psycopg.pq.__impl__)
before = count()
for _ in range(1000):
ctx = SimpleNamespace(adapters=AdaptersMap(psycopg.adapters))
register_vector_info(ctx, TypeInfo("vector", 16385, 16386))
del ctx
print("leaked dumper classes:", count() - before) # expected 0, prints 2000
Caching these classes as done by psycopg could fix the issue : https://github.com/psycopg/psycopg/blob/99356800e90aa0e7b604f652d714ef0f33422b47/psycopg/psycopg/types/hstore.py#L225
- 主要语言
- Python
- 星标
- 1.5k
- 派生
- 100
- 平均合并
- 13 分钟
- 30 天内合并 PR
- 1
环境准备
这个项目没有提供开发容器、Dockerfile 或贡献指南，环境需要你自己搭建：先看它的 README，通用步骤见我们的新手贡献指南。
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
pgvector/pgvector-python 的其他 Issue
-
难度 2/5 1-3 小时 新手友好度 72/100
pgvector/pgvector-python#169 ·
查看 pgvector/pgvector-python 的全部 Issue
相似的 Issue
-
难度 2/5 1-3 小时 新手友好度 77/100
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 67/100
维护者通常 1 天内回复
-
ChaosEngine: backup-remote doctor row passes while consumer mode hides bot memory from git可能已有人在做 @MohabMohie 今天认领。 未关闭agentic
难度 2/5 1-3 小时 新手友好度 76/100
ShaftHQ/SHAFT_ENGINE#7263 ·
维护者通常 1 天内回复
-
bug
难度 2/5 1-3 小时 新手友好度 87/100
huggingface/smolagents#3045 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 72/100
维护者通常 1 天内回复