partition_unique leaks metadata references for completed and duplicate records
評估
研究方向
Start at partition_unique and inspect the shared retain/release helpers, then add regressions for keep="first", keep="last", and metadata shape. Verify discarded references are released immediately, synchronous completion leaves no references retained, and buffered downstream nodes keep references until processing finishes. An open pull request (#502) is already linked to this issue.
由索引模型根據 Issue 內容生成。
描述
partition_unique retains metadata references for every input, but does not release them when a duplicate is discarded (keep="first") or replaced (keep="last"). It also emits nested metadata lists, while the shared retain/release helpers expect a flat list of dictionaries. Even a completed partition with a synchronous sink leaves its references retained.
Reproduced on master 5267116f49fc2257e39b44abd8d1e5c5460715f8, Windows Python 3.12.14:
from streamz import Stream, RefCounter
for keep in ("first", "last"):
source = Stream()
output = source.partition_unique(2, keep=keep).sink_to_list()
refs = [RefCounter() for _ in range(3)]
for value, ref in zip((1, 1, 2), refs):
source.emit(value, metadata=[{"ref": ref}])
print(keep, output, [ref.count for ref in refs])
Actual: both modes output [(1, 2)] with counts [1, 1, 1]. Expected: [0, 0, 0] after the synchronous sink finishes; discarded records should release their references immediately. Buffered downstream nodes must still retain references until they finish processing.
I plan a narrow fix in partition_unique: release discarded metadata, flatten retained metadata for emission, and add first/last ownership and metadata-shape regressions. Searches found the original feature PR #389 but no current fix for this issue. Is there existing work or a preferred approach?
AI disclosure: this report and proposed contribution were prepared and reproduced by Codex under the account owner authorization. I am not claiming an active production use case or manual human review.
- 主要語言
- Python
- 星號
- 1.3k
- 分支
- 150
- 平均合併
- 17 小時 39 分鐘
- 30 天內合併 PR
- 1
環境準備
- 提供 Dockerfile 或 Docker Compose 檔案
- 沒有 Pull Request 範本
- 閱讀貢獻指南
從這裡開始
- 先讀完整個 Issue，再讀專案的貢獻指南。
- 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
- Fork 儲存庫，在一個分支上完成修改。
- 送出 Pull Request，並在描述裡引用這個 Issue 編號。
python-streamz/streamz 的其他 Issue
-
難度 3/5 1-2 天 新手友好度 35/100
python-streamz/streamz#481 · 4 則留言 ·
-
難度 5/5 一週以上 新手友好度 25/100
python-streamz/streamz#480 · 2 則留言 ·
-
難度 5/5 一週以上 新手友好度 20/100
python-streamz/streamz#479 · 1 則留言 ·
-
難度 5/5 一週以上 新手友好度 20/100
python-streamz/streamz#478 · 6 則留言 ·
-
難度 4/5 3-5 天 新手友好度 10/100
python-streamz/streamz#476 · 17 則留言 · 2 個 reaction ·
查看 python-streamz/streamz 的全部 Issue
相似的 Issue
-
難度 2/5 1-3 小時 新手友好度 74/100
vllm-project/vllm#61131 ·
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 78/100
kubernetes-client/python#2735 ·
維護者通常 1 天內回覆
-
desktop help wanted test
難度 2/5 1-3 小時 新手友好度 78/100
維護者通常 1 天內回覆
-
docs pydanty:is-working
難度 2/5 1-3 小時 新手友好度 68/100
pydantic/pydantic-ai#10113 ·
維護者通常 1 天內回覆
-
Tutorial unit test leaks sample data into tutorial doctest可能已有人在做 @Gambit-Checkmate 今天認領。 未關閉
難度 2/5 1-3 小時 新手友好度 84/100