Hacktoberfest 2026: as issues que os mantenedores marcaram para outubro, abertas e boas para iniciantes. Ver issues do Hacktoberfest

partition_unique leaks metadata references for completed and duplicate records

Aberta
  • Algo errado?
#501 0 comentários 0 reações 0 responsáveis Ver no GitHub

@harbinresearcher já está trabalhando nisso.

Desde 11/10/2026.

  • #502 de @harbinresearcher — aberto

Avaliação

Dificuldade
3/5
Tempo estimado
1-2 dias
Facilidade para iniciantes
25/100
Tipo de issue
Bug
Clareza
Claramente especificada
Status de atividade
Estagnada
Stack de tecnologia
python
Domínio
backend, testing

Direção de pesquisa

Start at partition_unique and inspect the shared retain/release helpers, then add regressions for keep="first", keep="last", and metadata shape. Verify discarded references are released immediately, synchronous completion leaves no references retained, and buffered downstream nodes keep references until processing finishes. An open pull request (#502) is already linked to this issue.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

partition_unique retains metadata references for every input, but does not release them when a duplicate is discarded (keep="first") or replaced (keep="last"). It also emits nested metadata lists, while the shared retain/release helpers expect a flat list of dictionaries. Even a completed partition with a synchronous sink leaves its references retained.

Reproduced on master 5267116f49fc2257e39b44abd8d1e5c5460715f8, Windows Python 3.12.14:

from streamz import Stream, RefCounter

for keep in ("first", "last"):
    source = Stream()
    output = source.partition_unique(2, keep=keep).sink_to_list()
    refs = [RefCounter() for _ in range(3)]
    for value, ref in zip((1, 1, 2), refs):
        source.emit(value, metadata=[{"ref": ref}])
    print(keep, output, [ref.count for ref in refs])

Actual: both modes output [(1, 2)] with counts [1, 1, 1]. Expected: [0, 0, 0] after the synchronous sink finishes; discarded records should release their references immediately. Buffered downstream nodes must still retain references until they finish processing.

I plan a narrow fix in partition_unique: release discarded metadata, flatten retained metadata for emission, and add first/last ownership and metadata-shape regressions. Searches found the original feature PR #389 but no current fix for this issue. Is there existing work or a preferred approach?

AI disclosure: this report and proposed contribution were prepared and reproduced by Codex under the account owner authorization. I am not claiming an active production use case or manual human review.

Linguagem predominante
Python
Estrelas
1.3k
Forks
151
Merge médio
17h 39min
PRs com merge (30d)
1

Preparar o ambiente

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de python-streamz/streamz

Todas as issues de python-streamz/streamz

Issues semelhantes

Mais issues de Python

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.