Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

bug(langchain): failed streaming LLM runs retain IDs in callback memo

Aperta
  • Qualcosa non va?
#1,977 0 commenti 0 reazioni 1 assegnatario Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@hassiebp ci sta già lavorando.

Dal 11/10/2026.

  • #1978 di @dhionson2 — aperta

Valutazione

Questa issue non è ancora stata valutata.

Descrizione

bug integration-langchain sdk-python

Bug description

LangchainCallbackHandler retains completed LLM run IDs in _updated_completion_start_time_memo when a streaming execution terminates with an error.

The successful completion path (on_llm_end) removes the run ID from the set, but the error path (on_llm_error) does not perform the same cleanup.

As a result, long-lived callback handlers reused across failed streaming executions can accumulate stale UUID entries.

Root cause

In langfuse/langchain/CallbackHandler.py:

  • on_llm_new_token() records the first-token timestamp and adds the run_id to _updated_completion_start_time_memo.
  • on_llm_end() removes the run_id in its finally block.
  • on_llm_error() ends the observation but does not remove the run_id from the memo.
  • _detach_observation() and _reset() do not clear this set.

Steps to reproduce

Using a LangchainCallbackHandler with an in-memory Langfuse test client:

  1. Start an LLM run using on_llm_start().
  2. Simulate the first streamed token using on_llm_new_token().
  3. Confirm the run ID exists in _updated_completion_start_time_memo.
  4. Terminate the run using on_llm_error(RuntimeError("synthetic failure"), run_id=run_id).
  5. Inspect _updated_completion_start_time_memo.

Actual behavior: The failed run ID remains in the set after the observation has ended.

Expected behavior: The failed run ID should be removed, while unrelated active run IDs remain untouched.

Impact

This causes incremental retention of UUID identifiers when streaming LLM executions fail and the same callback handler is reused.

The observed retention is limited to UUID entries. There is no evidence of retained generations, spans, or a significant process-level memory leak.

Proposed solution

Ensure on_llm_error() discards its own run_id from _updated_completion_start_time_memo in a finally block, mirroring the existing cleanup in on_llm_end().

This should preserve:

  • Active nested or concurrent run state.
  • Existing OpenTelemetry context handling.
  • LangGraph control-flow and resume behavior.
  • Current public SDK behavior.

Validation

I reproduced this issue locally using Langfuse Python SDK 4.17.0 and created a deterministic regression test.

Before the fix (RED):

  • The failed run ID remained in the memo.
  • The regression assertion failed as expected.

After the minimal fix (GREEN):

  • The failed run ID was removed.
  • Another active run ID remained unaffected.
  • All 24 tests in tests/unit/test_langchain.py passed.
  • Targeted Ruff lint and formatting checks passed.
  • git diff --check passed.

Full unit suite on Windows:

  • 883 passed
  • 2 skipped
  • 3 failed in unrelated tests (two subprocess import failures and one platform-dependent path serialization assertion).

The full suite has not yet been confirmed green on Linux.

Contribution

I have a minimal fix and regression test ready locally.

I'm happy to open a focused PR if the maintainers agree with the proposed approach.

Lingua principale
Python
Stelle
498
Fork
361
Merge medio
16h 40m
PR unite (30g)
38

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di langfuse/langfuse-python

Tutte le issue di langfuse/langfuse-python

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.