bug(langchain): failed streaming LLM runs retain IDs in callback memo
Maintainer antworten meist innerhalb von 1 Tag
Bewertung
Dieses Issue wurde noch nicht bewertet.
Beschreibung
Bug description
LangchainCallbackHandler retains completed LLM run IDs in _updated_completion_start_time_memo when a streaming execution terminates with an error.
The successful completion path (on_llm_end) removes the run ID from the set, but the error path (on_llm_error) does not perform the same cleanup.
As a result, long-lived callback handlers reused across failed streaming executions can accumulate stale UUID entries.
Root cause
In langfuse/langchain/CallbackHandler.py:
on_llm_new_token()records the first-token timestamp and adds therun_idto_updated_completion_start_time_memo.on_llm_end()removes therun_idin itsfinallyblock.on_llm_error()ends the observation but does not remove therun_idfrom the memo._detach_observation()and_reset()do not clear this set.
Steps to reproduce
Using a LangchainCallbackHandler with an in-memory Langfuse test client:
- Start an LLM run using
on_llm_start(). - Simulate the first streamed token using
on_llm_new_token(). - Confirm the run ID exists in
_updated_completion_start_time_memo. - Terminate the run using
on_llm_error(RuntimeError("synthetic failure"), run_id=run_id). - Inspect
_updated_completion_start_time_memo.
Actual behavior: The failed run ID remains in the set after the observation has ended.
Expected behavior: The failed run ID should be removed, while unrelated active run IDs remain untouched.
Impact
This causes incremental retention of UUID identifiers when streaming LLM executions fail and the same callback handler is reused.
The observed retention is limited to UUID entries. There is no evidence of retained generations, spans, or a significant process-level memory leak.
Proposed solution
Ensure on_llm_error() discards its own run_id from _updated_completion_start_time_memo in a finally block, mirroring the existing cleanup in on_llm_end().
This should preserve:
- Active nested or concurrent run state.
- Existing OpenTelemetry context handling.
- LangGraph control-flow and resume behavior.
- Current public SDK behavior.
Validation
I reproduced this issue locally using Langfuse Python SDK 4.17.0 and created a deterministic regression test.
Before the fix (RED):
- The failed run ID remained in the memo.
- The regression assertion failed as expected.
After the minimal fix (GREEN):
- The failed run ID was removed.
- Another active run ID remained unaffected.
- All 24 tests in
tests/unit/test_langchain.pypassed. - Targeted Ruff lint and formatting checks passed.
git diff --checkpassed.
Full unit suite on Windows:
- 883 passed
- 2 skipped
- 3 failed in unrelated tests (two subprocess import failures and one platform-dependent path serialization assertion).
The full suite has not yet been confirmed green on Linux.
Contribution
I have a minimal fix and regression test ready locally.
I'm happy to open a focused PR if the maintainers agree with the proposed approach.
- Vorherrschende Sprache
- Python
- Sterne
- 498
- Forks
- 361
- Ø Merge
- 16 Std. 40 Min.
- Gemergte PRs (30 T.)
- 38
Entwicklungsumgebung
- Kein Dockerfile und keine Docker-Compose-Datei
- Hat eine Pull-Request-Vorlage
- Beitragsleitfaden lesen
Erste Schritte
- Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
- Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
- Forken Sie das Repository und arbeiten Sie in einem Branch.
- Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.
Mehr aus langfuse/langfuse-python
-
tests/unit/test_prompt.py fails on a fresh clone without LANGFUSE_* env varsEvtl. vergeben @Mathoholic hat das vor 1 Tag übernommen. Offenbug feat-prompt-management sdk-python
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 80/100
langfuse/langfuse-python#1976 · 3 Kommentare ·
Maintainer antworten meist innerhalb von 1 Tag
-
bug: ChatPromptClient.compile appends str(whole list) once per non-dict placeholder itemEvtl. vergeben @hassiebp hat das vor 3 Tagen übernommen. Offenbug feat-prompt-management sdk-python
langfuse/langfuse-python#1971 · 1 zugewiesene Person ·
Maintainer antworten meist innerhalb von 1 Tag
-
bug: LANGFUSE_MAX_EVENT_SIZE_BYTES is parsed but never enforced for score eventsEvtl. vergeben @hassiebp hat das vor 3 Tagen übernommen. Offenbug feat-scores sdk-python
langfuse/langfuse-python#1966 · 1 zugewiesene Person ·
Maintainer antworten meist innerhalb von 1 Tag
-
bug: non-numeric status in a 207 error entry raises inside handle_exception, kills the score consumer thread, flush()/shutdown() then hangEvtl. vergeben @hassiebp hat das vor 3 Tagen übernommen. Offen[Integrations] Language Clients bug feat-scores sdk-python
langfuse/langfuse-python#1967 · 1 zugewiesene Person ·
Maintainer antworten meist innerhalb von 1 Tag
-
bug: Partial-failure (207) ingestion response retries the whole batch up to 3x; per-item error details only logged at DEBUGEvtl. vergeben @hassiebp hat das vor 3 Tagen übernommen. Offenbug feat-ingestion
langfuse/langfuse-python#1965 · 1 zugewiesene Person ·
Maintainer antworten meist innerhalb von 1 Tag
Alle Issues in langfuse/langfuse-python
Ähnliche Issues
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 76/100
spec-kitty/spec-kitty#6112 ·
Maintainer antworten meist innerhalb von 1 Tag
-
[Bug]: OpenRouter streaming chunks report the requested router name instead of the model OpenRouter servedEvtl. vergeben @Mrun25 hat das heute übernommen. Offenbug llm translation
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 75/100
BerriAI/litellm#46095 · 1 Kommentar ·
Maintainer antworten meist innerhalb von 1 Tag
-
bug
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 72/100
uberdeveloper/fastbt#29 ·
-
bug
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 78/100
andrew-blake/melcloudhome#368 ·
Maintainer antworten meist innerhalb von 1 Tag
-
lane:discover
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 72/100
SPARDiffData/differential-data-collection#33 ·
Maintainer antworten meist innerhalb von 1 Tag