bug: non-numeric status in a 207 error entry raises inside handle_exception, kills the score consumer thread, flush()/shutdown() then hang
Maintainer antworten meist innerhalb von 1 Tag
Bewertung
Dieses Issue wurde noch nicht bewertet.
Beschreibung
Describe the bug
ScoreIngestionConsumer.upload() calls handle_exception(e) from inside its except block (score_ingestion_consumer.py#L141-L153). generate_error_message does int(error.status) on string statuses with no guard (parse_error.py#L75-L99), so a non-numeric status such as "n/a" raises ValueError from inside the error handler.
Nothing catches it in upload() or in run() (L129-L139), so the daemon consumer thread exits, and nothing restarts it (only the fork re-init does). The batch that triggered the error is still acknowledged by the finally block, but every later score event stays in the queue, unfinished_tasks never reaches 0, and Langfuse.flush() / shutdown() block forever on Queue.join() (resource_manager.py#L629-L640, no timeout).
Likelihood: the trigger needs a 207 response whose errors[].status is non-numeric. The Langfuse server does not send that today, so this is a defensive-hardening bug (a misbehaving proxy/gateway or a future server change would trigger it). The consequence is silent loss of later scores plus an unbounded hang at exit, and the fix is small.
Steps to reproduce
Uses httpx.MockTransport, no server needed:
import httpx, threading
from queue import Queue
from langfuse._utils.request import LangfuseClient
from langfuse._task_manager.score_ingestion_consumer import ScoreIngestionConsumer
import time; time.sleep = lambda s: None
def h(req): return httpx.Response(207, json={"errors": [{"status": "n/a", "message": "m"}]})
cl = LangfuseClient("pk", "sk", "http://x", "1", 5, httpx.Client(transport=httpx.MockTransport(h)))
q = Queue(); c = ScoreIngestionConsumer(ingestion_queue=q, identifier=0, client=cl, public_key="pk", flush_at=1, flush_interval=0.2)
c.start(); q.put({"id": "a", "type": "t", "body": {}}); c.join(3)
print("consumer alive:", c.is_alive())
q.put({"id": "b", "type": "t", "body": {}})
t = threading.Thread(target=q.join, daemon=True); t.start(); t.join(3); print("flush hung:", t.is_alive())
Observed output (traceback goes to stderr):
File ".../langfuse/_utils/parse_error.py", line 86, in generate_error_message
int(error.status) if isinstance(error.status, str) else error.status,
ValueError: invalid literal for int() with base 10: 'n/a'
consumer alive: False
flush hung: True
Direct check: handle_exception(APIErrors([APIError("abc", "m")])) raises ValueError: invalid literal for int() with base 10: 'abc' (a None status is fine).
Langfuse Cloud or self-hosted?
Not server dependent (reproduced with a mock transport).
If self-hosted, what version are you running?
n/a
SDK and integration versions
langfuse 4.17.0 (main @ bf11ec121479145923ff27b32ec2ad1d06fd2d51), Python 3.14.0.
Additional information
Possible direction: convert statuses with a safe helper (try: int(s) except (TypeError, ValueError): None) in parse_error.py; make handle_exception never raise (fall back to logger.error); and wrap the loop body of ScoreIngestionConsumer.run() in try/except Exception that logs and continues so a single bad batch cannot end the thread. A bounded join() in flush() would be related hardening.
Related, different root cause: #1799 (shutdown() hangs on _score_ingestion_queue.join() after a cached resource manager is reused). Both end with consumers that are gone and an unbounded join().
Are you interested in contributing a fix for this bug?
Yes
- Vorherrschende Sprache
- Python
- Sterne
- 498
- Forks
- 361
- Ø Merge
- 16 Std. 42 Min.
- Gemergte PRs (30 T.)
- 51
Entwicklungsumgebung
- Kein Dockerfile und keine Docker-Compose-Datei
- Hat eine Pull-Request-Vorlage
- Beitragsleitfaden lesen
Erste Schritte
- Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
- Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
- Forken Sie das Repository und arbeiten Sie in einem Branch.
- Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.
Mehr aus langfuse/langfuse-python
-
tests/unit/test_prompt.py fails on a fresh clone without LANGFUSE_* env varsEvtl. vergeben @Mathoholic hat das vor 2 Tagen übernommen. Offenbug feat-prompt-management sdk-python
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 80/100
langfuse/langfuse-python#1976 · 3 Kommentare ·
Maintainer antworten meist innerhalb von 1 Tag
-
bug(langchain): failed streaming LLM runs retain IDs in callback memoEvtl. vergeben @hassiebp hat das vor 3 Tagen übernommen. Offenbug integration-langchain sdk-python
langfuse/langfuse-python#1977 · 1 zugewiesene Person ·
Maintainer antworten meist innerhalb von 1 Tag
-
bug: ChatPromptClient.compile appends str(whole list) once per non-dict placeholder itemEvtl. vergeben @hassiebp hat das vor 4 Tagen übernommen. Offenbug feat-prompt-management sdk-python
langfuse/langfuse-python#1971 · 1 zugewiesene Person ·
Maintainer antworten meist innerhalb von 1 Tag
-
bug: LANGFUSE_MAX_EVENT_SIZE_BYTES is parsed but never enforced for score eventsEvtl. vergeben @hassiebp hat das vor 4 Tagen übernommen. Offenbug feat-scores sdk-python
langfuse/langfuse-python#1966 · 1 zugewiesene Person ·
Maintainer antworten meist innerhalb von 1 Tag
-
bug: Partial-failure (207) ingestion response retries the whole batch up to 3x; per-item error details only logged at DEBUGEvtl. vergeben @hassiebp hat das vor 4 Tagen übernommen. Offenbug feat-ingestion
langfuse/langfuse-python#1965 · 1 zugewiesene Person ·
Maintainer antworten meist innerhalb von 1 Tag
Alle Issues in langfuse/langfuse-python
Ähnliche Issues
-
Broken links in the docsOffen
Schwierigkeit 1/5 Unter einer Stunde Anfängerfreundlichkeit 78/100
uniswap-python/uniswap-python#487 ·
-
[Bug]: agent_final_response still returns the abandoned branch's answer after navigate moves HEAD backEvtl. vergeben @enyst hat das heute übernommen. Offenbug priority:low ready-for-dev
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 72/100
OpenHands/software-agent-sdk#5822 ·
Maintainer antworten meist innerhalb von 1 Tag
-
Schwierigkeit 1/5 Unter einer Stunde Anfängerfreundlichkeit 72/100
-
area: server enhancement good first issue
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 82/100
scs0209/laya-guard#52 ·
Maintainer antworten meist innerhalb von 1 Tag
-
Schwierigkeit 2/5 1-3 Stunden Anfängerfreundlichkeit 82/100
Maintainer antworten meist innerhalb von 2 Tagen