bug: LANGFUSE_MAX_EVENT_SIZE_BYTES is parsed but never enforced for score events
Los mantenedores suelen responder en 1 día
Evaluación
Este issue todavía no se ha evaluado.
Descripción
Describe the bug
MAX_EVENT_SIZE_BYTES = int(os.environ.get("LANGFUSE_MAX_EVENT_SIZE_BYTES", 1_000_000)) (score_ingestion_consumer.py#L18) is not referenced anywhere else in the repo (grep -rn MAX_EVENT_SIZE langfuse only finds the definition). The env var was added in #1103 ("make max event and batch size configurable"), but it currently has no effect: an oversized score event is posted as-is instead of being dropped or truncated.
Secondary point: in _next() the running total is compared to MAX_BATCH_SIZE_BYTES only after the event has already been appended (score_ingestion_consumer.py#L83-L108), so a batch can exceed the batch limit by up to one event (and since single events are not capped, by an arbitrary amount).
Expected: events above LANGFUSE_MAX_EVENT_SIZE_BYTES are dropped (with a logged error) or truncated, and the batch limit is checked before appending.
Actual: a 10 MB score comment is sent in a single request. Whether the server then rejects it and how that is surfaced depends on the 4xx handling; today a rejected batch is dropped without a log line (see #1874, which is about that part). Users who set the env var, as in langfuse/langfuse#7027, get no client-side effect from it.
Severity is low to medium: it only affects unusually large score payloads, but the env var suggests protection that is not there.
Steps to reproduce
import httpx
from queue import Queue
from langfuse._utils.request import LangfuseClient
from langfuse._task_manager.score_ingestion_consumer import ScoreIngestionConsumer, MAX_EVENT_SIZE_BYTES, MAX_BATCH_SIZE_BYTES
sizes = []
def h(req): sizes.append(len(req.content)); return httpx.Response(200, json={})
cl = LangfuseClient("pk", "sk", "http://x", "1", 5, httpx.Client(transport=httpx.MockTransport(h)))
q = Queue(); c = ScoreIngestionConsumer(ingestion_queue=q, identifier=0, client=cl, public_key="pk", flush_at=15)
q.put({"id": "big", "type": "score-create", "body": {"comment": "x" * 10_000_000}})
c.upload()
print("MAX_EVENT", MAX_EVENT_SIZE_BYTES, "MAX_BATCH", MAX_BATCH_SIZE_BYTES, "posted bytes", sizes)
Observed output:
MAX_EVENT 1000000 MAX_BATCH 2500000 posted bytes [10000173]
Langfuse Cloud or self-hosted?
Either; the behaviour is client side only.
If self-hosted, what version are you running?
n/a
SDK and integration versions
langfuse 4.17.0 (main @ bf11ec121479145923ff27b32ec2ad1d06fd2d51), Python 3.14.0.
Additional information
Possible direction: after computing item_size in _next(), if item_size > MAX_EVENT_SIZE_BYTES: logger.error(...); self._ingestion_queue.task_done(); continue, and check total_size + item_size > MAX_BATCH_SIZE_BYTES before appending, carrying the event into the next batch. Alternatively remove the constant and the env var if client-side enforcement is not wanted.
Minor and separate: int(os.environ[...]) at import time makes import langfuse raise on a malformed value (this was also noted in the review of #1103).
Related: #1874 (4xx batch drops are not logged), langfuse/langfuse#7027 (413 from the server with LANGFUSE_MAX_EVENT_SIZE_BYTES set).
Are you interested in contributing a fix for this bug?
Yes
- Lenguaje dominante
- Python
- Estrellas
- 498
- Forks
- 361
- Merge medio
- 16 h 40 min
- PR fusionados (30 d)
- 38
Preparar el entorno
- Sin Dockerfile ni archivo de Docker Compose
- Tiene una plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de langfuse/langfuse-python
-
tests/unit/test_prompt.py fails on a fresh clone without LANGFUSE_* env varsPosiblemente ocupada @Mathoholic la tomó hoy. Abiertobug feat-prompt-management sdk-python
Dificultad 2/5 1-3 horas Aptitud para principiantes 80/100
langfuse/langfuse-python#1976 · 3 comentarios ·
Los mantenedores suelen responder en 1 día
-
bug(langchain): failed streaming LLM runs retain IDs in callback memoPosiblemente ocupada @hassiebp la tomó hace 1 día. Abiertobug integration-langchain sdk-python
langfuse/langfuse-python#1977 · 1 asignado ·
Los mantenedores suelen responder en 1 día
-
bug: ChatPromptClient.compile appends str(whole list) once per non-dict placeholder itemPosiblemente ocupada @hassiebp la tomó hace 2 días. Abiertobug feat-prompt-management sdk-python
langfuse/langfuse-python#1971 · 1 asignado ·
Los mantenedores suelen responder en 1 día
-
bug: non-numeric status in a 207 error entry raises inside handle_exception, kills the score consumer thread, flush()/shutdown() then hangPosiblemente ocupada @hassiebp la tomó hace 2 días. Abierto[Integrations] Language Clients bug feat-scores sdk-python
langfuse/langfuse-python#1967 · 1 asignado ·
Los mantenedores suelen responder en 1 día
-
bug: Partial-failure (207) ingestion response retries the whole batch up to 3x; per-item error details only logged at DEBUGPosiblemente ocupada @hassiebp la tomó hace 2 días. Abiertobug feat-ingestion
langfuse/langfuse-python#1965 · 1 asignado ·
Los mantenedores suelen responder en 1 día
Todos los issues de langfuse/langfuse-python
Issues similares
-
Broken links found in docsAbiertodocs pydanty:is-working
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
pydantic/pydantic-ai#10113 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 77/100
GreedyBear-Project/GreedyBear#1727 ·
Los mantenedores suelen responder en 2 días
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 84/100
ARM-software/CMSIS-DSP#354 ·
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
kiat/codeAssist#391 ·
-
needs triage
Dificultad 2/5 1-3 horas Aptitud para principiantes 83/100
darnitdevorg/darnit#587 ·
Los mantenedores suelen responder en 1 día