Hacktoberfest 2026: as issues que os mantenedores marcaram para outubro, abertas e boas para iniciantes. Ver issues do Hacktoberfest

bug: get_dataset_run / get_dataset_runs / delete_dataset_run are unusable on Langfuse v4

Aberta
  • Algo errado?
#1,906 0 comentários 0 reações 1 responsável Ver no GitHub

Mantenedores costumam responder em até 1 dia

@hassiebp já está trabalhando nisso.

Desde 27/9/2026.

Avaliação

Esta issue ainda não foi avaliada.

Descrição

bug feat-datasets feat-experiments sdk-python
Describe the bug

The three dataset-run read helpers on the Langfuse client still call the pre-v4
dataset-run endpoints:

Method Defined at Request
Langfuse.get_dataset_run() langfuse/_client/client.py:2532 GET /api/public/datasets/{name}/runs/{run_name}
Langfuse.get_dataset_runs() langfuse/_client/client.py:2557 GET /api/public/datasets/{name}/runs
Langfuse.delete_dataset_run() langfuse/_client/client.py:2588 DELETE /api/public/datasets/{name}/runs/{run_name}

On a Langfuse v4 deployment these paths are rejected, so all three raise
NotFoundError: 404 with the body

This endpoint is not available on deployments running in Langfuse v4 events_only mode.
Learn more about Langfuse v4 at: https://langfuse.com/docs/v4

The same SDK already ships the v4 read client for exactly this data —
client.api.experiments.list() and client.api.experiments.list_items()
(langfuse/api/experiments/raw_client.py:87 and :253) — but no helper uses it.

run_experiment() makes the gap directly visible: it writes its run through
POST /api/public/dataset-run-items, which is accepted on v4, returns a populated
dataset_run_id, and hands the caller a dataset_run_url. That run cannot then be
fetched back with the SDK's own read method.

Steps to reproduce

Against a v4 deployment (LANGFUSE_HOST pointing at it):

import datetime as dt
import time
import uuid

from langfuse import Langfuse

client = Langfuse()
dataset = f"repro-dataset-run-read-{uuid.uuid4().hex[:6]}"

client.create_dataset(name=dataset)
client.create_dataset_item(dataset_name=dataset, input={"q": "hello"}, id=f"{dataset}-item-1")

result = client.run_experiment(
    name="repro-experiment",
    run_name=f"{dataset}-run",
    data=client.api.dataset_items.list(dataset_name=dataset).data,
    task=lambda *, item, **kwargs: "answer",
)
print("run_experiment -> dataset_run_id:", result.dataset_run_id)

try:
    client.get_dataset_run(dataset_name=dataset, run_name=f"{dataset}-run")
    print("get_dataset_run -> OK")
except Exception as exc:
    print("get_dataset_run ->", type(exc).__name__, "status:", getattr(exc, "status_code", None))
    print("  body:", getattr(exc, "body", None))

# The events pipeline is asynchronous, so poll briefly for the run to become readable.
deadline = time.time() + 30
while True:
    experiments = client.api.experiments.list(
        from_start_time=dt.datetime.now(dt.timezone.utc) - dt.timedelta(hours=1),
        name=f"{dataset}-run",
    )
    if experiments.data or time.time() > deadline:
        break
    time.sleep(2)

print("api.experiments.list ->", len(experiments.data), "run(s)")
for experiment in experiments.data:
    print("  ", experiment.id, experiment.name, "items:", experiment.item_count)

client.flush()
Actual behavior
run_experiment -> dataset_run_id: d591fd225d7edec7
get_dataset_run -> NotFoundError status: 404
  body: {'message': 'This endpoint is not available on deployments running in Langfuse v4 events_only mode. Learn more about Langfuse v4 at: https://langfuse.com/docs/v4'}
api.experiments.list -> 1 run(s)
   d591fd225d7edec7 repro-dataset-run-read-6ca29b-run items: 1

The run exists and is readable — just not through the documented helper. Note that
dataset_run_id returned by run_experiment() and the id returned by
api.experiments.list() are the same value, so the data is not lost, only unreachable
from the helper that the SDK exposes for it.

Expected behavior

Either route these helpers through the v4 read path when the deployment is v4
(api.experiments.list / list_items, mapping by id/name and dataset_id), or
state in their docstrings that they are unavailable on v4 and point at
client.api.experiments. Today the docstrings say nothing about v4, so the only signal
a user gets is a 404.

If the first route is taken, the mapping is not a drop-in replacement. Measured against a
v4 deployment for a run written by run_experiment():

DatasetRunItem field v4 source
dataset_run_id / dataset_run_name ExperimentItem.experimentId / .experimentName
dataset_item_id ExperimentItem.experimentItemId
trace_id ExperimentItem.traceId
observation_id ExperimentItem.id — the experiment item id is the task span id
created_at / updated_at ExperimentItem.startTime / .endTime
id no equivalent — the legacy row id (a UUID) is not part of the experiment item

and for DatasetRun: id/name/description/metadata/dataset_id come from
Experiment, but dataset_name has no counterpart (only datasetId is returned, so it
needs a datasets.list() lookup), metadata requires fields=metadata, and
startTime/endTime are clipped to the requested fromStartTime window rather than being
the run's createdAt/updatedAt. from_start_time is required and pagination is
cursor-based, whereas the current helpers are page-based.

Additional context
  • handle_fern_exception in these helpers only logs; the raised NotFoundError keeps the
    generated dataclass str() (it starts with headers: {...}), so the useful server
    message is only visible via exc.body.
  • This is the SDK-side counterpart of langfuse/langfuse#16835, and the two are
    complementary rather than duplicates. #16835 reports that on a non-events_only
    deployment the legacy GET /api/public/datasets/{name}/runs still returns runs created
    via POST /dataset-run-items, while the Experiments UI and GET /api/public/experiments
    do not show them; the open PR langfuse/langfuse#16859 addresses that side. Here the
    deployment is events_only: the legacy read path is rejected outright (404), while
    GET /api/public/experiments returns the run correctly. #16859 touches only
    web/, worker/ and packages/ — no langfuse/ files.
  • The rejection itself is intentional on the server side
    (langfuse/langfuse#15313 "reject dataset-run-item surface in events_only mode",
    #15526 "do not read dataset-runs in events_only mode"), which is why this looks like
    an SDK-alignment issue rather than a server regression.
Linguagem predominante
Python
Estrelas
468
Forks
349
Merge médio
16h 1min
PRs com merge (30d)
20

Preparar o ambiente

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de langfuse/langfuse-python

Todas as issues de langfuse/langfuse-python

Issues semelhantes

Mais issues de Python

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.