Hacktoberfest 2026: die Issues, die Maintainer für den Oktober markiert haben – offen und einsteigerfreundlich. Hacktoberfest-Issues durchsuchen

bug: get_dataset_run / get_dataset_runs / delete_dataset_run are unusable on Langfuse v4

Offen
  • Stimmt etwas nicht?
#1,906 0 Kommentare 0 Reaktionen 1 zugewiesene Person Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

@hassiebp arbeitet bereits daran.

Seit 27.9.2026.

Bewertung

Dieses Issue wurde noch nicht bewertet.

Beschreibung

bug feat-datasets feat-experiments sdk-python
Describe the bug

The three dataset-run read helpers on the Langfuse client still call the pre-v4
dataset-run endpoints:

Method Defined at Request
Langfuse.get_dataset_run() langfuse/_client/client.py:2532 GET /api/public/datasets/{name}/runs/{run_name}
Langfuse.get_dataset_runs() langfuse/_client/client.py:2557 GET /api/public/datasets/{name}/runs
Langfuse.delete_dataset_run() langfuse/_client/client.py:2588 DELETE /api/public/datasets/{name}/runs/{run_name}

On a Langfuse v4 deployment these paths are rejected, so all three raise
NotFoundError: 404 with the body

This endpoint is not available on deployments running in Langfuse v4 events_only mode.
Learn more about Langfuse v4 at: https://langfuse.com/docs/v4

The same SDK already ships the v4 read client for exactly this data —
client.api.experiments.list() and client.api.experiments.list_items()
(langfuse/api/experiments/raw_client.py:87 and :253) — but no helper uses it.

run_experiment() makes the gap directly visible: it writes its run through
POST /api/public/dataset-run-items, which is accepted on v4, returns a populated
dataset_run_id, and hands the caller a dataset_run_url. That run cannot then be
fetched back with the SDK's own read method.

Steps to reproduce

Against a v4 deployment (LANGFUSE_HOST pointing at it):

import datetime as dt
import time
import uuid

from langfuse import Langfuse

client = Langfuse()
dataset = f"repro-dataset-run-read-{uuid.uuid4().hex[:6]}"

client.create_dataset(name=dataset)
client.create_dataset_item(dataset_name=dataset, input={"q": "hello"}, id=f"{dataset}-item-1")

result = client.run_experiment(
    name="repro-experiment",
    run_name=f"{dataset}-run",
    data=client.api.dataset_items.list(dataset_name=dataset).data,
    task=lambda *, item, **kwargs: "answer",
)
print("run_experiment -> dataset_run_id:", result.dataset_run_id)

try:
    client.get_dataset_run(dataset_name=dataset, run_name=f"{dataset}-run")
    print("get_dataset_run -> OK")
except Exception as exc:
    print("get_dataset_run ->", type(exc).__name__, "status:", getattr(exc, "status_code", None))
    print("  body:", getattr(exc, "body", None))

# The events pipeline is asynchronous, so poll briefly for the run to become readable.
deadline = time.time() + 30
while True:
    experiments = client.api.experiments.list(
        from_start_time=dt.datetime.now(dt.timezone.utc) - dt.timedelta(hours=1),
        name=f"{dataset}-run",
    )
    if experiments.data or time.time() > deadline:
        break
    time.sleep(2)

print("api.experiments.list ->", len(experiments.data), "run(s)")
for experiment in experiments.data:
    print("  ", experiment.id, experiment.name, "items:", experiment.item_count)

client.flush()
Actual behavior
run_experiment -> dataset_run_id: d591fd225d7edec7
get_dataset_run -> NotFoundError status: 404
  body: {'message': 'This endpoint is not available on deployments running in Langfuse v4 events_only mode. Learn more about Langfuse v4 at: https://langfuse.com/docs/v4'}
api.experiments.list -> 1 run(s)
   d591fd225d7edec7 repro-dataset-run-read-6ca29b-run items: 1

The run exists and is readable — just not through the documented helper. Note that
dataset_run_id returned by run_experiment() and the id returned by
api.experiments.list() are the same value, so the data is not lost, only unreachable
from the helper that the SDK exposes for it.

Expected behavior

Either route these helpers through the v4 read path when the deployment is v4
(api.experiments.list / list_items, mapping by id/name and dataset_id), or
state in their docstrings that they are unavailable on v4 and point at
client.api.experiments. Today the docstrings say nothing about v4, so the only signal
a user gets is a 404.

If the first route is taken, the mapping is not a drop-in replacement. Measured against a
v4 deployment for a run written by run_experiment():

DatasetRunItem field v4 source
dataset_run_id / dataset_run_name ExperimentItem.experimentId / .experimentName
dataset_item_id ExperimentItem.experimentItemId
trace_id ExperimentItem.traceId
observation_id ExperimentItem.id — the experiment item id is the task span id
created_at / updated_at ExperimentItem.startTime / .endTime
id no equivalent — the legacy row id (a UUID) is not part of the experiment item

and for DatasetRun: id/name/description/metadata/dataset_id come from
Experiment, but dataset_name has no counterpart (only datasetId is returned, so it
needs a datasets.list() lookup), metadata requires fields=metadata, and
startTime/endTime are clipped to the requested fromStartTime window rather than being
the run's createdAt/updatedAt. from_start_time is required and pagination is
cursor-based, whereas the current helpers are page-based.

Additional context
  • handle_fern_exception in these helpers only logs; the raised NotFoundError keeps the
    generated dataclass str() (it starts with headers: {...}), so the useful server
    message is only visible via exc.body.
  • This is the SDK-side counterpart of langfuse/langfuse#16835, and the two are
    complementary rather than duplicates. #16835 reports that on a non-events_only
    deployment the legacy GET /api/public/datasets/{name}/runs still returns runs created
    via POST /dataset-run-items, while the Experiments UI and GET /api/public/experiments
    do not show them; the open PR langfuse/langfuse#16859 addresses that side. Here the
    deployment is events_only: the legacy read path is rejected outright (404), while
    GET /api/public/experiments returns the run correctly. #16859 touches only
    web/, worker/ and packages/ — no langfuse/ files.
  • The rejection itself is intentional on the server side
    (langfuse/langfuse#15313 "reject dataset-run-item surface in events_only mode",
    #15526 "do not read dataset-runs in events_only mode"), which is why this looks like
    an SDK-alignment issue rather than a server regression.
Vorherrschende Sprache
Python
Sterne
468
Forks
349
Ø Merge
16 Std. 1 Min.
Gemergte PRs (30 T.)
20

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus langfuse/langfuse-python

Alle Issues in langfuse/langfuse-python

Ähnliche Issues

Weitere Issues zu Python

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.