[Feature] OpenTelemetry GenAI exporter for Agent Traces (Phase 1: Java, spans-only)

Open
#970 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Quiet
Tech stack
java

Research direction

Start by reading the serialized JSON Event Log contract in #924 and the existing trace_tree.py diagnostic shape. Define the standalone Java converter around the issue's span, ID, grouping, and incomplete-execution rules, then verify that repeated exports are deterministic and malformed records produce the specified diagnostics.

Written by the indexing model from the issue text.

Description

feature priority/major
Search before asking
Description

Tracking issue for Phase 1 of the OTel exporter discussed in #929, as a follow-up to #900 and the Execution Event recording work in #924.

Scope (Phase 1):

  • A batch, offline converter over already-collected Event Log files (event-log.trace.enabled: true). It maps Execution Events to OpenTelemetry GenAI spans: one inputRunId becomes one trace with a synthesized invoke_agent root, and the recorded execution hierarchy defines the rest of the span tree. Records are grouped by inputRunId, sorted by timestamp, and end-of-input marks completion.
  • Span mapping follows #929: llm uses CLIENT; tool, action, parser, and the run root use INTERNAL. parser maps to parse {name} with gen_ai.operation.name: "parse".
  • Incomplete executions are still exported rather than dropped. A start without a terminal becomes a zero-duration span with status UNSET and flink_agents.execution.incomplete = true, plus an INCOMPLETE_EXECUTION diagnostic. The inverse case uses MISSING_START.
  • Diagnostics follow the existing trace_tree.py shape (code, id, message, file location), including MALFORMED_RECORD for invalid input.
  • The input contract is the serialized JSON Event Log record, not a particular file name. Explicit files are read as-is, unknown fields are ignored, and directory inputs may discover events-*.log files for convenience. For multi-subtask jobs, completeness of the supplied file set remains the caller's responsibility.
  • Framework IDs are deterministically mapped to fixed-width OTel trace/span IDs so repeated exports produce the same IDs. Delivery remains at-least-once.
  • Configuration is provided through converter arguments such as endpoint, protocol, and service.name. Content capture is excluded from Phase 1.
  • Delivered as an optional standalone module, not bundled into dist, with OTel dependencies aligned through the opentelemetry-bom.
  • Because the serialized Event Log format is shared across runtimes, the same converter works for both Java and Python agents.

Out of scope, tracked separately:

  • A durable/replayable Event Log sink and a continuous OTel consumer with checkpoint/replay semantics.
  • Content capture and richer usage metadata, pending the entityMetadata work discussed in #929 / #924.
  • Optional OpenInference classification (openinference.span.kind) for backends such as Phoenix/Langfuse.

Rejected alternative (Flink's native OpenTelemetryTraceReporterFactory): see the note in #929.

Depends on #924 (recording contract).

Are you willing to submit a PR?
  • I'm willing to submit a PR!
Dominant language
Java
Stars
454
Forks
167
Avg merge
5d 1h
Merged PRs (30d)
46

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/flink-agents

All issues in apache/flink-agents

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.