Span helpers, WaitConfig and batch insights are not reachable from a public import

Open
#663 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
48/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
python

Research direction

Start by reading bedrock_agentcore._utils.polling.WaitConfig and the public runtime exports, then inspect EvaluationClient's span helpers and BatchEvaluationRunner in the evaluation package. Done means callers can import the wait and span utilities publicly, use insights through a runner, and the insights/evaluators constraint is enforced; also review the region argument mismatch between the named clients.

Written by the indexing model from the issue text.

Description

bug

Building an evaluation sample on top of bedrock-agentcore 1.23.0, I ended up reimplementing three things the SDK already has internally, because they are not reachable from a public import. Each one is small on its own; together they are the difference between a sample importing the SDK and a sample duplicating it.

1. WaitConfig has no public import path.

AgentCoreRuntimeClient.create_agent_runtime_and_wait(wait_config=...) is public and documented, but its argument type only exists at bedrock_agentcore._utils.polling.WaitConfig. Neither bedrock_agentcore nor bedrock_agentcore.runtime exports it:

from bedrock_agentcore.runtime import WaitConfig
# ImportError: cannot import name 'WaitConfig' from 'bedrock_agentcore.runtime'

So a caller who needs a timeout other than the default 300 seconds has to import from _utils. I chose to accept the default instead, which means a slow create can now time out where my own loop would have waited longer.

Suggestion: export WaitConfig from bedrock_agentcore.runtime alongside AgentCoreRuntimeClient.

2. Tool-span detection is private, and every span consumer needs it.

EvaluationClient._is_tool_span recognises the three conventions in use:

attrs.get("gen_ai.operation.name") == "execute_tool"
or attrs.get("openinference.span.kind") == "TOOL"
or attrs.get("traceloop.span.kind") == "tool"

Anyone reading spans for their own metrics needs exactly this and cannot call it. I wrote a character-for-character copy, which now silently drifts if the SDK adds a fourth convention. _extract_tool_span_ids and _get_evaluator_level have the same problem.

Suggestion: a public bedrock_agentcore.evaluation.spans module with is_tool_span(span), tool_span_ids(spans, trace_id=None) and evaluator_level(evaluator_id).

3. Insights are not reachable through BatchEvaluationRunner.

StartBatchEvaluation accepts an insights list (Builtin.Insight.FailureAnalysis, UserIntent, ExecutionSummary), and those insights are what make the improvement loop usable: they cluster failures and tag root causes as prompt-addressable. BatchEvaluationRunner covers the evaluator path but grepping the evaluation package for "insight" returns nothing, so the insights path stays raw boto3 plus a hand-written poll loop.

Suggestion: support insights in BatchEvaluationRunner, or add a runner for them, including the constraint that insights and evaluators cannot be requested in the same job.

Minor: the client constructors disagree on the region argument.

ConfigBundleClient(region_name="us-west-2")     # region_name
AgentCoreRuntimeClient(region="us-west-2")      # region

AgentCoreRuntimeClient(region_name=...) raises TypeError. Worth aligning, or accepting both.

Versions: bedrock-agentcore 1.23.0, Python 3.13, us-west-2.

Dominant language
Python
Stars
764
Forks
149
Avg merge
1d 19h
Merged PRs (30d)
7

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from aws/bedrock-agentcore-sdk-python

All issues in aws/bedrock-agentcore-sdk-python

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.