🐛 Bug Report: Bedrock instrumentation does not record top_k / stop_sequences, and records no request parameters at all when the call fails

Open
#4,557 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@aniketkrs is already working on this.

Since Oct 5, 2026.

  • #4562 by @aniketkrs — open

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
20/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Stale
Tech stack
aws, python
Domain
observability

Research direction

An open linked pull request (#4562) is already working on this issue; review it before starting to avoid duplicating work. The issue identifies span_utils.py, set_converse_model_span_attributes, and _set_anthropic_messages_span_attributes as relevant entry points. Check the Bedrock instrumentation tests for coverage of Converse and InvokeModel success and failure cases; done means all five request attributes are recorded in each case.

Written by the indexing model from the issue text.

Description

Which component is this bug for?

Bedrock Instrumentation

📜 Description

Two gaps in how opentelemetry-instrumentation-bedrock records request parameters:

1. top_k and stop_sequences are never recorded.

  • converse / converse_stream: set_converse_model_span_attributes (span_utils.py) reads only topP, temperature and maxTokens from inferenceConfig. inferenceConfig.stopSequences is ignored, and so is additionalModelRequestFields (where top_k goes for Anthropic models on Converse).
  • invoke_model / invoke_model_with_response_stream with an Anthropic Messages body: _set_anthropic_messages_span_attributes reads only top_p, temperature and max_tokens; the body's top_k and stop_sequences are ignored.

So gen_ai.request.top_k and gen_ai.request.stop_sequences are never set, although they were sent.

2. A failed call records no request parameters.
All request attributes are set in _handle_call / the converse handlers after the response is received. When the call raises (e.g. ValidationException, ThrottlingException), the wrapper records the exception and re-raises, so the error span has only provider, operation and model: no max_tokens, temperature, top_p etc. Those are exactly the spans where the request parameters are most useful.

👟 Reproduction steps

Self-contained: a botocore before-send hook answers the requests (no AWS account or network needed). Every call sends max_tokens, temperature, top_p, top_k and stop_sequences; each API is called once successfully and once with a 400 ValidationException.

import json, logging, os
logging.disable(logging.WARNING)
os.environ["TRACELOOP_TRACE_CONTENT"] = "true"

import boto3
from botocore.awsrequest import AWSResponse
from botocore.config import Config
from opentelemetry.sdk.trace import SpanProcessor, TracerProvider
from opentelemetry.instrumentation.bedrock import BedrockInstrumentor

MODEL = "anthropic.claude-3-haiku-20240307-v1:0"

class Raw:
    def __init__(self, data): self.data = data
    def stream(self, *a, **k): yield self.data
    def read(self, amt=None): return self.data
    def close(self): pass
    release_conn = close

CONVERSE_OK = {"output": {"message": {"role": "assistant", "content": [{"text": "hello"}]}}, "stopReason": "end_turn",
               "usage": {"inputTokens": 3, "outputTokens": 1, "totalTokens": 4}, "metrics": {"latencyMs": 1}}
INVOKE_OK = {"id": "m", "type": "message", "role": "assistant", "model": "m", "stop_reason": "end_turn",
             "content": [{"type": "text", "text": "hello"}], "usage": {"input_tokens": 3, "output_tokens": 1}}
fail = {"on": False}

def before_send(request, **_):
    if fail["on"]:
        return AWSResponse(request.url, 400, {"x-amzn-ErrorType": "ValidationException:http://internal.amazon.com/",
                                              "content-type": "application/json"}, Raw(b'{"message": "bad request"}'))
    data = json.dumps(CONVERSE_OK if request.url.endswith("/converse") else INVOKE_OK).encode()
    return AWSResponse(request.url, 200, {"content-type": "application/json", "content-length": str(len(data))}, Raw(data))

class Rec(SpanProcessor):
    def __init__(self): self.ended = []
    def on_end(self, span): self.ended.append(span)

rec = Rec()
provider = TracerProvider(); provider.add_span_processor(rec)
BedrockInstrumentor().instrument(tracer_provider=provider)
client = boto3.client("bedrock-runtime", region_name="us-east-1", aws_access_key_id="x",
                      aws_secret_access_key="x", config=Config(retries={"total_max_attempts": 1}))
client.meta.events.register("before-send.bedrock-runtime.*", before_send)

def converse():
    client.converse(modelId=MODEL, messages=[{"role": "user", "content": [{"text": "hi"}]}],
                    inferenceConfig={"maxTokens": 50, "temperature": 0.3, "topP": 0.9, "stopSequences": ["END"]},
                    additionalModelRequestFields={"top_k": 40})

def invoke():
    client.invoke_model(modelId=MODEL, body=json.dumps({
        "anthropic_version": "bedrock-2023-05-31", "max_tokens": 50, "temperature": 0.3, "top_p": 0.9,
        "top_k": 40, "stop_sequences": ["END"],
        "messages": [{"role": "user", "content": [{"type": "text", "text": "hi"}]}]}))["body"].read()

WANTED = ["gen_ai.request.max_tokens", "gen_ai.request.temperature", "gen_ai.request.top_p",
          "gen_ai.request.top_k", "gen_ai.request.stop_sequences"]
for name, fn in [("converse", converse), ("invoke_model", invoke)]:
    for failing in (False, True):
        fail["on"] = failing; rec.ended.clear()
        try:
            fn(); outcome = "succeeded"
        except Exception as e:
            outcome = f"failed ({type(e).__name__})"
        attrs = rec.ended[0].attributes
        print(f"--- {name}, call {outcome}")
        for k in WANTED:
            v = attrs.get(k, "MISSING")
            print(f"  {k:32s} {list(v) if isinstance(v, tuple) else v}")

Output:

boto3 1.43.108 | opentelemetry-instrumentation-bedrock 0.62.4
all calls send: max_tokens=50, temperature=0.3, top_p=0.9, top_k=40, stop_sequences=['END']
--- converse, call succeeded
  gen_ai.request.max_tokens        50
  gen_ai.request.temperature       0.3
  gen_ai.request.top_p             0.9
  gen_ai.request.top_k             MISSING
  gen_ai.request.stop_sequences    MISSING
--- converse, call failed (ValidationException)
  gen_ai.request.max_tokens        MISSING
  gen_ai.request.temperature       MISSING
  gen_ai.request.top_p             MISSING
  gen_ai.request.top_k             MISSING
  gen_ai.request.stop_sequences    MISSING
--- invoke_model, call succeeded
  gen_ai.request.max_tokens        50
  gen_ai.request.temperature       0.3
  gen_ai.request.top_p             0.9
  gen_ai.request.top_k             MISSING
  gen_ai.request.stop_sequences    MISSING
--- invoke_model, call failed (ValidationException)
  gen_ai.request.max_tokens        MISSING
  gen_ai.request.temperature       MISSING
  gen_ai.request.top_p             MISSING
  gen_ai.request.top_k             MISSING
  gen_ai.request.stop_sequences    MISSING
👍 Expected behavior

All five request parameters are recorded (gen_ai.request.max_tokens=50, temperature=0.3, top_p=0.9, top_k=40, stop_sequences=["END"]) for both converse and invoke_model, whether the call succeeds or fails.

👎 Actual Behavior with Screenshots

On success, gen_ai.request.top_k and gen_ai.request.stop_sequences are missing for both APIs. On failure, none of the request parameters are recorded.

🤖 Python Version

3.12

📃 Provide any additional context for the Bug.

opentelemetry-instrumentation-bedrock 0.62.4, boto3 / botocore 1.43.108, Linux.
Possible fix: read stopSequences from inferenceConfig and top_k from additionalModelRequestFields for Converse, and top_k / stop_sequences from the Anthropic body for InvokeModel; and set the request attributes before calling the wrapped method (they depend only on the request), so failed calls keep them. The OpenAI package had the same stop_sequences gap (#4538).

👀 Have you spent some time to check if this bug has been raised before?
  • I checked and didn't find similar issue
Are you willing to submit PR?

Yes I am willing to submit a PR!

Dominant language
Python
Stars
7.5k
Forks
1.2k
Avg merge
2d 6h
Merged PRs (30d)
2

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from traceloop/openllmetry

All issues in traceloop/openllmetry

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.