🐛 Bug Report: Bedrock instrumentation does not record top_k / stop_sequences, and records no request parameters at all when the call fails
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 20/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Stale
- Tech stack
- aws, python
- Domain
- observability
Research direction
An open linked pull request (#4562) is already working on this issue; review it before starting to avoid duplicating work. The issue identifies span_utils.py, set_converse_model_span_attributes, and _set_anthropic_messages_span_attributes as relevant entry points. Check the Bedrock instrumentation tests for coverage of Converse and InvokeModel success and failure cases; done means all five request attributes are recorded in each case.
Written by the indexing model from the issue text.
Description
Which component is this bug for?
Bedrock Instrumentation
📜 Description
Two gaps in how opentelemetry-instrumentation-bedrock records request parameters:
1. top_k and stop_sequences are never recorded.
converse/converse_stream:set_converse_model_span_attributes(span_utils.py) reads onlytopP,temperatureandmaxTokensfrominferenceConfig.inferenceConfig.stopSequencesis ignored, and so isadditionalModelRequestFields(wheretop_kgoes for Anthropic models on Converse).invoke_model/invoke_model_with_response_streamwith an Anthropic Messages body:_set_anthropic_messages_span_attributesreads onlytop_p,temperatureandmax_tokens; the body'stop_kandstop_sequencesare ignored.
So gen_ai.request.top_k and gen_ai.request.stop_sequences are never set, although they were sent.
2. A failed call records no request parameters.
All request attributes are set in _handle_call / the converse handlers after the response is received. When the call raises (e.g. ValidationException, ThrottlingException), the wrapper records the exception and re-raises, so the error span has only provider, operation and model: no max_tokens, temperature, top_p etc. Those are exactly the spans where the request parameters are most useful.
👟 Reproduction steps
Self-contained: a botocore before-send hook answers the requests (no AWS account or network needed). Every call sends max_tokens, temperature, top_p, top_k and stop_sequences; each API is called once successfully and once with a 400 ValidationException.
import json, logging, os
logging.disable(logging.WARNING)
os.environ["TRACELOOP_TRACE_CONTENT"] = "true"
import boto3
from botocore.awsrequest import AWSResponse
from botocore.config import Config
from opentelemetry.sdk.trace import SpanProcessor, TracerProvider
from opentelemetry.instrumentation.bedrock import BedrockInstrumentor
MODEL = "anthropic.claude-3-haiku-20240307-v1:0"
class Raw:
def __init__(self, data): self.data = data
def stream(self, *a, **k): yield self.data
def read(self, amt=None): return self.data
def close(self): pass
release_conn = close
CONVERSE_OK = {"output": {"message": {"role": "assistant", "content": [{"text": "hello"}]}}, "stopReason": "end_turn",
"usage": {"inputTokens": 3, "outputTokens": 1, "totalTokens": 4}, "metrics": {"latencyMs": 1}}
INVOKE_OK = {"id": "m", "type": "message", "role": "assistant", "model": "m", "stop_reason": "end_turn",
"content": [{"type": "text", "text": "hello"}], "usage": {"input_tokens": 3, "output_tokens": 1}}
fail = {"on": False}
def before_send(request, **_):
if fail["on"]:
return AWSResponse(request.url, 400, {"x-amzn-ErrorType": "ValidationException:http://internal.amazon.com/",
"content-type": "application/json"}, Raw(b'{"message": "bad request"}'))
data = json.dumps(CONVERSE_OK if request.url.endswith("/converse") else INVOKE_OK).encode()
return AWSResponse(request.url, 200, {"content-type": "application/json", "content-length": str(len(data))}, Raw(data))
class Rec(SpanProcessor):
def __init__(self): self.ended = []
def on_end(self, span): self.ended.append(span)
rec = Rec()
provider = TracerProvider(); provider.add_span_processor(rec)
BedrockInstrumentor().instrument(tracer_provider=provider)
client = boto3.client("bedrock-runtime", region_name="us-east-1", aws_access_key_id="x",
aws_secret_access_key="x", config=Config(retries={"total_max_attempts": 1}))
client.meta.events.register("before-send.bedrock-runtime.*", before_send)
def converse():
client.converse(modelId=MODEL, messages=[{"role": "user", "content": [{"text": "hi"}]}],
inferenceConfig={"maxTokens": 50, "temperature": 0.3, "topP": 0.9, "stopSequences": ["END"]},
additionalModelRequestFields={"top_k": 40})
def invoke():
client.invoke_model(modelId=MODEL, body=json.dumps({
"anthropic_version": "bedrock-2023-05-31", "max_tokens": 50, "temperature": 0.3, "top_p": 0.9,
"top_k": 40, "stop_sequences": ["END"],
"messages": [{"role": "user", "content": [{"type": "text", "text": "hi"}]}]}))["body"].read()
WANTED = ["gen_ai.request.max_tokens", "gen_ai.request.temperature", "gen_ai.request.top_p",
"gen_ai.request.top_k", "gen_ai.request.stop_sequences"]
for name, fn in [("converse", converse), ("invoke_model", invoke)]:
for failing in (False, True):
fail["on"] = failing; rec.ended.clear()
try:
fn(); outcome = "succeeded"
except Exception as e:
outcome = f"failed ({type(e).__name__})"
attrs = rec.ended[0].attributes
print(f"--- {name}, call {outcome}")
for k in WANTED:
v = attrs.get(k, "MISSING")
print(f" {k:32s} {list(v) if isinstance(v, tuple) else v}")
Output:
boto3 1.43.108 | opentelemetry-instrumentation-bedrock 0.62.4
all calls send: max_tokens=50, temperature=0.3, top_p=0.9, top_k=40, stop_sequences=['END']
--- converse, call succeeded
gen_ai.request.max_tokens 50
gen_ai.request.temperature 0.3
gen_ai.request.top_p 0.9
gen_ai.request.top_k MISSING
gen_ai.request.stop_sequences MISSING
--- converse, call failed (ValidationException)
gen_ai.request.max_tokens MISSING
gen_ai.request.temperature MISSING
gen_ai.request.top_p MISSING
gen_ai.request.top_k MISSING
gen_ai.request.stop_sequences MISSING
--- invoke_model, call succeeded
gen_ai.request.max_tokens 50
gen_ai.request.temperature 0.3
gen_ai.request.top_p 0.9
gen_ai.request.top_k MISSING
gen_ai.request.stop_sequences MISSING
--- invoke_model, call failed (ValidationException)
gen_ai.request.max_tokens MISSING
gen_ai.request.temperature MISSING
gen_ai.request.top_p MISSING
gen_ai.request.top_k MISSING
gen_ai.request.stop_sequences MISSING
👍 Expected behavior
All five request parameters are recorded (gen_ai.request.max_tokens=50, temperature=0.3, top_p=0.9, top_k=40, stop_sequences=["END"]) for both converse and invoke_model, whether the call succeeds or fails.
👎 Actual Behavior with Screenshots
On success, gen_ai.request.top_k and gen_ai.request.stop_sequences are missing for both APIs. On failure, none of the request parameters are recorded.
🤖 Python Version
3.12
📃 Provide any additional context for the Bug.
opentelemetry-instrumentation-bedrock 0.62.4, boto3 / botocore 1.43.108, Linux.
Possible fix: read stopSequences from inferenceConfig and top_k from additionalModelRequestFields for Converse, and top_k / stop_sequences from the Anthropic body for InvokeModel; and set the request attributes before calling the wrapped method (they depend only on the request), so failed calls keep them. The OpenAI package had the same stop_sequences gap (#4538).
👀 Have you spent some time to check if this bug has been raised before?
- I checked and didn't find similar issue
Are you willing to submit PR?
Yes I am willing to submit a PR!
- Dominant language
- Python
- Stars
- 7.5k
- Forks
- 1.2k
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 2
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from traceloop/openllmetry
-
🐛 Bug Report: OpenAI Responses: with_raw_response.create() raises when the body can't be parsed, though the uninstrumented call returnsPossibly taken A pull request linked to this issue is open or already merged. Openbug python
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
traceloop/openllmetry#4717 ·
Maintainers usually reply within 1 day
-
🐛 Bug Report: Incomplete cache token migration after #4243 breaks Bedrock and Groq instrumentationPossibly taken @Quantum-Enigma-dot claimed this 21 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
traceloop/openllmetry#4665 · 1 comment ·
Maintainers usually reply within 1 day
-
🐛 Bug Report: OpenAI instrumentation drops gen_ai.usage.reasoning_tokens on streamed chat completionsPossibly taken @jayfeng7 claimed this 23 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
traceloop/openllmetry#4660 · 1 comment ·
Maintainers usually reply within 1 day
-
Bug Report: Failed dataset deletes still remove local rows and columnsPossibly taken @inchang-ing claimed this 32 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
traceloop/openllmetry#4630 ·
Maintainers usually reply within 1 day
-
🐛 Bug Report: LiteLLM instrumentation does not record seed, stop, frequency_penalty, presence_penaltyPossibly taken @shantam77 claimed this 38 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
traceloop/openllmetry#4598 · 2 comments ·
Maintainers usually reply within 1 day
All issues in traceloop/openllmetry
Similar issues
-
opensubtitlescom: provider rejects pt-PT even though its own converter maps itPossibly taken @paulo-0308 claimed this today. Open
Difficulty 1/5 Under an hour Newbie friendliness 88/100
Diaoul/subliminal#1405 ·
-
Internal ErrorOpenbug
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
frappe/frappe_docker#2071 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
The-Pocket/PocketFlow#161 ·
-
comp/desktop P2 type/bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
NousResearch/hermes-agent#157702 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 83/100
zai-org/zcode-plugins#202 ·
Maintainers usually reply within 1 day