Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

[bot] AWS Bedrock InvokeModel for embeddings not instrumented

Aperta
#134 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
68/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Tranquilla
Stack tecnologico
aws, java

Direzione di ricerca

Inizia da BraintrustBedrockInterceptor.java e BraintrustAWSBedrock.java per tracciare come vengono selezionate le operazioni Converse, quindi leggi InstrumentationSemConv.java per esaminare il tagging esistente delle richieste e delle risposte Bedrock. Esamina BraintrustAWSBedrockTest.java e aggiungi la copertura per InvokeModelRequest usando payload strutturati come quelli di Titan e Cohere. Il lavoro è completato quando le chiamate di embedding creano span con metadati del modello/provider, l’input acquisito, il tipo di embedding e le metriche dei token specificate.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Summary

The AWS Bedrock instrumentation in BraintrustBedrockInterceptor explicitly skips all operations other than Converse and ConverseStream. This means embedding model calls — which use the InvokeModel API, not Converse — produce no spans, no input capture, and no token metrics.

All embedding models on AWS Bedrock require InvokeModel:

  • Amazon Titan Text Embeddings v2 (amazon.titan-embed-text-v2:0)
  • Amazon Titan Text Embeddings v1 (amazon.titan-embed-text-v1)
  • Amazon Titan Multimodal Embeddings (amazon.titan-embed-image-v1)
  • Cohere Embed English v3 (cohere.embed-english-v3)
  • Cohere Embed Multilingual v3 (cohere.embed-multilingual-v3)

None of these models support the Converse API — InvokeModel is the only invocation path for Bedrock embeddings.

What is missing

BraintrustBedrockInterceptor.beforeExecution() (lines 60–71) explicitly limits instrumentation to two operations:

private static final Set<String> INSTRUMENTED_OPERATIONS = Set.of("Converse", "ConverseStream");

// Only instrument Converse and ConverseStream — other Bedrock operations
// (InvokeModel, ApplyGuardrail, etc.) are not LLM calls we know how to tag.
if (!INSTRUMENTED_OPERATIONS.contains(operationName)) {
    return;
}

When a user calls client.invokeModel(...) with an embedding model (e.g. amazon.titan-embed-text-v2:0), the interceptor returns immediately without creating a span. No span is created, no input text is captured, and no embedding output or token metrics are recorded.

The BraintrustAWSBedrock.wrap() Javadoc on the sync builder explicitly states it traces "every converse call" and the async builder traces "every converseStream call" — embedding calls via InvokeModel are not in scope at all today.

A minimal InvokeModel embedding span should capture:

  • braintrust.metadata: model ID (from the request URI path /model/{modelId}/invoke), provider
  • braintrust.input_json: the inputText (or equivalent input field) from the request body
  • braintrust.metrics: prompt_tokens from the response inputTextTokenCount (Titan) or meta.inputTokenCount (Cohere)
  • braintrust.span_attributes: {"type": "embedding"}

Braintrust docs status

  • AWS Bedrock listed as a supported cloud provider at https://www.braintrust.dev/docs/integrations/ai-providers: supported (for generative use)
  • No mention of Bedrock embeddings or InvokeModel instrumentation in any Braintrust docs page: not_found
  • For comparison, OpenAI and Google GenAI embedding instrumentation exists in this repo (though with detail gaps tracked in #66 and #65)

Upstream sources

Local files inspected

  • braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustBedrockInterceptor.java — line 60: Set.of("Converse", "ConverseStream"); lines 63–71: beforeExecution returns without span for any other operation name including InvokeModel
  • braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustAWSBedrock.java — Javadoc on both wrap() overloads confirms only Converse/ConverseStream are traced; no invokeModel wrapping exists
  • braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/test/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustAWSBedrockTest.java — all tests use ConverseRequest and ConverseStreamRequest; no InvokeModelRequest test exists
  • braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.javatagBedrockRequest() and tagBedrockResponse() only handle Converse-shaped JSON (messages, output.message, usage.inputTokens); InvokeModel request/response bodies have entirely different schemas
Lingua principale
Java
Stelle
21
Fork
5
Merge medio
2g 7h
PR unite (30g)
8

Guida per i contributori

Apri la guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di braintrustdata/braintrust-sdk-java

Tutte le issue di braintrustdata/braintrust-sdk-java

Issue simili

Altre issue su Java

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.