[bot] AWS Bedrock batch inference (CreateModelInvocationJob) is not instrumented at all

Aperta
#178 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
35/100
Tipo di issue
Funzionalità
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
aws, java

Direzione di ricerca

Inizia da BraintrustAWSBedrock.java, AWSBedrockInstrumentationModule.java, BraintrustBedrockInterceptor.java e InstrumentationSemConv.java, quindi esamina BraintrustAWSBedrockTest.java e le API client del control plane di AWS Bedrock. Aggiungi la copertura per le operazioni di batch job indicate, comprese le relative strutture di richiesta e risposta e il relativo ciclo di vita, e verificala con test mirati o cassette per l’invio, lo stato, l’elenco e l’arresto.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Summary

The aws_bedrock_2_30_0 module only ever wraps the Bedrock data-plane runtime client builders (software.amazon.awssdk.services.bedrockruntime.BedrockRuntimeClientBuilder / BedrockRuntimeAsyncClientBuilder), and only tags Converse/ConverseStream operations on that client (tracked separately in #134 for InvokeModel). AWS Bedrock's official batch inference API — submitted through the entirely separate control-plane client software.amazon.awssdk.services.bedrock.BedrockClient (CreateModelInvocationJob / GetModelInvocationJob / ListModelInvocationJobs / StopModelInvocationJob) — has no wrapping mechanism at all. This is a different Maven artifact/class hierarchy (bedrock vs. bedrockruntime) that this module never references, so unlike the Gemini Batches gap (#165, where calls at least reach the wrapper and get mis-tagged), a Bedrock batch inference job call today produces zero span — not even a generic or mis-tagged one.

What is missing

  • BraintrustBedrockInterceptor.java line 55: private static final Set<String> INSTRUMENTED_OPERATIONS = Set.of("Converse", "ConverseStream");, enforced at lines 62–66 (beforeExecution returns immediately for any other operation name). This interceptor is only ever attached to a BedrockRuntimeClientBuilder/BedrockRuntimeAsyncClientBuilder in the first place (see next point), so it structurally cannot see batch-inference-job calls regardless of this allowlist.
  • BraintrustAWSBedrock.java (lines 23–24, 53–54): both public wrap() overloads are typed to accept only BedrockRuntimeClientBuilder / BedrockRuntimeAsyncClientBuilder. There is no wrap(BedrockClientBuilder) overload, so a manual user has no supported way to instrument the control-plane client that hosts batch inference at all.
  • AWSBedrockInstrumentationModule.java (BedrockBuilderAdvice.build(), lines 74–83): the auto-instrumentation instanceof checks are likewise limited to BedrockRuntimeClientBuilder/BedrockRuntimeAsyncClientBuilder. A BedrockClientBuilder (control plane) passing through SdkDefaultClientBuilder.build() falls through both branches untouched.
  • InstrumentationSemConv.tagBedrockRequest()/tagBedrockResponse() only understand Converse-shaped JSON (messages, output.message, usage.inputTokens) — even if a batch job call were reached, there's no field extraction for CreateModelInvocationJobRequest/GetModelInvocationJobResponse shapes (jobArn, inputDataConfig/S3 URI, outputDataConfig/S3 URI, status, roleArn).
  • No test, cassette, or example anywhere in the repo references CreateModelInvocationJob, GetModelInvocationJob, ModelInvocationJob, or the bedrock (non-bedrockruntime) client package — confirmed via repo-wide search.
  • No async submit → poll-status → retrieve-result lifecycle is modeled for this provider, unlike the (also-gapped, but at least structurally reachable) OpenAI (#164), Anthropic (#155), and Google GenAI (#165) batch APIs.

Braintrust docs status: not_found

Fetched https://www.braintrust.dev/docs/integrations/ai-providers/bedrock in full. It documents per-language tracing coverage for Converse/ConverseStream/InvokeModel/InvokeModelWithResponseStream across the TypeScript, Python, Go, and Java SDKs (explicitly noting for Java: "Other Bedrock operations such as InvokeModel and ApplyGuardrail are not traced" — consistent with #134). No mention anywhere on the page, for any language SDK, of batch inference / model invocation jobs / CreateModelInvocationJob. This is a documentation gap across the whole product, not just Java, but the Java SDK itself also has zero code path that could produce a span for this API even if docs existed.

Upstream sources

Local repo files inspected

  • braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustBedrockInterceptor.java — line 55 (INSTRUMENTED_OPERATIONS), lines 62–66 (allowlist enforcement), imports at lines 31–32 (bedrockruntime.model.ConverseRequest/ConverseStreamRequest only)
  • braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustAWSBedrock.java — lines 23–24 and 53–54 (wrap() overload signatures, both bedrockruntime builder types only)
  • braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/auto/AWSBedrockInstrumentationModule.java — lines 74–83 (BedrockBuilderAdvice.build() instanceof checks limited to bedrockruntime builders)
  • braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.javatagBedrockRequest()/tagBedrockResponse() handle only Converse-shaped bodies
  • braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/test/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustAWSBedrockTest.java — all tests use ConverseRequest/ConverseStreamRequest; no batch-job test exists
  • Repo-wide search for ModelInvocationJob, CreateModelInvocationJob, and the software.amazon.awssdk.services.bedrock. (control-plane) package — zero matches outside this issue
  • Existing related/non-duplicate issues reviewed: #134 (Bedrock InvokeModel embeddings — same module, different operation, not batch), #165 (Google GenAI Batches — different provider), #155 (Anthropic Message Batches — different provider), #164 (OpenAI Batch API — different provider)
Lingua principale
Java
Stelle
21
Fork
5
Merge medio
2g 7h
PR unite (30g)
8

Guida per i contributori

Apri la guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di braintrustdata/braintrust-sdk-java

Tutte le issue di braintrustdata/braintrust-sdk-java

Issue simili

Altre issue su Java

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.