[bot] AWS Bedrock batch inference (CreateModelInvocationJob) is not instrumented at all
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Idoneità per principianti
- 35/100
- Tipo di issue
- Funzionalità
- Chiarezza
- Specificata chiaramente
- Stato di attività
- Attiva
- Stack tecnologico
- aws, java
- Ambito
- backend, observability-sre
Direzione di ricerca
Inizia da BraintrustAWSBedrock.java, AWSBedrockInstrumentationModule.java, BraintrustBedrockInterceptor.java e InstrumentationSemConv.java, quindi esamina BraintrustAWSBedrockTest.java e le API client del control plane di AWS Bedrock. Aggiungi la copertura per le operazioni di batch job indicate, comprese le relative strutture di richiesta e risposta e il relativo ciclo di vita, e verificala con test mirati o cassette per l’invio, lo stato, l’elenco e l’arresto.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Summary
The aws_bedrock_2_30_0 module only ever wraps the Bedrock data-plane runtime client builders (software.amazon.awssdk.services.bedrockruntime.BedrockRuntimeClientBuilder / BedrockRuntimeAsyncClientBuilder), and only tags Converse/ConverseStream operations on that client (tracked separately in #134 for InvokeModel). AWS Bedrock's official batch inference API — submitted through the entirely separate control-plane client software.amazon.awssdk.services.bedrock.BedrockClient (CreateModelInvocationJob / GetModelInvocationJob / ListModelInvocationJobs / StopModelInvocationJob) — has no wrapping mechanism at all. This is a different Maven artifact/class hierarchy (bedrock vs. bedrockruntime) that this module never references, so unlike the Gemini Batches gap (#165, where calls at least reach the wrapper and get mis-tagged), a Bedrock batch inference job call today produces zero span — not even a generic or mis-tagged one.
What is missing
BraintrustBedrockInterceptor.javaline 55:private static final Set<String> INSTRUMENTED_OPERATIONS = Set.of("Converse", "ConverseStream");, enforced at lines 62–66 (beforeExecutionreturns immediately for any other operation name). This interceptor is only ever attached to aBedrockRuntimeClientBuilder/BedrockRuntimeAsyncClientBuilderin the first place (see next point), so it structurally cannot see batch-inference-job calls regardless of this allowlist.BraintrustAWSBedrock.java(lines 23–24, 53–54): both publicwrap()overloads are typed to accept onlyBedrockRuntimeClientBuilder/BedrockRuntimeAsyncClientBuilder. There is nowrap(BedrockClientBuilder)overload, so a manual user has no supported way to instrument the control-plane client that hosts batch inference at all.AWSBedrockInstrumentationModule.java(BedrockBuilderAdvice.build(), lines 74–83): the auto-instrumentationinstanceofchecks are likewise limited toBedrockRuntimeClientBuilder/BedrockRuntimeAsyncClientBuilder. ABedrockClientBuilder(control plane) passing throughSdkDefaultClientBuilder.build()falls through both branches untouched.InstrumentationSemConv.tagBedrockRequest()/tagBedrockResponse()only understand Converse-shaped JSON (messages,output.message,usage.inputTokens) — even if a batch job call were reached, there's no field extraction forCreateModelInvocationJobRequest/GetModelInvocationJobResponseshapes (jobArn,inputDataConfig/S3 URI,outputDataConfig/S3 URI,status,roleArn).- No test, cassette, or example anywhere in the repo references
CreateModelInvocationJob,GetModelInvocationJob,ModelInvocationJob, or thebedrock(non-bedrockruntime) client package — confirmed via repo-wide search. - No async submit → poll-status → retrieve-result lifecycle is modeled for this provider, unlike the (also-gapped, but at least structurally reachable) OpenAI (#164), Anthropic (#155), and Google GenAI (#165) batch APIs.
Braintrust docs status: not_found
Fetched https://www.braintrust.dev/docs/integrations/ai-providers/bedrock in full. It documents per-language tracing coverage for Converse/ConverseStream/InvokeModel/InvokeModelWithResponseStream across the TypeScript, Python, Go, and Java SDKs (explicitly noting for Java: "Other Bedrock operations such as InvokeModel and ApplyGuardrail are not traced" — consistent with #134). No mention anywhere on the page, for any language SDK, of batch inference / model invocation jobs / CreateModelInvocationJob. This is a documentation gap across the whole product, not just Java, but the Java SDK itself also has zero code path that could produce a span for this API even if docs existed.
Upstream sources
- AWS Bedrock batch inference user guide: https://docs.aws.amazon.com/bedrock/latest/userguide/batch-inference.html and https://docs.aws.amazon.com/bedrock/latest/userguide/batch-inference-create.html — describes submitting a batch inference job against an S3 input file and retrieving results from S3 once the job completes.
- API reference: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_CreateModelInvocationJob.html (also
GetModelInvocationJob,ListModelInvocationJobs,StopModelInvocationJob). - AWS SDK for Java v2 control-plane client:
software.amazon.awssdk.services.bedrock.BedrockClient#createModelInvocationJob— https://docs.aws.amazon.com/java/api/latest/software/amazon/awssdk/services/bedrock/BedrockClient.html and https://docs.aws.amazon.com/java/api/latest/software/amazon/awssdk/services/bedrock/model/CreateModelInvocationJobRequest.html — this is a distinct Maven artifact (software.amazon.awssdk:bedrock) from the already-instrumentedsoftware.amazon.awssdk:bedrockruntime.
Local repo files inspected
braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustBedrockInterceptor.java— line 55 (INSTRUMENTED_OPERATIONS), lines 62–66 (allowlist enforcement), imports at lines 31–32 (bedrockruntime.model.ConverseRequest/ConverseStreamRequestonly)braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustAWSBedrock.java— lines 23–24 and 53–54 (wrap()overload signatures, bothbedrockruntimebuilder types only)braintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/main/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/auto/AWSBedrockInstrumentationModule.java— lines 74–83 (BedrockBuilderAdvice.build()instanceofchecks limited tobedrockruntimebuilders)braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java—tagBedrockRequest()/tagBedrockResponse()handle only Converse-shaped bodiesbraintrust-sdk/instrumentation/aws_bedrock_2_30_0/src/test/java/dev/braintrust/instrumentation/awsbedrock/v2_30_0/BraintrustAWSBedrockTest.java— all tests useConverseRequest/ConverseStreamRequest; no batch-job test exists- Repo-wide search for
ModelInvocationJob,CreateModelInvocationJob, and thesoftware.amazon.awssdk.services.bedrock.(control-plane) package — zero matches outside this issue - Existing related/non-duplicate issues reviewed: #134 (Bedrock
InvokeModelembeddings — same module, different operation, not batch), #165 (Google GenAI Batches — different provider), #155 (Anthropic Message Batches — different provider), #164 (OpenAI Batch API — different provider)
- Lingua principale
- Java
- Stelle
- 21
- Fork
- 5
- Merge medio
- 2g 7h
- PR unite (30g)
- 8
Guida per i contributori
Apri la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di braintrustdata/braintrust-sdk-java
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
Tutte le issue di braintrustdata/braintrust-sdk-java
Issue simili
-
bug
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 90/100
apache/cloudstack#14222 ·
-
[BUG]茶杯方块在取茶时会引发崩溃 Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 88/100
-
Cannot differ own consent and managed consents in My Consents view and detailed consent view. Aperta1.0.0-alpha2 Type/Improvement
Difficoltà 2/5 1-3 ore Idoneità per principianti 68/100
wso2/dpdp-accelerator#272 ·
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
infinispan/infinispan#18150 ·
-
area/frontend
Difficoltà 2/5 1-3 ore Idoneità per principianti 65/100