[sdk-platform-java] A unary call allocates 19-28 KB client-side, and the documented remedy is a single-threaded batcher

Open
  • Is something wrong?
#14,999 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
15/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
gcp, grpc, java
Domain
databases, performance

Research direction

This is a measurement and API-design discussion rather than a bounded fix. Start with the allocation table and the referenced classes: CodedOutputStream$AbstractBufferedEncoder for the per-message encode buffer, and Batcher/BatcherImpl.add for the threading contract. Done would mean a reproducible bytes-per-call benchmark or a documented answer to the three asks, which needs maintainer decisions.

Written by the indexing model from the issue text.

Description

priority: p3 type: feature request

Versions: gax 2.90.0, google-cloud-bigtable via libraries-bom 26.93.0, JDK 25

Measurements

Allocated per call
mutateRowAsync, one cell, emulator 20.2 KB
... through newBulkMutationBatcher 3.9 KB
readRowAsync of a missing row, emulator 22.9 KB
readRowAsync of a missing row, real instance over TLS 29.1 KB
... through newBulkReadRowsBatcher 2.5 KB
checkAndMutateRowAsync no batched form at any layer
The measurement loop alone, sending nothing 1.0 KB

ThreadMXBean.getTotalThreadAllocatedBytes() across all threads: 60,000 warm-up calls, then 40,000
measured, twice in the same JVM. Passes agree within 0.1 KB and the figure is flat between 100 and 1,000
concurrent calls. No sampling. Channel built by InstantiatingGrpcChannelProvider. Built-in Bigtable
metrics make no measurable difference.

The problem

Batching is the documented answer, and it works — 2.5 KB against 29 KB. But batching is a remedy for
network round-trips, and it is being asked to carry client-side heap cost. Those are different decisions
at the call site: grouping writes to save round-trips is about latency and throughput; grouping unrelated
writes so the library allocates less shapes application code around a library's internals.

Two things make that hard to accept as the answer.

CheckAndMutateRow cannot be batched at any layer. BigtableDataClient exposes batcher factories
only for MutateRows and ReadRows, and the v2 data API has no plural counterpart to CheckAndMutateRow
or ReadModifyWriteRow. Conditional and read-modify-write operations therefore pay the full ~28 KB per
call, and no client change could alter that.

Batcher is documented as single-threaded. From its javadoc: "Batcher instances are not thread safe.
To use across different threads, create a new Batcher instance per thread." A batcher per thread batches
nothing useful, so amortising across concurrent requests means building a funnel — a channel and a
dedicated consumer thread owning the batcher — and owning its shutdown and flow-control behaviour.

BatcherImpl.add does guard the batch under synchronized (elementLock), and concurrent adds work in
practice — the measurements above do exactly that. But that is undocumented, and the javadoc says
otherwise, so an application cannot rely on it.

Why the real service costs more on an already-open channel

Emulator Real Delta
TLS (SslHandler, SSLEngine) 0.00 KB 3.60 KB +3.60
GAX logging interceptor 0.15 KB 1.95 KB +1.79
Credentials / access token 0.12 KB 0.83 KB +0.71
+6.10 against a measured +6.2

No handshake is involved — the channel is pooled in both cases. TLS is per message. The token is cached;
what costs is that the bearer header is a 1–2 KB string written into metadata and serialized on every
call. These splits are from a JFR allocation profile and are indicative; the totals above are counted.

Also worth a look: the largest single allocator in the real-service profile is
CodedOutputStream$AbstractBufferedEncoder.<init>, roughly 10 KB per call, a fresh encode buffer per
message for a request that asks for one row by key. The emulator profile does not show it at that size,
so this may be sampler skew.

This is noticed in pieces, but tracked by nobody

The logging-interceptor row above is already diagnosed and fixed in
#12810, "skip metadata materialization when
logging is disabled", open since 15 April with one review comment and no activity since. We rediscovered
it independently before finding that PR.

On the batching path, #14016 (entry proto
built twice per batched element, with #14017
open to fix it) and #14018
(RowMutationEntry exposes no serialized size) have been open since August without response.

Each is small on its own. Together with the figures above they suggest per-call allocation is not a
tracked property of these clients, which is consistent with us finding no benchmark that reports bytes
per unary call.

Context

A single-vCPU Cloud Run instance taking 200 concurrent requests, 2.2 Bigtable calls per message. gRPC and
GAX account for about a third of everything the process allocates; all JSON parsing and serialisation is
about 10%. At one vCPU the JVM selects SerialGC, so every young collection stops request processing. Of
those 2.2 calls, 0.6 are conditional writes implementing exactly-once semantics, which cannot be batched.

Asks

  1. Benchmark bytes allocated per unary call, and treat it as a tracked property.
  2. Clarify whether ~19 KB per unary call is expected. If it is, we would suggest saying so where batching
    is recommended, and documenting that conditional writes have no equivalent remedy.
  3. Clarify Batcher's threading contract. If concurrent add is supported, saying so would remove the
    need for application-side funnelling.
  4. Merge or close #12810.

Happy to share the harness.

Dominant language
Java
Stars
2.1k
Forks
1.2k
Avg merge
2d 14h
Merged PRs (30d)
192

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from googleapis/google-cloud-java

All issues in googleapis/google-cloud-java

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.