AWS SDK for Java v2 — S3TransferManager + Netty-based S3AsyncClient.builder() does not perform parallel ranged GETs for a multipart-uploaded object

Open
#7,350 4 comments 0 reactions 1 assignee View on GitHub

@debora-ito is already working on this.

Since Sep 8, 2026.

Assessment

This issue has not been assessed yet.

Description

bug
Describe the bug

Summary

When downloading a large (multipart-uploaded) S3 object through S3TransferManager backed by the standard Netty-based S3AsyncClient.builder(), the download is executed as a single, non-ranged GET instead of multiple parallel ranged GETs. As a result the transfer runs at single-connection throughput and never uses the configured concurrency.

Control experiments confirm the problem is specific to this combination:

Client | Parallel ranged GETs? -- | -- S3AsyncClient.crtBuilder() + S3TransferManager | ✅ Yes — works as documented S3AsyncClient.builder() (Netty) + S3TransferManager | ❌ No — single GET, no Range header Manual S3AsyncClient.builder() (Netty) issuing per-part GetObjectRequest with partNumber/Range | ✅ Yes — client itself is fully capable

So the S3 object supports ranged reads, the Netty client is fine, and CRT + TransferManager works. Only S3TransferManager on top of the Netty async client fails to split the download.

Presence/absence of an HTTP proxy is irrelevant — the bug reproduces with no proxy at all.

Regression Issue
  • Select this option if this issue appears to be a regression.
Expected Behavior

S3TransferManager + Netty-based S3AsyncClient.builder() should perform parallel ranged GETs for a multipart-uploaded object

Current Behavior

For a ~140 MB MPU object, S3TransferManager.downloadFile(...) should — regardless of the underlying async client — issue multiple parallel GET requests with Range: bytes=start-end (or ?partNumber=N) to saturate maxConcurrency. This is the primary value proposition of S3TransferManager.

Either:

S3TransferManager on the Netty client should perform the split itself, or
The documentation should clearly state that split-download only works with the CRT client, and S3TransferManager.builder().s3Client(nettyAsyncClient) should log a prominent WARN at construction time.

Reproduction Steps

Configure an SSO profile default in ~/.aws/config.
Run the reproducer below (essentially S3TransforManagerTest.java in this repo).
Observe the wire logs and progress output.
Reproducer (Netty S3AsyncClient.builder() + S3TransferManager)
try (S3AsyncClient s3 = S3AsyncClient.builder()
.region(Region.of("us-east-1"))
.credentialsProvider(ProfileCredentialsProvider.create("default"))
.httpClientBuilder(NettyNioAsyncHttpClient.builder()
.maxConcurrency(200)
.connectionTimeout(Duration.ofSeconds(30))
.readTimeout(Duration.ofSeconds(60)))
.build();
S3TransferManager tm = S3TransferManager.builder().s3Client(s3).build()) {

FileDownload dl = tm.downloadFile(DownloadFileRequest.builder()
        .getObjectRequest(b -> b.bucket("pyzhoutest").key("ffff"))
        .destination(Paths.get("downloads/large-file.bin"))
        .build());

dl.completionFuture().join();

}
Control 1 — CRT client (works correctly)
Same code but with S3AsyncClient.crtBuilder()...build() → multiple parallel ranged GETs are observed, throughput scales with concurrency. ✅

Control 2 — Manual parallel GETs on Netty client (works correctly)
Using the same Netty S3AsyncClient.builder() client but issuing N getObject(...) calls in parallel, each with .partNumber(i) or .range("bytes=start-end"), achieves full parallelism. ✅ This proves the Netty client and the network path have no problem serving ranged parallel GETs; only S3TransferManager's split logic is missing on this client.

Possible Solution

No response

Additional Information/Context

Only one GET is ever sent, without Range and without partNumber, and bytes arrive on a single Netty event-loop thread:

开始下载: s3://pyzhoutest/ffff
对象大小: 147297280 bytes, 按 5MB 估算分片数: 29
HeadObject: ETag="af08b66120cc11d69332976330da8273-29", partsCount=null, acceptRanges=bytes
如果并行分片生效,下面会看到多条 GET 请求且带 Range=bytes=start-end
[HTTP-OUT][main] GET https://s3.amazonaws.com/pyzhoutest/ffff Range=
进度: 0 bytes, 平均速度: 0.00 MB/s
[HTTP-IN ][aws-java-sdk-NettyEventLoop-1-2] status=200
进度: 0.01% (16715/147297280 bytes), 平均速度: 0.02 MB/s
进度: 0.02% (34123/147297280 bytes), 平均速度: 0.02 MB/s
进度: 0.04% (53261/147297280 bytes), 平均速度: 0.02 MB/s
...
Key observations:

Exactly one GET line for the whole transfer.
No Range: bytes=... header, no ?partNumber=N.
All bytes on a single aws-java-sdk-NettyEventLoop-* thread → the client-level max concurrency is unused.
DEBUG logs on software.amazon.awssdk show no “split / schedule part / range” activity from TransferManager.
What I already tried
DEBUG logging on software.amazon.awssdk — no evidence of any split-download logic being invoked when the underlying client is the Netty S3AsyncClient.
Raised Netty maxConcurrency to 200 — no change; still 1 in-flight GET.
forcePathStyle(true) — no change.
Removed the HTTP proxy entirely — no change (single non-ranged GET).
Switched to S3AsyncClient.crtBuilder() with everything else identical — parallel ranged GETs work correctly.
Manual multi-part parallel GETs on the same Netty client — work correctly.

AWS Java SDK version used

2.54.7 & 2.35.11

JDK version used

oracle-17

Operating System and version

windows 11

Dominant language
Java
Stars
2.6k
Forks
1k
Avg merge
2d 17h
Merged PRs (30d)
39

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from aws/aws-sdk-java-v2

All issues in aws/aws-sdk-java-v2

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.