Unable to use 2g files as the content for an UploadPartRequest when performing multipart uploads

Start by reproducing the failure with the provided 2g and Integer.MAX_VALUE file cases using S3AsyncClient.crtBuilder() and UploadPartRequest. Read ByteBufferStoringSubscriber.transferTo and S3CrtRequestBodyStreamAdapter.sendRequestBody, then trace the large-part request path. Done means multipart uploads succeed for parts larger than Integer.MAX_VALUE without the reported exception.

Describe the bug

Unable to perform very large multipart uploads as the largest size for an individual part is Integer.MAX_VALUE.

The docs say the current limits for a part should be 5g and I could uploads parts of at least 2g via the CLI - just not via the java SDK

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

The multipart upload to work for large parts

Current Behavior

Fails with the error

Exception in thread "AwsEventLoop 6" java.lang.NullPointerException: Cannot invoke "java.nio.ByteBuffer.hasRemaining()" because "out" is null
	at software.amazon.awssdk.utils.async.ByteBufferStoringSubscriber.transferTo(ByteBufferStoringSubscriber.java:100)
	at software.amazon.awssdk.services.s3.internal.crt.S3CrtRequestBodyStreamAdapter.sendRequestBody(S3CrtRequestBodyStreamAdapter.java:47)
java.util.concurrent.CompletionException: software.amazon.awssdk.core.exception.SdkClientException: Failed to send the request: A callback has reported failure. (SDK Attempt Count: 1)
	at software.amazon.awssdk.utils.CompletableFutureUtils.errorAsCompletionException(CompletableFutureUtils.java:64)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncExecutionFailureExceptionReportingStage.lambda$execute$0(AsyncExecutionFailureExceptionReportingStage.java:51)
	at java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:934)
	at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:911)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2194)
	at software.amazon.awssdk.utils.CompletableFutureUtils.lambda$forwardExceptionTo$0(CompletableFutureUtils.java:78)
	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2194)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeAttemptExecute(AsyncRetryableStage.java:139)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.maybeRetryExecute(AsyncRetryableStage.java:157)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.AsyncRetryableStage$RetryingExecutor.lambda$attemptExecute$1(AsyncRetryableStage.java:117)
	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2194)
	at software.amazon.awssdk.utils.CompletableFutureUtils.lambda$forwardExceptionTo$0(CompletableFutureUtils.java:78)
	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2194)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.lambda$execute$0(MakeAsyncHttpRequestStage.java:108)
	at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
	at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
	at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2194)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.completeResponseFuture(MakeAsyncHttpRequestStage.java:255)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.MakeAsyncHttpRequestStage.lambda$executeHttpRequest$3(MakeAsyncHttpRequestStage.java:167)
	at java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:934)
	at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:911)
	at java.base/java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:482)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)
Caused by: software.amazon.awssdk.core.exception.SdkClientException: Failed to send the request: A callback has reported failure. (SDK Attempt Count: 1)
	at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:130)
	at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:95)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.utils.RetryableStageHelper.retryPolicyDisallowedRetryException(RetryableStageHelper.java:168)
	... 25 more

When debug was enabled on the CRT client the logs had an entry with error code 2064 but little else

Reproduction Steps

Create 2 files via

mkfile -n 2147483647 file2g_1
mkfile -n 2g file2g

Then pass these paths (and their file size of (long) Integer.MAX_VALUE and Integer.MAX_VALUE + 1L) to the following bit of code

    private void reproducer(String bucket, String s3Key, Path filePath, long fileSize) throws Exception {
        try (var client = S3AsyncClient.crtBuilder().build()) {
            var uploadId = client.createMultipartUpload(CreateMultipartUploadRequest.builder()
                    .bucket(bucket)
                    .key(s3Key)
                    .build())
                    .get()
                    .uploadId();

            var partRequest = UploadPartRequest.builder()
                    .bucket(bucket)
                    .key(s3Key)
                    .uploadId(uploadId)
                    .partNumber(1)
                    .contentLength(fileSize)
                    .build();

            var eTag = client.uploadPart(partRequest, fromFile(filePath)).get().eTag();
            var parts = List.of(CompletedPart.builder()
                    .partNumber(1)
                    .eTag(eTag)
                    .build());

            client.completeMultipartUpload(CompleteMultipartUploadRequest.builder()
                    .bucket(bucket)
                    .key(s3Key)
                    .uploadId(uploadId)
                    .multipartUpload(CompletedMultipartUpload.builder()
                            .parts(parts)
                            .build())
                    .build())
                    .get();
        }
    }

It works OK for file2g_1 but fails on file2g.

Possible Solution

No response

Additional Information/Context

I tried via the input stream method of AsyncRequestBody and that failed too

I had a suspicion it might be related to the checksum calculations following some step-debugging but changing the algorithms had no effect sadly

I also tried with a file of size 4g to check it wasn't a boundary issue on Integer.MAX but that failed as well.

AWS Java SDK version used

2.41.4

JDK version used

21

Operating System and version

Mac and Linux (EC2)