Improvement: Optimize CompressingRequestBody memory usage by removing Okio and redundant buffering
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 48/100
- Issue type
- Refactor
- Clarity
- Clearly specified
- Activity status
- Stale
- Tech stack
- java
- Domain
- backend, performance
Research direction
Start by locating CompressingRequestBody.attemptWrite in the profiling-uploader code and inspect its current Okio copy path, compression stream, and buffering. Verify the Java 8-compatible stream-copy behavior and confirm that the change preserves compression output, stream ownership, and the intended memory reduction.
Written by the indexing model from the issue text.
Description
Library Name
dd-java-agent (profiling-uploader)
Library Version(s)
v1.54.0
Describe the feature you'd like
I propose optimizing CompressingRequestBody.attemptWrite to reduce memory allocation.
Currently, the implementation uses Okio for stream copying and wraps the compression stream with an extra BufferedOutputStream. This results in excessive object creation (Segments, Buffers) and double buffering overhead.
Suggested Implementation:
I suggest replacing Okio with a standard Java IO loop using a fixed-size byte[] buffer and removing the redundant outer BufferedOutputStream.
private void attemptWrite(@Nonnull InputStream inputStream, @Nonnull OutputStream outputStream)
throws IOException {
// Keep the inner buffer to aggregate compressed bytes before sending to the socket
OutputStream bufferedInnerStream = new BufferedOutputStream(outputStream) {
@Override
public void close() throws IOException {
flush(); // Prevent closing the underlying stream
}
};
// Remove the outer BufferedOutputStream.
// Direct writing to the compression stream is efficient enough with a 8KB copy buffer.
OutputStream processingStream = isCompressed(inputStream)
? bufferedInnerStream
: outputStreamMapper.apply(bufferedInnerStream);
try (OutputStream out = processingStream) {
byte[] buffer = new byte[8192];
int bytesRead;
while ((bytesRead = inputStream.read(buffer)) != -1) {
out.write(buffer, 0, bytesRead);
}
}
// implicit close() on processingStream finalizes compression and flushes the inner stream
}
Is your feature request related to a problem?
Yes. While profiling our application, we noticed that CompressingRequestBody.attemptWrite is a major contributor to heap usage.
In our specific profile, this method accounted for approximately 94% of the Heap Live Size. The heavy use of Okio intermediate objects for simple stream copying seems to be the primary cause.
Please see the attached Flame Graph for evidence:
Describe alternatives you've considered
Using InputStream.transferTo(OutputStream) is an option for Java 9+, but the proposed byte-array loop ensures compatibility with Java 8, which I believe the agent still supports.
Additional context
Note: I am not a native English speaker, so I used an AI assistant to help draft this issue to ensure clarity. Thank you for your understanding.
- Dominant language
- Java
- Stars
- 737
- Forks
- 361
- Avg merge
- 3d 20h
- Merged PRs (30d)
- 173
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from DataDog/dd-trace-java
-
type: feature request
Difficulty 1/5 1-3 hours Newbie friendliness 70/100
DataDog/dd-trace-java#10245 · 1 comment ·
-
Difficulty 4/5 3-5 days Newbie friendliness 62/100
DataDog/dd-trace-java#12608 ·
-
type: bug report
Difficulty 4/5 3-5 days Newbie friendliness 35/100
DataDog/dd-trace-java#12597 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
DataDog/dd-trace-java#12540 · 3 comments · 1 assignee ·
-
Difficulty 3/5 1-2 days Newbie friendliness 25/100
DataDog/dd-trace-java#12480 ·
All issues in DataDog/dd-trace-java
Similar issues
-
certification
Difficulty 1/5 Under an hour Newbie friendliness 80/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
[BUG] ECR GetAuthorizationToken returns a proxyEndpoint for the default region, not the request's Openbug ecr
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Needs: Triage Type: Feature request
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
AntennaPod/AntennaPod#8794 ·
-
agentic-workflows
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
github/copilot-sdk#2760 ·