Best way to handle events that are too large for buffering
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
Research direction
Start in RawSocketSender.send(), especially the pendings buffer and the BufferOverflowException path shown in the stack trace; compare that behavior with the BufferedOutputStream usage. Determine and document the intended handling for events larger than the buffer and for sender errors, then verify that the chosen behavior is covered by tests.
Written by the indexing model from the issue text.
Description
When looking through RawSocketSender to troubleshoot our other issue (see comment on #18), I noticed that RawSocketSender does not handle large events very well in send(). It assumes that flush() always makes enough capacity in the pendings buffer to save the new bytes. If the event was bigger than the buffer, you get a BufferOverflowException.
java.nio.BufferOverflowException
at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:189)
at java.nio.ByteBuffer.put(ByteBuffer.java:859)
at org.fluentd.logger.sender.RawSocketSender.send(RawSocketSender.java:171)
Also, if every log message you send is larger than the buffer, then the RawSocketSender will attempt to flush to the socket on every message.
Is this the desired behavior? I could imagine adding a size check to the send() method that immediately returns false if the event is too large, to avoid the BufferOverflowException. Or perhaps the large event could skip the buffer and write directly out to the socket. Since we are already using a BufferedOutputStream, I also wonder why we need the internal pendings buffer at all?
What is the normal way of handling errors in RawSocketSender?
- Dominant language
- Java
- Stars
- 210
- Forks
- 86
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from fluent/fluent-logger-java
-
Difficulty 3/5 1-2 days Newbie friendliness 45/100
fluent/fluent-logger-java#100 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 15/100
fluent/fluent-logger-java#99 · 1 comment · 2 reactions ·
-
Difficulty 3/5 1-2 days Newbie friendliness 45/100
fluent/fluent-logger-java#98 · 1 comment ·
-
Null Pointer Exception with slf4j-log4j12-1.7.30 (Fluent-logger incompatible with the new version) Open
Difficulty 3/5 1-2 days Newbie friendliness 42/100
fluent/fluent-logger-java#89 · 3 comments ·
-
Difficulty 3/5 1-2 days Newbie friendliness 25/100
fluent/fluent-logger-java#88 · 1 comment ·
All issues in fluent/fluent-logger-java
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
elastic/gradle-plugins#157 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
cryptomator/hub#497 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
johanhaleby/occurrent#1120 ·