[CI] NettyClientTest.testServerDisconnection may fail with DisconnectException
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 86/100
Research direction
Start with NettyClientTest.testServerDisconnection around the failing assertion at line 165. Read how the test waits for the connection map to become empty after the request fails, then add a wait after nettyServer.close() and before the next request while retaining the existing cleanup wait. Run the targeted test repeatedly; it should consistently assert ConnectException.
Written by the indexing model from the issue text.
Description
Search before asking
- I searched in the issues and found nothing similar.
Fluss version
1.0.0 (latest release)
Please describe the bug 🐞
NettyClientTest.testServerDisconnection can still fail intermittently in CI after #4344.
The test expects a ConnectException when sending a request after shutting down the Netty server, but occasionally receives a DisconnectException instead:
NettyClientTest.testServerDisconnection:165
Expecting actual throwable to be an instance of:
java.net.ConnectException
but was:
org.apache.fluss.exception.DisconnectException:
Cannot send request to server localhost:40463
(id: cs-1, rack: null) because it is disconnected.
https://github.com/apache/fluss/actions/runs/39053988878/job/117458107688?pr=4545
After nettyServer.close(), the client detects the disconnection and asynchronously removes the old ServerConnection from its connection map.
However, the test immediately sends another request without waiting for the old connection to be removed.
If the request arrives after the connection has entered the DISCONNECTED state but before it is removed from the map, NettyClient.getOrCreateConnection() may return the disconnected connection.
In this case, ServerConnection.doSend() fails the request with a DisconnectException rather than attempting a new connection that would fail with a ConnectException.
This makes the exception assertion sensitive to the timing of asynchronous connection cleanup.
The previous fix in #4344 added a retry to wait for the connection map to become empty, but that retry happens after the failed request and therefore does not prevent this race.
Solution
Wait for the old connection to be removed after shutting down the server and before sending the next request.
Keep the existing retry after the failed request to ensure that the newly created, failed connection is also cleaned up before restarting the server.
This preserves the existing ConnectException assertion and requires no changes to production code.
Are you willing to submit a PR?
- I'm willing to submit a PR!
- Dominant language
- Java
- Stars
- 2.2k
- Forks
- 656
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 120
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/fluss
-
[lake/iceberg] Reserved write-mode properties can be set and reset through their `iceberg.` keyPossibly taken @aphenix-r7 claimed this 3 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
apache/fluss#4658 · 1 comment ·
Maintainers usually reply within 1 day
-
[common] Decimal#hashCode is inconsistent with Decimal#equalsPossibly taken @firasbouzazi claimed this 14 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
apache/fluss#4538 · 1 comment ·
Maintainers usually reply within 1 day
-
[security] PlainSaslServerProvider conflicts with Hadoop SASL provider causing authentication failurePossibly taken @xinzhang-hello claimed this 20 days ago. Open
Difficulty 1/5 Under an hour Newbie friendliness 88/100
Maintainers usually reply within 1 day
-
[server] KV snapshot scheduling stops permanently after a transient snapshot-init failurePossibly taken @zhang-arvin claimed this 20 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
apache/fluss#4489 · 1 comment ·
Maintainers usually reply within 1 day
-
[docs] Document advertised-resource and ACL principal ignore-case optionsPossibly taken @EchoLee5 claimed this 32 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Maintainers usually reply within 1 day
Similar issues
-
NullPointerException when a `DeserializationProblemHandler` returns `null` for `byte[]` (single value unwrapped) and `char[]` elementsPossibly taken @Marti2203 claimed this today. Open
Difficulty 2/5 Half a day Newbie friendliness 88/100
FasterXML/jackson-databind#6300 ·
Maintainers usually reply within 1 day
-
content good first issue
Difficulty 1/5 Under an hour Newbie friendliness 95/100
cpinitiative/usaco-guide#6699 ·
Maintainers usually reply within 1 day
-
BC remittance upload: a claim MSP refused (S03) is saved with a blank status instead of refused (F)Possibly taken A pull request linked to this issue is open or already merged. Opentype: bug
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
carlos-emr/carlos#4933 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
apache/rocketmq-dashboard#6925 ·
Maintainers usually reply within 3 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Vakore/ZappierGames#147 ·