udp: default-on SO_REUSEPORT doesn't work with listening on port 0
#19.283 aperta il 14 dic 2021
Metriche repository
- Star
- (27.997 star)
- Metriche merge PR
- (Merge medio 8g) (378 PR mergiate in 30 g)
Descrizione
This is for tracking the issue we discussed about SO_REUSEPORT in https://github.com/envoyproxy/envoy/pull/19188. In Linux, if multiple Envoy instances are all configured to listen on port 0 for their UDP listeners, they may end up sharing the same port because of the default-on SO_REUSEPORT socket option if they are under the same user ID. This kernel behavior somewhat aligns with the contract of binding port 0 and SO_REUSEPORT. And a similar issue is explored here https://gavv.github.io/articles/ephemeral-port-reuse/.
This unexpected behavior was uncovered in QUIC integration tests because it affects UDP listeners which do not listen() more than TCP listeners. Currently SO_REUSEPORT is disabled in most of the integration tests(https://github.com/envoyproxy/envoy/pull/19188) to prevent parallel bazel tests sharing the same port.
The mitigation is only in tests, not in production. Given that we aren't aware of any use case where Envoy is configured to listen on port 0 in production. But if there is, the Envoy should also explicitly disable SO_REUSEPORT like https://github.com/envoyproxy/envoy/pull/19188 does. If both SO_REUSEPORT and listening on port 0 are desired, we need to revisit this issue and look into more sound solution, i.e. making Envoy manage a pool of ports.