openssl/openssl

DTLS should silently drop messages with invalid MAC instead of sending fatal alert

Open

#24,972 opened on Jul 23, 2024

View on GitHub
 (1 comment) (0 reactions) (1 assignee)C (11,262 forks)batch import
help wantedtriaged: bug

Repository metrics

Stars
 (30,157 stars)
PR merge metrics
 (No merged PRs in 30d)

Description

According to RFC 9147 4.5.2. Handling Invalid Records:

Unlike TLS, DTLS is resilient in the face of invalid records (e.g., invalid formatting, length, MAC, etc.). In general, invalid records SHOULD be silently discarded, thus preserving the association; however, an error MAY be logged for diagnostic purposes. Implementations which choose to generate an alert instead MUST generate fatal alerts to avoid attacks where the attacker repeatedly probes the implementation to see how it responds to various types of error. Note that if DTLS is run over UDP, then any implementation which does this will be extremely susceptible to DoS attacks because UDP forgery is so easy. Thus, generating fatal alerts is NOT RECOMMENDED for such transports, both to increase the reliability of DTLS service and to avoid the risk of spoofing attacks sending traffic to unrelated third parties.

With the current behavior I'm having issues with a use case of repeated connections on the same ports. It looks like Application Data messages that arrived from the client on the previous connection are buffered and handled after the new connection's handshake, possibly due to #20597. Since these Application Data messages are handled as part of a different connection, their MAC validation fails, which generates a fatal alert on ssl/record/methods/dtls_meth.c:173. I confirmed that removing the fatal alert fixes the issue I'm facing - the buffered messages from the previous connection are silently dropped. This is the recommendation from the RFC. I'll open a PR for the fix, please tell me if it's acceptable

Contributor guide