Connections can get stuck when upgrading to SSL

Offen
  • Stimmt etwas nicht?
#916 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

Dieses Issue hat noch niemand übernommen.

Bewertung

Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Anfängerfreundlichkeit
45/100
Issue-Typ
Bug
Klarheit
Größtenteils klar
Aktivitätsstatus
Ruhig
Tech-Stack
erlang
Bereich
networking

Rechercherichtung

Beginne in src/hackney_conn.erl bei hackney_conn:connected/3 und dem Aufruf ssl:connect/2. Verfolge anschließend, wie sich eine gepoolte Verbindung verhält, während der TLS-Handshake wartet. Bestätige das gewünschte optionale Timeout-Verhalten und identifiziere die relevanten Verbindungstests; abgeschlossen ist die Aufgabe, wenn ein fehlgeschlagenes oder festgefahrenes TLS-Upgrade nicht mehr auf unbestimmte Zeit eine Poolverbindung belegt.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

Environment
  • hackney: 4.7.2
  • Erlang/OTP: 26
Description

I observed a behavior in an integration environment where the default connection pool becomes fully occupied by connections which were stuck attempting to upgrade to SSL. Within hackney_con:connected there is a call to ssl:connect/2 which results in utilizing the default timeout of infinity. If an issue occurs during the TLS handshake, the connection process remains in a waiting state and attempts to utilize the connection time out repeatedly as the call to hackney:maybe_register_h2/6 must pull the protocol from the state of the pooled connection which times out.

Is it possible to implement a concept of tls upgrade timeout or tls handshake timeout which can be optionally passed to an ssl:connect/3 call in the upgrade to TLS path?

More Details

Hackney Connection Supervisor has 4 linked processes in the pool and appears in a healthy state

[{registered_name,hackney_conn_sup},
 {current_function,{gen_server,loop,7}},
 {initial_call,{proc_lib,init_p,5}},
 {status,waiting},
 {message_queue_len,0},
 {links,[<0.663122.0>,<0.663147.0>,<0.663279.0>,<0.663134.0>,
         <0.1568.0>]},
 {dictionary,[{'$ancestors',[hackney_sup,<0.1567.0>]},
              {'$initial_call',{supervisor,hackney_conn_sup,1}}]},
 {trap_exit,true},
 {error_handler,error_handler},
 {priority,normal},
 {group_leader,<0.1566.0>},
 {total_heap_size,2208},
 {heap_size,1598},
 {stack_size,11},
 {reductions,865131},
 {garbage_collection,[{max_heap_size,#{error_logger => true,include_shared_binaries => false,
                                       kill => true,size => 0}},
                      {min_bin_vheap_size,46422},
                      {min_heap_size,233},
                      {fullsweep_after,65535},
                      {minor_gcs,35}]},
 {suspending,[]}]

One of the connections in the pool is inspected and is found in the waiting state with two messages in the queue. Connection is waiting on ssl handshake to complete

23> process_info(<0.663122.0>).
[{current_function,{gen,do_call,4}},
 {initial_call,{proc_lib,init_p,5}},
 {status,waiting},
 {message_queue_len,2},
 {links,[<0.1570.0>]},
 {dictionary,[{'$ancestors',[hackney_conn_sup,hackney_sup,
                             <0.1567.0>]},
              {'$initial_call',{hackney_conn,init,1}}]},
 {trap_exit,true},
 {error_handler,error_handler},
 {priority,normal},
 {group_leader,<0.1566.0>},
 {total_heap_size,3203},
 {heap_size,1598},
 {stack_size,47},
 {reductions,14839},
 {garbage_collection,[{max_heap_size,#{error_logger => true,include_shared_binaries => false,
                                       kill => true,size => 0}},
                      {min_bin_vheap_size,46422},
                      {min_heap_size,233},
                      {fullsweep_after,65535},
                      {minor_gcs,9}]},
 {suspending,[]}]

26> process_info(<0.663122.0>, [current_stacktrace]).
[{current_stacktrace,[{gen,do_call,4,
                           [{file,"gen.erl"},{line,240}]},
                      {gen_statem,call,3,[{file,"gen_statem.erl"},{line,923}]},
                      {ssl_gen_statem,call,2,
                                      [{file,"ssl_gen_statem.erl"},{line,1372}]},
                      {ssl_gen_statem,handshake,2,
                                      [{file,"ssl_gen_statem.erl"},{line,253}]},
                      {ssl_gen_statem,connect,8,
                                      [{file,"ssl_gen_statem.erl"},{line,222}]},
                      {ssl,connect,3,[{file,"ssl.erl"},{line,637}]},
                      {hackney_conn,connected,3,
                                    [{file,"/codebuild/output/src3735438855/src/mirror/_build/default/lib/hackney/src/hackney_conn.erl"},
                                     {line,909}]},
                      {gen_statem,loop_state_callback,11,
                                  [{file,"gen_statem.erl"},{line,1395}]}]}]

At this point in time I am not exactly sure what failure occurred to cause the handshake attempt to fail in this way so I can not reliably reproduce this issue. Regardless of the failure scenario I believe a timeout in this location would allow configuration of some safety rails against whatever kind of failures could happen.

Vorherrschende Sprache
Erlang
Sterne
1.4k
Forks
458
Ø Merge
11 Std. 32 Min.
Gemergte PRs (30 T.)
47

Entwicklungsumgebung

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus benoitc/hackney

Alle Issues in benoitc/hackney

Ähnliche Issues

Weitere Issues zu Networking

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.