Connections can get stuck when upgrading to SSL

Aperta
  • Qualcosa non va?
#916 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
45/100
Tipo di issue
Bug
Chiarezza
Abbastanza chiara
Stato di attività
Tranquilla
Stack tecnologico
erlang
Ambito
networking

Direzione di ricerca

Inizia in src/hackney_conn.erl, in hackney_conn:connected/3 e nella chiamata a ssl:connect/2, quindi traccia il comportamento di una connessione del pool mentre l’handshake TLS è in attesa. Conferma il comportamento desiderato del timeout opzionale e identifica i test di connessione pertinenti; il lavoro è completato quando un upgrade TLS fallito o bloccato non occupa più indefinitamente una connessione del pool.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Environment
  • hackney: 4.7.2
  • Erlang/OTP: 26
Description

I observed a behavior in an integration environment where the default connection pool becomes fully occupied by connections which were stuck attempting to upgrade to SSL. Within hackney_con:connected there is a call to ssl:connect/2 which results in utilizing the default timeout of infinity. If an issue occurs during the TLS handshake, the connection process remains in a waiting state and attempts to utilize the connection time out repeatedly as the call to hackney:maybe_register_h2/6 must pull the protocol from the state of the pooled connection which times out.

Is it possible to implement a concept of tls upgrade timeout or tls handshake timeout which can be optionally passed to an ssl:connect/3 call in the upgrade to TLS path?

More Details

Hackney Connection Supervisor has 4 linked processes in the pool and appears in a healthy state

[{registered_name,hackney_conn_sup},
 {current_function,{gen_server,loop,7}},
 {initial_call,{proc_lib,init_p,5}},
 {status,waiting},
 {message_queue_len,0},
 {links,[<0.663122.0>,<0.663147.0>,<0.663279.0>,<0.663134.0>,
         <0.1568.0>]},
 {dictionary,[{'$ancestors',[hackney_sup,<0.1567.0>]},
              {'$initial_call',{supervisor,hackney_conn_sup,1}}]},
 {trap_exit,true},
 {error_handler,error_handler},
 {priority,normal},
 {group_leader,<0.1566.0>},
 {total_heap_size,2208},
 {heap_size,1598},
 {stack_size,11},
 {reductions,865131},
 {garbage_collection,[{max_heap_size,#{error_logger => true,include_shared_binaries => false,
                                       kill => true,size => 0}},
                      {min_bin_vheap_size,46422},
                      {min_heap_size,233},
                      {fullsweep_after,65535},
                      {minor_gcs,35}]},
 {suspending,[]}]

One of the connections in the pool is inspected and is found in the waiting state with two messages in the queue. Connection is waiting on ssl handshake to complete

23> process_info(<0.663122.0>).
[{current_function,{gen,do_call,4}},
 {initial_call,{proc_lib,init_p,5}},
 {status,waiting},
 {message_queue_len,2},
 {links,[<0.1570.0>]},
 {dictionary,[{'$ancestors',[hackney_conn_sup,hackney_sup,
                             <0.1567.0>]},
              {'$initial_call',{hackney_conn,init,1}}]},
 {trap_exit,true},
 {error_handler,error_handler},
 {priority,normal},
 {group_leader,<0.1566.0>},
 {total_heap_size,3203},
 {heap_size,1598},
 {stack_size,47},
 {reductions,14839},
 {garbage_collection,[{max_heap_size,#{error_logger => true,include_shared_binaries => false,
                                       kill => true,size => 0}},
                      {min_bin_vheap_size,46422},
                      {min_heap_size,233},
                      {fullsweep_after,65535},
                      {minor_gcs,9}]},
 {suspending,[]}]

26> process_info(<0.663122.0>, [current_stacktrace]).
[{current_stacktrace,[{gen,do_call,4,
                           [{file,"gen.erl"},{line,240}]},
                      {gen_statem,call,3,[{file,"gen_statem.erl"},{line,923}]},
                      {ssl_gen_statem,call,2,
                                      [{file,"ssl_gen_statem.erl"},{line,1372}]},
                      {ssl_gen_statem,handshake,2,
                                      [{file,"ssl_gen_statem.erl"},{line,253}]},
                      {ssl_gen_statem,connect,8,
                                      [{file,"ssl_gen_statem.erl"},{line,222}]},
                      {ssl,connect,3,[{file,"ssl.erl"},{line,637}]},
                      {hackney_conn,connected,3,
                                    [{file,"/codebuild/output/src3735438855/src/mirror/_build/default/lib/hackney/src/hackney_conn.erl"},
                                     {line,909}]},
                      {gen_statem,loop_state_callback,11,
                                  [{file,"gen_statem.erl"},{line,1395}]}]}]

At this point in time I am not exactly sure what failure occurred to cause the handshake attempt to fail in this way so I can not reliably reproduce this issue. Regardless of the failure scenario I believe a timeout in this location would allow configuration of some safety rails against whatever kind of failures could happen.

Lingua principale
Erlang
Stelle
1.4k
Fork
458
Merge medio
11h 32m
PR unite (30g)
47

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di benoitc/hackney

Tutte le issue di benoitc/hackney

Issue simili

Altre issue su Networking

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.