bigscience-workshop/petals

Investigate QUIC (v1) reliability

Open

#208 aperta il 12 gen 2023

Vedi su GitHub
 (3 commenti) (0 reazioni) (1 assegnatario)Python (424 fork)batch import
help wanted

Metriche repository

Star
 (8248 star)
Metriche merge PR
 (Nessuna PR mergiata in 30 g)

Descrizione

Our network layer supports quic like this: hivemind.DHT(..., host_maddrs=['/ip4/1.2.3.4/udp/1337/quic']) However, petals servers currently default to TCP-only host maddrs, unless user specifies --host_maddrs.

In other hivemind-based experiments, we found that QUIC is superior to TCP when operating under household NAT because udp hole punching is more reliable than tcp hole punching. It would be great if we could enable it by default.

The reason why QUIC is in default maddrs is that we haven't tested it thoroughly enough and we fear that it might cause throughput issues.

Quest: try running a quic-only peer in the public swarm, bombard it with requests from (your pc, colab, some publicly accessible machine), check if it works alright.

Criteria (suggestion):

  • cycles per second, forward and inference (vs TCP)
  • retries / relay fallbacks (vs TCP)

We should check for cases where QUIC makes the system unusable (10x slow or does not work at all).

If some cases are slower by tens of %%, this is fine. If there are cases where quic is 2x slower or similar, we can check that running a server with both tcp and quic is still as fast as tcp-only - and if so, it is fine to enable quic in main.

Guida contributor