Add request timeout support to client-side pool APIs

Issue by mpilquist Tuesday Aug 02, 2016 at 14:54 GMT Originally opened as https://github.com/akka/akka/issues/21098

---

Consider the following program, which incorrectly fails to process response entities:

import $ivy.`com.typesafe.akka::akka-http-experimental:2.4.8`

import scala.concurrent.Future

import akka.actor.ActorSystem
import akka.stream.ActorMaterializer
import akka.stream.scaladsl._
import akka.http.scaladsl._
import akka.http.scaladsl.client.RequestBuilding._
import akka.http.scaladsl.server._

implicit val system = ActorSystem()
implicit val materializer = ActorMaterializer()
implicit val ec = system.dispatcher

// Let's create an HTTP server that responds to all requests with a very large response entity
val serverFlow = Route.handlerFlow(Directives.complete("." * 5000000))
Http().bindAndHandle(serverFlow, "0.0.0.0", 5555).flatMap { serverBinding =>
  // Now that server has started, let's make a bunch of requests in succession using the superPool API
  val pool = Http().superPool[Unit]()
  Future.traverse(0 to 20) { i =>
    val response = Source.single(Get("http://localhost:5555") -> ()).
      via(pool).
      runWith(Sink.head)
    response.map { case (tryResponse, _) =>
      // We incorrectly fail to drain the response entity here
      println(s"Got response $i")
      tryResponse
    }
  }
}

Running this program results in:

Got response 2
Got response 0
Got response 4
Got response 1

4 requests run and then, due to the client failing to process response entities, the program hangs forever. This is a bug in this code -- the response entities must be processed. However, that's not what this ticket is about.

The super pool API could be made much safer by supporting some form of (optional?) timeouts on requests. For example, if the fifth request hasn't been assigned a host connector within X seconds, fail the response with a timeout exception indicating the pool was too busy.

In presence of such a feature, the above program would complete - 4 requests would execute like they did in the above output and then 16 requests would fail with timeout exceptions, indicating the connector pool was too busy. This is a much safer default behavior.

More generally, I'd like a programmatic way to recover from such scenarios -- e.g., if many requests are timing out, throw away the super pool and instantiate a new one. I could build this feature on top of the super pool and Future API but I suspect many folks will want similar behavior.

For some background, see this Twitter stream: https://twitter.com/wildsebastian/status/758937740410888192

/cc @rkuhn @ktoso