akka/akka-http

Merge stream elements into one http chunk

開放

#1,792 建立於 2018年1月17日

 (4 則留言) (0 個反應) (0 位負責人)Scala (598 個分叉)batch import
1 - triageddiscusshelp wantedt:http:marshallingt:routingt:stream

倉庫指標

星標
 (1,311 顆星)
PR 合併指標
 (平均合併 1天 10小時) (30 天內合併 2 個 PR)

描述

Streams are great but I got a stream of thousand elements. Unfortunately every element is emitted in one http chunk (and the elements are small). Would be nice if the framework supports "merging" of multiple elements into one chunk and return a valid json array of elements.

My workaround

                      def toJsonBytes(source: Seq[T]): ByteString =
                        ByteString(source.map(t => t.asJson.noSpaces).mkString(","))
                      // emit 100 elements in one http chunk
                      val chunk = t.groupedWithin(100, 100.millis)
                      val fst   = Flow[Seq[T]].map(toJsonBytes)
                      val rst   = Flow[Seq[T]].map(ByteString(",") ++ toJsonBytes(_))
                      val together = chunk.prefixAndTail(1).flatMapConcat {
                        case (head, tail) =>
                          Source(head).via(fst).concat(tail.via(rst))
                      }
                      complete(
                        HttpEntity(
                          ContentTypes.`application/json`,
                          Source.single(ByteString("[")) ++ together ++ Source.single(ByteString("]"))
                        )
                      )

貢獻者指南