akka/akka-http

Merge stream elements into one http chunk

オープン

#1,792 opened on 2018/01/17

 (4 件のコメント) (0 件のリアクション) (0 人の担当者)Scala (598 件のフォーク)batch import
1 - triageddiscusshelp wantedt:http:marshallingt:routingt:stream

Repository metrics

Stars
 (1,311 個のスター)
PR merge metrics
 (平均マージ 1d 10h) (30d で 2 merged PRs)

説明

Streams are great but I got a stream of thousand elements. Unfortunately every element is emitted in one http chunk (and the elements are small). Would be nice if the framework supports "merging" of multiple elements into one chunk and return a valid json array of elements.

My workaround

                      def toJsonBytes(source: Seq[T]): ByteString =
                        ByteString(source.map(t => t.asJson.noSpaces).mkString(","))
                      // emit 100 elements in one http chunk
                      val chunk = t.groupedWithin(100, 100.millis)
                      val fst   = Flow[Seq[T]].map(toJsonBytes)
                      val rst   = Flow[Seq[T]].map(ByteString(",") ++ toJsonBytes(_))
                      val together = chunk.prefixAndTail(1).flatMapConcat {
                        case (head, tail) =>
                          Source(head).via(fst).concat(tail.via(rst))
                      }
                      complete(
                        HttpEntity(
                          ContentTypes.`application/json`,
                          Source.single(ByteString("[")) ++ together ++ Source.single(ByteString("]"))
                        )
                      )

コントリビューターガイド