akka/akka-core

Add support for skipping too long frames in Framing.delimiter

Open

#22,385 创建于 2017年2月23日

在 GitHub 查看
 (8 评论) (0 反应) (0 负责人)Scala (13,277 star) (3,547 fork)batch import
3 - in progresshelp wantedt:stream

描述

In Akka Stream, the Framing object allows to split a stream (e.g. a FileIO) in chunks. More specifically, the Framing.delimiter method splits the stream when encountering a given delimiter.

def delimiter(delimiter: ByteString, maximumFrameLength: Int, allowTruncation: Boolean = false): Flow[ByteString, ByteString, NotUsed]

I've been using it to parse the lines of a file using the \n delimiter. When a line is longer than the maximumFrameLength which is passed as a parameter, the Flow fails the Stream.

It means that as soon as we encounter a line too long, the stream is failed and we have no way to read the remaining of the file.

To circumvent this issue, we could add another parameter to the delimiter method, e.g. skipTooLongFrames which, when set to true, would just make the Flow ignore too long frames and continue to read the stream. The method would look like:

def delimiter(delimiter: ByteString, maximumFrameLength: Int, allowTruncation: Boolean = false, skipTooLongFrames: Boolean = false): Flow[ByteString, ByteString, NotUsed]

Is there another way to deal with this? If not, could we consider adding the parameter? Or is this going against some principles of Akka Stream? (Note that the same question applies for the lengthField method of the same object.)

贡献者指南