in_http: leftover pipelined data is never re-parsed after a request is consumed

オープン
  • 何か問題がありますか？
#12,653 コメント 1 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
3/5
見積もり時間
1〜2日
初心者へのやさしさ
67/100
issue の種類
バグ
明瞭さ
明確に書かれている
活発さ
活発
技術スタック
c
領域
backend

調査の方向性

Start in plugins/in_http/http_conn.c at the MK_HTTP_PARSER_OK handling, then trace how http_conn_event() invokes the parser and how request completion resets the session. Reproduce with two pipelined requests and confirm the second stalls until another read event. Done when buffered requests are parsed and answered immediately; run the relevant HTTP input tests.

索引モデルが issue の本文から書いたものです。

説明

Bug Report

Describe the bug

After a request is consumed, the leftover bytes are moved to the front of the connection buffer and the parser is
re-initialised — but the remaining buffer is never parsed again. A pipelined request therefore sits unprocessed
until more data happens to arrive on the socket. If the client is waiting for the second response, it waits
until the keep-alive timeout.

This is separate from #12652 (which is about the body length being wrong): it reproduces with
Transfer-Encoding: chunked, where the request length is exact and the leftover bytes are correctly preserved.

To Reproduce

One connection, two pipelined chunked requests, then wait:

POST / HTTP/1.1
Host: x
Content-Type: application/json
Transfer-Encoding: chunked

7\r\n{"a":1}\r\n0\r\n\r\n
POST /2 HTTP/1.1
Host: x
Content-Type: application/json
Content-Length: 7

{"b":2}

Result: one response, one record ({"a"=>1}). The second request produces nothing.

Now send a third request on the same connection. The second one is then finally parsed — its record
({"b"=>2}) appears, tagged 2, before the third is handled. So the bytes were buffered correctly all along;
nothing re-ran the parser on them.

Root cause

plugins/in_http/http_conn.c, after MK_HTTP_PARSER_OK:

if (0 < (conn->buf_len - request_len)) {
    memmove(conn->buf_data, &conn->buf_data[request_len],
            conn->buf_len - request_len);
    conn->buf_data[conn->buf_len - request_len] = '\0';
    conn->buf_len -= request_len;
}
else { ... }

/* Reinitialize the parser ... */
memset(&conn->session.parser, 0, sizeof(struct mk_http_parser));
mk_http_parser_init(&conn->session.parser);
http_conn_request_init(&conn->session, &conn->request);

The parser is reset and the leftover data is in the buffer, but the function returns without calling
mk_http_parser() on it. Parsing only happens from http_conn_event(), which runs on a read event.

Expected behavior

After moving the leftover bytes down, the parser should be run again on them (loop, or re-enter the parse) so
pipelined requests are handled immediately.

Your Environment

  • Fluent Bit 5.1.5 (fluent/fluent-bit:latest)
  • Linux, aarch64, official container image
  • Input: http plugin, default settings

Impact

Pre-auth. HTTP pipelining is effectively broken: the second and later requests on a connection are not answered
until unrelated data arrives, and are lost entirely if it never does. Clients that pipeline see stalls up to the
keep-alive timeout.

主要言語
C
スター
8.2k
フォーク
2k
平均マージ
3日 20時間
マージ済み PR（30日）
99

環境構築

Codespaces で開く

このプロジェクトの開発コンテナを、あなたの GitHub アカウントでブラウザ上に起動します。

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

fluent/fluent-bit のほかの issue

fluent/fluent-bit の issue をすべて見る

似ている issue

C の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。