in_http: leftover pipelined data is never re-parsed after a request is consumed

Aperta
  • Qualcosa non va?
#12,653 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
3/5
Tempo stimato
1-2 giorni
Idoneità per principianti
67/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
c
Ambito
backend

Direzione di ricerca

Start in plugins/in_http/http_conn.c at the MK_HTTP_PARSER_OK handling, then trace how http_conn_event() invokes the parser and how request completion resets the session. Reproduce with two pipelined requests and confirm the second stalls until another read event. Done when buffered requests are parsed and answered immediately; run the relevant HTTP input tests.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Bug Report

Describe the bug

After a request is consumed, the leftover bytes are moved to the front of the connection buffer and the parser is
re-initialised — but the remaining buffer is never parsed again. A pipelined request therefore sits unprocessed
until more data happens to arrive on the socket. If the client is waiting for the second response, it waits
until the keep-alive timeout.

This is separate from #12652 (which is about the body length being wrong): it reproduces with
Transfer-Encoding: chunked, where the request length is exact and the leftover bytes are correctly preserved.

To Reproduce

One connection, two pipelined chunked requests, then wait:

POST / HTTP/1.1
Host: x
Content-Type: application/json
Transfer-Encoding: chunked

7\r\n{"a":1}\r\n0\r\n\r\n
POST /2 HTTP/1.1
Host: x
Content-Type: application/json
Content-Length: 7

{"b":2}

Result: one response, one record ({"a"=>1}). The second request produces nothing.

Now send a third request on the same connection. The second one is then finally parsed — its record
({"b"=>2}) appears, tagged 2, before the third is handled. So the bytes were buffered correctly all along;
nothing re-ran the parser on them.

Root cause

plugins/in_http/http_conn.c, after MK_HTTP_PARSER_OK:

if (0 < (conn->buf_len - request_len)) {
    memmove(conn->buf_data, &conn->buf_data[request_len],
            conn->buf_len - request_len);
    conn->buf_data[conn->buf_len - request_len] = '\0';
    conn->buf_len -= request_len;
}
else { ... }

/* Reinitialize the parser ... */
memset(&conn->session.parser, 0, sizeof(struct mk_http_parser));
mk_http_parser_init(&conn->session.parser);
http_conn_request_init(&conn->session, &conn->request);

The parser is reset and the leftover data is in the buffer, but the function returns without calling
mk_http_parser() on it. Parsing only happens from http_conn_event(), which runs on a read event.

Expected behavior

After moving the leftover bytes down, the parser should be run again on them (loop, or re-enter the parse) so
pipelined requests are handled immediately.

Your Environment

  • Fluent Bit 5.1.5 (fluent/fluent-bit:latest)
  • Linux, aarch64, official container image
  • Input: http plugin, default settings

Impact

Pre-auth. HTTP pipelining is effectively broken: the second and later requests on a connection are not answered
until unrelated data arrives, and are lost entirely if it never does. Clients that pipeline see stalls up to the
keep-alive timeout.

Lingua principale
C
Stelle
8.2k
Fork
2k
Merge medio
3g 20h
PR unite (30g)
99

Preparare l'ambiente

Apri in Codespaces

Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di fluent/fluent-bit

Tutte le issue di fluent/fluent-bit

Issue simili

Altre issue su C

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.