in_tcp: a parser error drops the other lines in the same read and closes the connection

Aperta
  • Qualcosa non va?
#5,629 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
3/5
Tempo stimato
1-2 giorni
Idoneità per principianti
64/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
ruby
Ambito
backend, networking

Direzione di ricerca

Start in lib/fluent/plugin/in_tcp.rb at the parser calls around lines 144 and 195, then read how warn_pattern_not_matched handles invalid lines. Reproduce with the three-line TCP write from the issue and the supplied JSON parser configuration. Done means a bad line is warned about and dropped while valid lines before and after it are emitted and the connection stays open.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug
Describe the bug

in_tcp does not rescue errors from @parser.parse(msg) for each line. When the parser raises, the error goes up to the server helper. The helper logs unexpected error on reading data and closes the connection.

  • With batch emit (the default), the lines already parsed in the same read are not emitted. The lines after the bad line are lost too.
  • With single emit (<extract> with tag_key), the lines before the bad line are emitted. The lines after it are lost.

A parser can raise on a normal input. For example, the json parser with time_format raises ParserError when the time field does not match the format. So one bad line from a client can drop its valid lines.

https://github.com/fluent/fluentd/blob/e7d3c3e4/lib/fluent/plugin/in_tcp.rb#L144
https://github.com/fluent/fluentd/blob/e7d3c3e4/lib/fluent/plugin/in_tcp.rb#L195

To Reproduce

Run Fluentd with the configuration below. Then send these three lines in one write:

{"k":"1"}
{"time":"bad","k":"2"}
{"k":"3"}

ruby -rsocket -e 'TCPSocket.open("127.0.0.1", 5170) { |s| s.write(%Q({"k":"1"}\n{"time":"bad","k":"2"}\n{"k":"3"}\n)); sleep 1 }'

Results on master (e7d3c3e4):

  • Batch emit: no event is emitted. {"k":"1"} and {"k":"3"} are lost. The server closes the connection.
  • Single emit (add <extract> with tag_key tag): {"k":"1"} is emitted. {"k":"3"} is lost. The server closes the connection.
Expected behavior

Only the bad line is dropped, with a warning. This is how in_tcp handles a line that does not match the pattern (pattern not matched). The other lines are emitted, and the connection stays open.

Your Environment
- Fluentd version: master (e7d3c3e4). v1.20.1 and v1.19.4 have the same code (not run).
- Package version: -
- Operating system: Linux
- Kernel version: -
Your Configuration
<source>
  @type tcp
  port 5170
  bind 127.0.0.1
  tag test
  <parse>
    @type json
    time_key time
    time_format %Y-%m-%dT%H:%M:%S
  </parse>
</source>
<match test>
  @type stdout
</match>
Your Error Log
[error]: unexpected error on reading data host="127.0.0.1" port=39444 error_class=Fluent::Plugin::Parser::ParserError error="invalid time format: value = bad, error_class = ArgumentError, error = string doesn't match"
  [error]: lib/fluent/plugin/parser.rb:195:in 'Fluent::Plugin::Parser#parse_time'
  [error]: lib/fluent/plugin/parser_json.rb:70:in 'Fluent::Plugin::JSONParser#parse'
  [error]: lib/fluent/plugin/in_tcp.rb:195:in 'block in Fluent::Plugin::TcpInput#start'
  [error]: lib/fluent/plugin_helper/server.rb:661:in 'on_read_with_connection'

No test: {...} line is printed with batch emit.

Additional context

A possible fix is to rescue the error for each line. Then log a warning in the same way as warn_pattern_not_matched (truncated as in #5592), and go on to the next line.

Lingua principale
Ruby
Stelle
13.6k
Fork
1.4k
Merge medio
17h 28m
PR unite (30g)
66

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di fluent/fluentd

Tutte le issue di fluent/fluentd

Issue simili

Altre issue su Ruby

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.