parser_syslog, in_syslog: text before a PRI on a later line is silently dropped

Đang mở
  • Có gì không ổn?
#5,593 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

@lbarrosby đang làm issue này rồi.

Từ ngày 6/11/2026.

  • #5598 của @lbarrosby — đang mở

Đánh giá

Độ khó
3/5
Thời gian dự kiến
1-2 ngày
Mức phù hợp với người mới
72/100
Loại issue
Lỗi
Độ rõ ràng
Đặc tả rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
ruby
Lĩnh vực
backend

Hướng nghiên cứu

Read lib/fluent/plugin/parser_syslog.rb and lib/fluent/plugin/in_syslog.rb, focusing on the listed PRI and detection regexps and how both parser engines use them. Check the existing parser and input tests for multiline messages, then reproduce the examples with the regexp engine. Done means headers are recognized only at the start of the message, malformed leading text is emitted as unmatched_line with a warning, and auto detection retains an initial RFC3164 header.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

bug
Describe the bug

With the default parser_engine regexp, parser_syslog looks for the PRI with a regexp that starts with ^. ^ matches at the start of any line. When a message contains a newline and its first line is not a syslog header, the parser starts from a later line that looks like one. The text before that line is dropped without a warning.

in_syslog has the same problem in its own SYSLOG_REGEXP. It uses this regexp when the parser does not parse the priority, for example with <parse> @type none.

A message with a newline can reach the parser through in_syslog over UDP, or over TCP with frame_type octet_count. It can also come through filter_parser.

Input Configuration Result
first line\n<14>1 2026-04-25T16:43:29Z host app 123 ID - second line message_format rfc5424 A record from the second line. first line is dropped.
first line\n<14>Apr 25 16:43:29 host app: second line message_format rfc3164 A record from the second line with pri 0, so in_syslog tags it kern.emerg. first line is dropped.
<13>Apr 25 16:43:29 web01 nginx: request failed:\n<14>1 2026-04-25T16:43:29Z host app 123 ID - second line message_format auto Detected as RFC5424 and parsed from the second line. The RFC3164 header with web01 and nginx is lost.
first line\n<14>second line in_syslog with <parse> @type none {"message":"second line"}. first line is dropped.
To Reproduce

  1. Run Fluentd with the configuration below.

  2. Send the datagrams:

    require 'socket'
s = UDPSocket.new
s.send("first line\n<14>1 2026-04-25T16:43:29Z host app 123 ID - second line\n", 0, '127.0.0.1', 5141)
s.send("first line\n<14>Apr 25 16:43:29 host app: second line\n", 0, '127.0.0.1', 5142)
s.send("first line\n<14>second line\n", 0, '127.0.0.1', 5143)
Expected behavior

None of these datagrams starts with a syslog header. Each one is emitted as unmatched_line, and a warning is logged. parser_engine string already does this for the first two. in_syslog with @type none already does this for first line alone.

For message_format auto, a message that starts with an RFC3164 header is detected as RFC3164.

Your Environment
- Fluentd version: 1.20.1 (master at bd36a9ee)
- Package version: N/A (from source)
- Operating system: CachyOS
- Kernel version: 7.3.1-1-cachyos
- Ruby: 4.0.9
Your Configuration
<source>
  @type syslog
  port 5141
  bind 127.0.0.1
  tag rfc5424
  emit_unmatched_lines true
  <parse>
    @type syslog
    message_format rfc5424
  </parse>
</source>

<source>
  @type syslog
  port 5142
  bind 127.0.0.1
  tag rfc3164
  emit_unmatched_lines true
</source>

<source>
  @type syslog
  port 5143
  bind 127.0.0.1
  tag none
  emit_unmatched_lines true
  <parse>
    @type none
  </parse>
</source>

<match **>
  @type stdout
</match>
Your Error Log
No warning or error is logged. The output is:

2026-04-26 01:43:29.000000000 +0900 rfc5424.user.info: {"host":"host","ident":"app","pid":"123","msgid":"ID","extradata":"-","message":"second line"}
2026-04-25 16:43:29.000000000 +0900 rfc3164.kern.emerg: {"host":"host","ident":"app","message":"second line"}
2026-11-05 17:45:19.501588606 +0900 none.user.info: {"message":"second line"}
Additional context

These regexps start with ^:

  • RFC5424_PRI_REGEXP, RFC3164_PRI_REGEXP and REGEXP_DETECT_RFC5424 in lib/fluent/plugin/parser_syslog.rb
  • SYSLOG_REGEXP in lib/fluent/plugin/in_syslog.rb

REGEXP_DETECT_RFC5424 is used by both engines. With parser_engine string, the third input is also detected as RFC5424, and the parser raises TimeParseError.

Using \A would anchor them to the start of the message. Note that a message with a leading empty line would then be unmatched too. The string parser already treats it as unmatched.

SYSLOG_REGEXP has used ^ since in_syslog was added. In parser_syslog, the results changed in v1.10.0. v1.9.3 yields nil, nil for the first and third inputs, because it matched RFC5424 with \A. For the second input, v1.9.3 also parses from the second line, but with pri 14.

#2873 asks to support messages that contain newlines. This issue is only about where the header is looked for.

Ngôn ngữ chính
Ruby
Star
13.6k
Fork
1.4k
Merge trung bình
1 ngày 4 giờ
Pull request đã merge (30 ngày)
56

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của fluent/fluentd

Tất cả issue của fluent/fluentd

Issue tương tự

Thêm issue về Ruby

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.