parser_syslog: the regexp engine for RFC 5424 can match the header later in the message and corrupt the record

未关闭 适合新手
  • 哪里不对？
#5,630 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
2/5
预计耗时
1-3 小时
新手友好度
79/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
活跃
技术栈
ruby
领域
backend

调研方向

从 lib/fluent/plugin/parser_syslog.rb 开始，查看 issue 中链接的 parse_plain 对 Regexp#match(text, idx) 的调用，并追踪 RFC 5424 头部字段的解析过程。检查 RFC 5424 的 parser 测试，以及 regexp 和 string 引擎。当字段在 idx 处不匹配时，regexp 引擎会报告不匹配，且不会丢弃前面的文本或破坏记录，即为完成。

由索引模型根据 Issue 内容生成。

描述

Describe the bug

With message_format rfc5424 and the default parser_engine regexp, the parser matches the header fields after the timestamp with re.match(text, idx). Regexp#match with a position does not anchor the match at that position. When the text at idx does not match, the regexp can match later in the text. The record then gets values from the wrong place, and the text before the match is dropped. No warning is logged.

https://github.com/fluent/fluentd/blob/e7d3c3e4/lib/fluent/plugin/parser_syslog.rb#L269

To Reproduce

Run the script below with bundle exec ruby -Ilib repro.rb in this repository.

Results on master (e7d3c3e4):

  • <14>1 2026-04-25T16:43:29Z hôst app 123 ID - msg: the regexp engine returns "host" => "st". The string engine returns "host" => "hôst".
  • <14>1 2026-04-25T16:43:29Z first line\nhost app 123 ID - second line: the regexp engine returns "host" => "host" and "message" => "second line". first line is dropped.
require 'fluent/test'
require 'fluent/test/driver/parser'
require 'fluent/plugin/parser_syslog'

Fluent::Test.setup

%w[regexp string].each do |engine|
  d = Fluent::Test::Driver::Parser.new(Fluent::Plugin::SyslogParser).configure(%[
    message_format rfc5424
    with_priority true
    parser_engine #{engine}
  ])
  [
    '<14>1 2026-04-25T16:43:29Z hôst app 123 ID - msg',
    "<14>1 2026-04-25T16:43:29Z first line\nhost app 123 ID - second line",
  ].each do |text|
    d.instance.parse(text) { |_time, record| puts "#{engine}: #{record.inspect}" }
  end
end
Expected behavior

The regexp engine does not skip text. If the header fields do not match at that position, the message is reported as not matched. Anchoring the match at idx (for example with \G) would do this.

Your Environment
- Fluentd version: master (e7d3c3e4). v1.20.1 and v1.19.4 have the same code (not run).
- Package version: -
- Operating system: Linux
- Kernel version: -
Your Configuration
<parse>
  @type syslog
  message_format rfc5424
  with_priority true
</parse>
Your Error Log
No error or warning is logged.
Additional context

parse_rfc3164_regex also uses the same parse_plain. I did not check whether RFC 3164 messages are affected.

主要语言
Ruby
星标
13.6k
派生
1.4k
平均合并
17 小时 28 分钟
30 天内合并 PR
66

环境准备

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

fluent/fluentd 的其他 Issue

查看 fluent/fluentd 的全部 Issue

相似的 Issue

更多 Ruby Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。