in_tcp, in_syslog: a non-ASCII delimiter closes the connection with Encoding::CompatibilityError

オープン
  • 何か問題がありますか？
#5,601 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
3/5
見積もり時間
1〜2日
初心者へのやさしさ
68/100
issue の種類
バグ
明瞭さ
おおむね明確
活発さ
活発
技術スタック
ruby
領域
backend

調査の方向性

Read lib/fluent/plugin/in_tcp.rb around lines 126 and 176, then inspect the TCP transport path in lib/fluent/plugin/in_syslog.rb around line 252. Reproduce with the provided delimiter and socket example, and check related tests for TCP and syslog parsing. Done means non-ASCII delimiters either split messages correctly without closing the connection or are rejected in configuration; the issue does not name a specific test file.

索引モデルが issue の本文から書いたものです。

説明

bug
Describe the bug

in_tcp closes the connection when the delimiter contains a non-ASCII character, such as delimiter "§", and the received data contains a non-ASCII byte.

The data read from the socket is ASCII-8BIT. When the connection buffer gets a non-ASCII byte, the buffer becomes ASCII-8BIT too. The delimiter from the configuration is UTF-8, so buf.index(@delimiter, pos) raises Encoding::CompatibilityError. The server helper logs the error and closes the connection.

Data that contains such a delimiter always contains a non-ASCII byte. So a non-ASCII delimiter can never split messages. Data with only ASCII bytes does not raise, but it is not split either.

in_syslog with <transport tcp> has the same problem. It raises at in_syslog.rb:252.

To Reproduce

  1. Run Fluentd with the configuration below.

  2. Send two messages that end with the delimiter:

    ruby -rsocket -e 's = TCPSocket.new("127.0.0.1", 5170); s.write("hello§world§"); puts(IO.select([s], nil, nil, 3) ? "closed by server" : "still open")'

    It prints closed by server, and no record is emitted.

Expected behavior

hello and world are emitted. If non-ASCII delimiters are not supported, the configuration is rejected instead.

Your Environment
- Fluentd version: 1.20.1 (master at 1d6b1129)
- Package version: N/A (from source)
- Operating system: CachyOS
- Kernel version: 7.3.1-1-cachyos
- Ruby: 4.0.9
Your Configuration
<source>
  @type tcp
  tag test
  port 5170
  bind 127.0.0.1
  delimiter "§"
  <parse>
    @type none
  </parse>
</source>

<match test>
  @type stdout
</match>
Your Error Log
2026-11-06 12:31:44 +0900 [error]: unexpected error on reading data host="127.0.0.1" port=51668 error_class=Encoding::CompatibilityError error="incompatible character encodings: BINARY (ASCII-8BIT) and UTF-8"
  2026-11-06 12:31:44 +0900 [error]: lib/fluent/plugin/in_tcp.rb:176:in 'String#index'
  2026-11-06 12:31:44 +0900 [error]: lib/fluent/plugin/in_tcp.rb:176:in 'block in Fluent::Plugin::TcpInput#start'
  2026-11-06 12:31:44 +0900 [error]: lib/fluent/plugin_helper/server.rb:661:in 'on_read_with_connection'
  2026-11-06 12:31:44 +0900 [error]: cool.io-1.11.0/lib/cool.io/io.rb:123:in 'Coolio::IO#on_readable'
Additional context

With <extract>, the same error is raised at in_tcp.rb:126.

del_size is @delimiter.length, which counts characters, not bytes. Matching the delimiter as bytes might fix both. I have not tested it.

主要言語
Ruby
スター
13.6k
フォーク
1.4k
平均マージ
1日 1時間
マージ済み PR（30日）
61

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

fluent/fluentd のほかの issue

fluent/fluentd の issue をすべて見る

似ている issue

Ruby の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。