in_tcp, in_syslog: a non-ASCII delimiter closes the connection with Encoding::CompatibilityError

Abierto
  • ¿Algo no está bien?
#5,601 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
3/5
Tiempo estimado
1-2 días
Aptitud para principiantes
68/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
ruby
Área
backend

Línea de trabajo

Read lib/fluent/plugin/in_tcp.rb around lines 126 and 176, then inspect the TCP transport path in lib/fluent/plugin/in_syslog.rb around line 252. Reproduce with the provided delimiter and socket example, and check related tests for TCP and syslog parsing. Done means non-ASCII delimiters either split messages correctly without closing the connection or are rejected in configuration; the issue does not name a specific test file.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

bug
Describe the bug

in_tcp closes the connection when the delimiter contains a non-ASCII character, such as delimiter "§", and the received data contains a non-ASCII byte.

The data read from the socket is ASCII-8BIT. When the connection buffer gets a non-ASCII byte, the buffer becomes ASCII-8BIT too. The delimiter from the configuration is UTF-8, so buf.index(@delimiter, pos) raises Encoding::CompatibilityError. The server helper logs the error and closes the connection.

Data that contains such a delimiter always contains a non-ASCII byte. So a non-ASCII delimiter can never split messages. Data with only ASCII bytes does not raise, but it is not split either.

in_syslog with <transport tcp> has the same problem. It raises at in_syslog.rb:252.

To Reproduce

  1. Run Fluentd with the configuration below.

  2. Send two messages that end with the delimiter:

    ruby -rsocket -e 's = TCPSocket.new("127.0.0.1", 5170); s.write("hello§world§"); puts(IO.select([s], nil, nil, 3) ? "closed by server" : "still open")'

    It prints closed by server, and no record is emitted.

Expected behavior

hello and world are emitted. If non-ASCII delimiters are not supported, the configuration is rejected instead.

Your Environment
- Fluentd version: 1.20.1 (master at 1d6b1129)
- Package version: N/A (from source)
- Operating system: CachyOS
- Kernel version: 7.3.1-1-cachyos
- Ruby: 4.0.9
Your Configuration
<source>
  @type tcp
  tag test
  port 5170
  bind 127.0.0.1
  delimiter "§"
  <parse>
    @type none
  </parse>
</source>

<match test>
  @type stdout
</match>
Your Error Log
2026-11-06 12:31:44 +0900 [error]: unexpected error on reading data host="127.0.0.1" port=51668 error_class=Encoding::CompatibilityError error="incompatible character encodings: BINARY (ASCII-8BIT) and UTF-8"
  2026-11-06 12:31:44 +0900 [error]: lib/fluent/plugin/in_tcp.rb:176:in 'String#index'
  2026-11-06 12:31:44 +0900 [error]: lib/fluent/plugin/in_tcp.rb:176:in 'block in Fluent::Plugin::TcpInput#start'
  2026-11-06 12:31:44 +0900 [error]: lib/fluent/plugin_helper/server.rb:661:in 'on_read_with_connection'
  2026-11-06 12:31:44 +0900 [error]: cool.io-1.11.0/lib/cool.io/io.rb:123:in 'Coolio::IO#on_readable'
Additional context

With <extract>, the same error is raised at in_tcp.rb:126.

del_size is @delimiter.length, which counts characters, not bytes. Matching the delimiter as bytes might fix both. I have not tested it.

Lenguaje dominante
Ruby
Estrellas
13.6k
Forks
1.4k
Merge medio
1 d 1 h
PR fusionados (30 d)
61

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de fluent/fluentd

Todos los issues de fluent/fluentd

Issues similares

Más issues de Ruby

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.