A <secondary> output cannot roll back a chunk: rollback_write always raises ArgumentError

Abierto Apto para principiantes
  • ¿Algo no está bien?
#5,633 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
1/5
Tiempo estimado
Menos de una hora
Aptitud para principiantes
88/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
ruby
Área
backend

Línea de trabajo

Lee en lib/fluent/plugin/output.rb la parte alrededor de acts_as_secondary y la definición de rollback_write. La solución reportada consiste en pasar update_retry como argumento con nombre en la delegación. Revisa las pruebas existentes para las salidas secundarias y añade cobertura que muestre que se revierte un chunk con commit retrasado; la tarea estará terminada cuando la delegación ya no genere una excepción y se pueda reintentar el chunk.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

bug
Describe the bug

When an output is used as <secondary>, Output#acts_as_secondary defines rollback_write on it to delegate to the primary:

define_method(:rollback_write){ |chunk_id, update_retry: true| @primary_instance.rollback_write(chunk_id, update_retry) }

https://github.com/fluent/fluentd/blob/d2c7287a/lib/fluent/plugin/output.rb#L256

Output#rollback_write takes update_retry as a keyword argument (def rollback_write(chunk_id, update_retry: true), line 1178). So this call always raises ArgumentError: wrong number of arguments (given 2, expected 1). rollback_write became a keyword argument in 59a47a19 (v0.14.25 and v1.0.0), and this delegation was not updated.

A delayed commit output, such as out_forward with require_ack_response true, calls rollback_write when the ack does not arrive. When this output is a <secondary>, the call raises, and AckHandler#collect_response logs unexpected error while receiving ack. The chunk is not rolled back, so it is not sent again. In the test below, it stayed in the delayed commit state until the shutdown.

To Reproduce
  1. Make nothing listen on port 24224, so the primary fails.
  2. Run a server on port 24225 that reads the data but never sends an ack:
ruby -rsocket -e 's = TCPServer.new("127.0.0.1", 24225); loop { c = s.accept; Thread.new(c) { |x| loop { x.readpartial(65536) } rescue nil } }'
  1. Run Fluentd with the configuration below for about 20 seconds.

Results on master (d2c7287a):

  • After the primary fails, the secondary sends the chunk. Each time the ack times out, the error below is logged, about every 2 seconds.
  • The chunk is not sent again. The silent server received 1014 bytes in 20 seconds.

With @primary_instance.rollback_write(chunk_id, update_retry: update_retry) instead, the error is gone. The chunk is sent again after each ack timeout, and the silent server received 3096 bytes in the same time.

Expected behavior

rollback_write on a <secondary> output rolls back the chunk, as on the primary. Passing update_retry as a keyword argument fixes it, as above.

Your Environment
- Fluentd version: master (d2c7287a). v1.20.1 and v1.19.4 have the same line (not run).
- Package version: -
- Operating system: Linux
- Kernel version: -
Your Configuration
<source>
  @type sample
  tag test
  size 1
  rate 1
</source>
<match test>
  @type forward
  heartbeat_type none
  require_ack_response true
  <server>
    host 127.0.0.1
    port 24224
  </server>
  <buffer>
    @type memory
    flush_interval 1s
    retry_type periodic
    retry_wait 1s
    retry_timeout 60s
    retry_secondary_threshold 0.05
  </buffer>
  <secondary>
    @type forward
    heartbeat_type none
    require_ack_response true
    ack_response_timeout 2
    <server>
      host 127.0.0.1
      port 24225
    </server>
  </secondary>
</match>
Your Error Log
[error]: unexpected error while receiving ack error_class=ArgumentError error="wrong number of arguments (given 2, expected 1)"
  [error]: lib/fluent/plugin/output.rb:256:in 'block (2 levels) in acts_as_secondary'
Additional context

I found this while reviewing #5627.

Lenguaje dominante
Ruby
Estrellas
13.6k
Forks
1.4k
Merge medio
17 h 39 min
PR fusionados (30 d)
67

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de fluent/fluentd

Todos los issues de fluent/fluentd

Issues similares

Más issues de Ruby

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.