A <secondary> output cannot roll back a chunk: rollback_write always raises ArgumentError

未关闭 适合新手
  • 哪里不对？
#5,633 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 1 天内回复

还没有人认领这个 Issue。

评估

难度
1/5
预计耗时
1 小时以内
新手友好度
88/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
活跃
技术栈
ruby
领域
backend

调研方向

阅读 lib/fluent/plugin/output.rb 中 acts_as_secondary 附近的内容以及 rollback_write 的定义。报告的修复方法是在委托时将 update_retry 作为关键字参数传递。检查现有的次级输出测试，并添加覆盖，证明延迟提交的 chunk 会被回滚；完成的标准是委托不再引发异常，且可以重试该 chunk。

由索引模型根据 Issue 内容生成。

描述

bug
Describe the bug

When an output is used as <secondary>, Output#acts_as_secondary defines rollback_write on it to delegate to the primary:

define_method(:rollback_write){ |chunk_id, update_retry: true| @primary_instance.rollback_write(chunk_id, update_retry) }

https://github.com/fluent/fluentd/blob/d2c7287a/lib/fluent/plugin/output.rb#L256

Output#rollback_write takes update_retry as a keyword argument (def rollback_write(chunk_id, update_retry: true), line 1178). So this call always raises ArgumentError: wrong number of arguments (given 2, expected 1). rollback_write became a keyword argument in 59a47a19 (v0.14.25 and v1.0.0), and this delegation was not updated.

A delayed commit output, such as out_forward with require_ack_response true, calls rollback_write when the ack does not arrive. When this output is a <secondary>, the call raises, and AckHandler#collect_response logs unexpected error while receiving ack. The chunk is not rolled back, so it is not sent again. In the test below, it stayed in the delayed commit state until the shutdown.

To Reproduce
  1. Make nothing listen on port 24224, so the primary fails.
  2. Run a server on port 24225 that reads the data but never sends an ack:
ruby -rsocket -e 's = TCPServer.new("127.0.0.1", 24225); loop { c = s.accept; Thread.new(c) { |x| loop { x.readpartial(65536) } rescue nil } }'
  1. Run Fluentd with the configuration below for about 20 seconds.

Results on master (d2c7287a):

  • After the primary fails, the secondary sends the chunk. Each time the ack times out, the error below is logged, about every 2 seconds.
  • The chunk is not sent again. The silent server received 1014 bytes in 20 seconds.

With @primary_instance.rollback_write(chunk_id, update_retry: update_retry) instead, the error is gone. The chunk is sent again after each ack timeout, and the silent server received 3096 bytes in the same time.

Expected behavior

rollback_write on a <secondary> output rolls back the chunk, as on the primary. Passing update_retry as a keyword argument fixes it, as above.

Your Environment
- Fluentd version: master (d2c7287a). v1.20.1 and v1.19.4 have the same line (not run).
- Package version: -
- Operating system: Linux
- Kernel version: -
Your Configuration
<source>
  @type sample
  tag test
  size 1
  rate 1
</source>
<match test>
  @type forward
  heartbeat_type none
  require_ack_response true
  <server>
    host 127.0.0.1
    port 24224
  </server>
  <buffer>
    @type memory
    flush_interval 1s
    retry_type periodic
    retry_wait 1s
    retry_timeout 60s
    retry_secondary_threshold 0.05
  </buffer>
  <secondary>
    @type forward
    heartbeat_type none
    require_ack_response true
    ack_response_timeout 2
    <server>
      host 127.0.0.1
      port 24225
    </server>
  </secondary>
</match>
Your Error Log
[error]: unexpected error while receiving ack error_class=ArgumentError error="wrong number of arguments (given 2, expected 1)"
  [error]: lib/fluent/plugin/output.rb:256:in 'block (2 levels) in acts_as_secondary'
Additional context

I found this while reviewing #5627.

主要语言
Ruby
星标
13.6k
派生
1.4k
平均合并
17 小时 39 分钟
30 天内合并 PR
67

环境准备

从这里开始

  1. 先读完整个 Issue，再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库，在一个分支上完成修改。
  4. 提交 Pull Request，并在描述里引用这个 Issue 编号。

fluent/fluentd 的其他 Issue

查看 fluent/fluentd 的全部 Issue

相似的 Issue

更多 Ruby Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。