Support secure (encrypted) handling of failed records in secondary outputs

首先查看 Fluentd 的次级输出缓冲要求、relabel 输出行为、文件输出的 阶段，以及 issue 中提到的 Fluent::Plugin::Formatter 扩展点。issue 未指定文件或测试；只有在项目就设计达成一致，并提供带有安全失败处理的加密失败记录路径、测试和文档后，工作才算完成。

Is your feature request related to a problem? Please describe.

Hello Fluentd maintainers,
thank you very much for your continued work on Fluentd.

I would like to propose a feature request based on a real operational use case related to securely handling failed records when using <secondary> outputs.
The following describes the background and the problem we encountered in production.

In our production environment, Fluentd receives HTTP requests and forwards them to an internal HTTP endpoint.

• When delivery succeeds, logs are already encrypted using AWS KMS and stored according to our security policy.
• When delivery fails, records are handled by a <secondary> output and written to a file.

The problem is that records written via <secondary> are currently stored in plain text, even though they may contain sensitive payloads.
From a security and compliance perspective, we would like to avoid storing failed messages in plain text and instead keep them in encrypted form, similar to how successful records are handled.

To address this, I initially tried to use @type relabel as a secondary output in order to route failed records to a separate label and apply encryption before writing them to a file. However, Fluentd fails to start with the following error:

secondary plugin 'Fluent::Plugin::RelabelOutput' must support buffering, but doesn't

Based on this behavior, my understanding is that:

• <secondary> outputs must be buffered
• relabel is a non-buffered output
• therefore, using relabel as a secondary output is not supported by design

If this understanding is correct, it appears that there is currently no straightforward way to transform or encrypt failed records before they are written by a secondary output.

Describe the solution you'd like

If using relabel in a secondary output is not considered appropriate by design, I would like to propose an alternative approach that seems to align well with Fluentd’s existing philosophy.

The idea is to preserve the current secondary design while changing only how data is written to disk:

• Keep <secondary> as @type file
• Keep all buffering and reliability guarantees unchanged
• Do not modify the record itself
• Encrypt or transform only the payload written to the file, at the formatting stage

More concretely, this would mean:

• Using @type file for the secondary output, as today
• Applying encryption in the <format> stage via a formatter plugin
• Ensuring that only the serialized output is encrypted, not the in-memory record

Conceptually:

<secondary>
  @type file
  path /var/log/fluent/failover

  <buffer tag,time>
    @type file
    path /var/log/fluent/buffer/failover
  </buffer>

  <format>
    @type encrypted_json
    # KMS-based key loading/decryption and encryption settings
  </format>
</secondary>

Describe alternatives you've considered

The main alternative considered was routing failed records through @type relabel and applying encryption in a separate label.
However, this approach conflicts with the current requirement that secondary outputs must be buffered.

Aside from that, I am not aware of any officially recommended pattern to achieve the following goal:

“Store logs only when delivery fails, and store them securely in encrypted form rather than plain text.”

If there are other recommended Fluentd configuration patterns (for example, using filters, labels, multiple match rules, or existing plugins) that can achieve the same outcome, I would greatly appreciate guidance.

Additionally, if the formatter-based approach described above aligns with the project direction, I would be interested in contributing back in the following ways:

• Implementing an encryption formatter based on Fluent::Plugin::Formatter
• Encrypting only the JSON payload written for failed records
• Using AWS KMS for key loading/decryption (as this is already used for successful records in our environment)
• Ensuring safe fallback behavior that never leaks plaintext on encryption failure
• Adding documentation and examples focused on securely handling failed records with secondary outputs
• Improving guidance around why relabel cannot be used in secondary outputs and what alternative patterns are recommended

I would appreciate any feedback on whether this direction is aligned with Fluentd’s design philosophy, whether it could be considered a recommended pattern, or whether similar ideas have been discussed and rejected in the past.

Thank you very much for your time and consideration.

Additional context

No response