vectordotdev/vector

Allow specifying the object key in the `aws_s3` sink

Open

#6,929 创建于 2021年3月30日

在 GitHub 查看
 (1 评论) (0 反应) (0 负责人)Rust (2,126 fork)batch import
good first issuesink: aws_s3type: enhancement

仓库指标

Star
 (21,837 star)
PR 合并指标
 (平均合并 8天 16小时) (30 天内合并 110 个 PR)

描述

Requested by user migrating from fluentd.

Currently, we expose some configuration knobs for the object prefix + key for the aws_s3 sink:

  filename_append_uuid = true # optional, default
  filename_extension = "log" # optional, default
  filename_time_format = "%s" # optional, default
  key_prefix = "date=%F/" # optional, default

However there is no way to directly set the key. Ideally this could be templated from the event (like key = "{{ host }}"). This would affect internal partitioning of events across S3 requests. It is likely we'll still want to support formatting the timestamp as part of that, as well.

Ref: fluentd's s3 output: https://docs.fluentd.org/output/s3#s-3-_object_key_format

贡献者指南