vectordotdev/vector

Time formats in sinks should be derived from events

Open

#1,743 opened on 2020年2月7日

GitHub で見る
 (4 comments) (0 reactions) (0 assignees)Rust (21,837 stars) (2,126 forks)batch import
domain: templatinggood first issuehave: shouldmeta: feedbacksink: aws_s3type: enhancement

説明

It was unclear for me which timestamp is used for making filename. It is not documented anywhere.

My first expectation was to get there timestamp of last event, or maybe timestamp of first event.

But current timestamp is least useful in my point of view — there is a file metadata with creation time. And it doesn't name files as expected when historical data is being processed.

Seems this idea is not new for you https://github.com/timberio/vector/blob/6f290d3e55d78438100b1bd31747c6c4b1630184/src/sinks/aws_s3.rs#L318 but I couldn't find a ticket to track that.

コントリビューターガイド

技術スタック
rust
領域
backend
Issue 種別
feature
難度新しい貢献者にとっての実装難度です。1 は非常に小さな変更、5 は専門的な作業です。
3
推定時間経験ある貢献者が調査、実装、テスト、pull request 準備にかけるおおよその時間範囲です。
1-3 hours
活動状況Issue が今どれだけ取り組みやすいかを示します。新しい、活発、古い、ブロック中、またはメンテナー入力待ちなどです。
stale
明確さ期待される変更、受け入れ条件、次の手順がどれだけ明確かを示します。
mostly clear
前提条件
Rust programmingUnderstanding of Vector sinks
初心者向け度初回貢献者にどれだけ取り組みやすいかを 1-100 で推定したスコアです。
30
調査方針
The issue suggests modifying the AWS S3 sink to derive the timestamp used in filename from the event's timestamp rather than the file creation time. The relevant code is in src/sinks/aws s3.rs line 318, which contains a comment hinting at this future work. To implement, one must understand how events carry timestamps in Vector's data model and how the sink currently obtains the timestamp. Research should focus on exploring the event timestamp availability and modifying the sink to use it, while ensuring backward compatibility.