vectordotdev/vector

Only warn about small files when they are not empty

開放

#1,065 建立於 2019年10月21日

 (10 則留言) (0 個反應) (0 位負責人)Rust (2,126 個分叉)batch import
domain: observabilitygood first issuemeta: feedbacksource: file

倉庫指標

星標
 (21,837 顆星)
PR 合併指標
 (PR 指標待抓取)

描述

This is a follow up to https://github.com/timberio/vector/pull/863#issuecomment-544314174.

To recap, when the file source is configured to use the checksum fingerprinting strategy a warning log message is emitted letting the user know the file is too small to generate a fingerprint:

warn!(message = "Ignoring file smaller than fingerprint_bytes", file = ?path);

This is a very welcome feature for files that are actually too small. It helps to prevent confusion around why small files aren't being tailed. Unfortunately, it can produce a lot of noise in production environments where many empty files are present.

I'm unsure if we should ignore empty files, given that it could lead to the same confusion that we experienced previously with small files. Then again, I can see the reasoning behind ignoring empty files. I'm opening this up for discussion so we can come to a conclusion about what to do in this scenario.

Update:

New log line for too small files is:

Currently ignoring file too small to fingerprint

貢獻者指南