vectordotdev/vector

Only warn about small files when they are not empty

Open

#1,065 创建于 2019年10月21日

在 GitHub 查看
 (9 评论) (0 反应) (0 负责人)Rust (21,837 star) (2,126 fork)batch import
domain: observabilitygood first issuemeta: feedbacksource: filetype: enhancement

描述

This is a follow up to https://github.com/timberio/vector/pull/863#issuecomment-544314174.

To recap, when the file source is configured to use the checksum fingerprinting strategy a warning log message is emitted letting the user know the file is too small to generate a fingerprint:

warn!(message = "Ignoring file smaller than fingerprint_bytes", file = ?path);

This is a very welcome feature for files that are actually too small. It helps to prevent confusion around why small files aren't being tailed. Unfortunately, it can produce a lot of noise in production environments where many empty files are present.

I'm unsure if we should ignore empty files, given that it could lead to the same confusion that we experienced previously with small files. Then again, I can see the reasoning behind ignoring empty files. I'm opening this up for discussion so we can come to a conclusion about what to do in this scenario.

Update:

New log line for too small files is:

Currently ignoring file too small to fingerprint

贡献者指南

技术栈
rust
领域
observabilitybackend
议题类型
feature
难度面向新贡献者的预计实现难度，1 表示很小改动，5 表示专家级工作。
3
预计时间有经验贡献者完成调查、实现、测试并准备 pull request 的粗略时间范围。
1-3 hours
活动状态议题当前的可参与程度：新鲜、活跃、陈旧、阻塞或等待维护者输入。
fresh
清晰度议题是否清楚说明期望改动、验收标准和下一步。
mostly clear
前置要求
basic Rust knowledgeunderstanding of file source fingerprinting
新手友好度1-100 的估计分数，表示该议题对首次贡献者的友好程度。
40
研究方向
Look at the `file` source implementation in `src/sources/file/` directory, particularly the fingerprinting logic and the warning message about small files. Review the discussion in the linked PR #863 and the subsequent comments. The goal is to decide whether to skip the warning for empty files or adjust the message. Implement a condition to check if the file is empty before emitting the warning, and ensure the behavior aligns with the maintainers' consensus from the issue.
Only warn about small files when they are not empty · vectordotdev/vector#1065 | Good First Issue