Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

fswatcher Should Dedup Writes or Detect "CloseWrites"

Open
  • Is something wrong?
#156 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Stale
Tech stack
go
Domain
operating-systems

Research direction

Start by reviewing the fsnotify.Write comment, fsnotify.go lines 211-219, and the dedup example in cmd/fsnotify/dedup.go, then check the upstream release status referenced in the issue. The work is complete when the watcher can use the available close-write support on Linux, or an agreed fallback, without producing duplicate onChange events for one file write.

Written by the indexing model from the issue text.

Description

Following up from #149 - it's really a bit of rabbit hole if you look at https://github.com/fsnotify/fsnotify/issues/635 and the numerous other related issues.

In short, the comment for the fsnotify.Write event describes the issue:

	// The pathname was written to; this does *not* mean the write has finished,
	// and a write can be followed by more writes.

So what I've observed when using os.WriteFile on Linux systems + K8s PVCs is that initially fswatcher processes a fsnotify.Write event where the file was rewritten / has size of 0 bytes. It then receives a subsequent fsnotify.Write and the file is finished being written with a non-zero size. The problem is both the 0 bytes and the non-zero bytes events change the tracked hash, causing two unnecessary onChange events in the case where the file did not actually change, and one extra onChange event when the file did change. I've had to add a defense in my use of this utility to ignore onChange if the size of the file is 0 for now (which still isn't great because of the second onChange). For example from my logs:

Config file change detected. Event=WRITE Name=/data/peers.json Size=0 Hash=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Config file change detected. Event=WRITE Name=/data/peers.json Size=3687 Hash=2926b16fe9c31543f05e269eff2bd7d6542050923c348b1e13c6110b3c667e71

There's examples of how we could go to the trouble deduping the writes: https://github.com/fsnotify/fsnotify/blob/main/cmd/fsnotify/dedup.go
But that feels a bit convoluted if we can avoid it (sorry Windows).

Instead it looks like the maintainer is actively working on support non-portable events for Linux systems like CLOSE_WRITE: https://github.com/fsnotify/fsnotify/blob/main/fsnotify.go#L211-L219

These features are still in main of the lib / unreleased, but once they are we can detect if such events are supported and change what events we listen for as a result - make Linux optimized for detecting file changes which is the main OS targetted for production use cases of this library since it is used primarily in containerized applications.

This issue to track the fsnotify upstream for the more efficient event types and fix this once they are released. If too much time passes, we can consider maintaining an fsnotify fork.

Dominant language
Go
Stars
5
Forks
16
Avg merge
3d 9h
Merged PRs (30d)
3

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from hyperledger-firefly/common

All issues in hyperledger-firefly/common

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.