Design and refactoring / general code quality

This is intended as a tracking issue for general code quality and design discussions. @sharkdp feel free to close this when you think quality has reached an overall satisfactory level.

To start, I'll quote a recent comment by @sharkdp from a discussion on a PR thread:

There are many things I have in mind that would hopefully help to improve the code quality:

• Group related functions into impl blocks of appropriate structs such that we don't have to pass around so many parameters.
• Think about a unified interface for filters (file type, file size, modification data, file extension, ..). Maybe a trait-based interface where the filters would be stored in a Vec<Box<dyn Filter>>.
• Extract some of the functionality like the OsStr-based path handling into separate libraries / crates.