sharkdp/fd

scatter search targets of threads to optimize search across multiple devices

Open

#559 opened on Apr 6, 2020

View on GitHub
 (9 comments) (0 reactions) (0 assignees)Rust (1,059 forks)batch import
feature-requesthelp wantedperformance

Repository metrics

Stars
 (42,984 stars)
PR merge metrics
 (Avg merge 15d 23h) (16 merged PRs in 30d)

Description

when searching across multiple devices, cpu utilization is rarely the bottleneck but rather io is.

an option to scatter the search targets of threads would speed up search massively when searching in /media/ for example across multiple external harddrives.

they may share a data link but its their latency and responsiveness that is the bottleneck. and can be exercised in parallel.

Contributor guide

Research direction
Examine the current thread pool and file traversal logic to add an option that distributes search targets across threads to parallelize I/O across devices.
Tech stack
rust
Domain
cliperformance
Issue type
Feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
Half day
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
Active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
Clear
Prerequisites
RustGit
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
65