opt: allgather extents on laminate
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- c
- Domain
- distributed-systems, performance
Research direction
Start with the laminate call and extent-gathering behavior described in the issue. Trace where extents are collected and determine how local caching, a tree-based gather, and merging during the gather would fit. Done means the design is agreed, the implementation is tested, and the proposed gathering performance improvement is measured.
Written by the indexing model from the issue text.
Description
For apps that don't actually read anything until after a laminate, we can cache extents locally on each server until the laminate call. Then during the laminate, we could execute an allgather, for example, using a gather to root followed by a bcast. This could improve performance in gathering in two ways:
- we can gather through a tree rather than using an N-to-1 pattern where each server sends directly to the root.
- we could also merge extents as they travel up the tree rather than simply collecting extents to all be merged as the server.
- Dominant language
- C
- Stars
- 122
- Forks
- 34
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from llnl/UnifyFS
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
-
build system
Difficulty 3/5 1-2 days Newbie friendliness 35/100
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
Similar issues
-
level/task module/gcp type/bug
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
-
Difficulty 1/5 Under an hour Newbie friendliness 86/100
hapostgres/pg_auto_failover#1190 ·
-
docs
Difficulty 1/5 Under an hour Newbie friendliness 85/100
-
P3 sonic-vpp
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
sonic-net/sonic-buildimage#29662 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 94/100
spack/spack-packages#6586 ·