Multiple tmp dirs created but only one used for file-backed dataset store
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 72/100
Research direction
Start by reading the temporary-directory creation in httomo/cli.py around lines 428-438 and the HDF5 filepath definition in httomo/data/dataset_store.py around line 174. Trace how the process rank is used in both places. Done means non-rank-0 processes no longer create unused temporary directories while the rank-0 HDF5 store continues to work.
Written by the indexing model from the issue text.
Description
This only applies to the case when the data processing has the dataset store backed by an hdf5 file.
In such a case, each process creates a temp dir to hold an hdf5 file: https://github.com/DiamondLightSource/httomo/blob/2acd1b2278a337458a00b63e8528117892eba647/httomo/cli.py#L428-L438
but when the writer actually defines the hdf5 filepath, only rank 0's temp dir is used: https://github.com/DiamondLightSource/httomo/blob/2acd1b2278a337458a00b63e8528117892eba647/httomo/data/dataset_store.py#L174
Meaning, each process creates a temp dir when really only rank 0 needs to create one.
This doesn't have any impact on the functionality, it's simply just a bit confusing when seeing multiple temp dirs created and only one is actually being used.
- Dominant language
- Python
- Stars
- 10
- Forks
- 5
- Avg merge
- 5d 17h
- Merged PRs (30d)
- 1
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from DiamondLightSource/httomo
-
ci
Difficulty 1/5 Under an hour Newbie friendliness 88/100
DiamondLightSource/httomo#714 ·
-
documentation question
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
DiamondLightSource/httomo#697 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 72/100
DiamondLightSource/httomo#609 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
DiamondLightSource/httomo#607 ·
-
documentation
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
DiamondLightSource/httomo#585 ·
All issues in DiamondLightSource/httomo
Similar issues
-
documentation help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
simonw/sqlite-utils#872 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100