Multiprocessing for dataset download
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 25/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- python
- Domain
- data-engineering, tooling
Research direction
Start by locating the dataset download and image synchronization entry points in the repository, then read how the current sequential workflow handles failures and completion. Run the existing download workflow before changing it; done means downloads and synchronization can proceed in parallel without losing or corrupting images.
Written by the indexing model from the issue text.
Description
In theory, nothing prevents us from downloading + synchronizing the images in parallel.
- Dominant language
- Python
- Stars
- 17
- Forks
- 8
- PR merge metrics
- No merged PRs in 30d
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from BlueBrain/Atlas-Download-Tools
-
Difficulty 2/5 1-3 hours Newbie friendliness 45/100
-
Some RMA queries never return all resultsMay be free again @Stannislav claimed this 1754 days ago, and no pull request is open. Open
BlueBrain/Atlas-Download-Tools#98 · 1 assignee ·
-
Add the "downloads faithful dataset" CLI commandMay be free again @Stannislav claimed this 1790 days ago, and no pull request is open. Open
BlueBrain/Atlas-Download-Tools#95 · 1 assignee ·
-
Difficulty 3/5 1-2 days Newbie friendliness 48/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 48/100
BlueBrain/Atlas-Download-Tools#93 · 1 reaction ·
All issues in BlueBrain/Atlas-Download-Tools
Similar issues
-
customer-reported
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Azure/azure-cli#34150 · 1 comment ·
Maintainers usually reply within 1 day
-
community-request
Difficulty 1/5 Under an hour Newbie friendliness 95/100
NVIDIA-NeMo/Curator#2464 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
WeblateOrg/translation-finder#1099 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
trezor/trezor-firmware#7997 ·
Maintainers usually reply within 2 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Maintainers usually reply within 1 day