Make Europeana fetch viable
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 42/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Stale
- Tech stack
- python
- Domain
- data-engineering
Research direction
Start with the Europeana section linked from sources.md and trace the existing Europeana fetch entry point and query-counting flow. Define how rights-only counts and a first-run quarterly plan should cover 30 runs over 30 days, then verify that the resulting schedule avoids fetching the full dataset at once.
Written by the indexing model from the issue text.
Description
Problem
The Europeana data set is too big to fetch all at once, or in its entirety.
- data providers: 4206
- rights: 64
Assuming approximately 1 second per query, the expected duration is approximately 75 hours.
Description
- Add counts for rights only (not including data providers)
- Write a query plan on first run per quarter to map out which queries to make over 30 runs (30 days)
Alternatives
Parallel queries (probably considered impolite, Timid Robot will ask)
Additional context
Implementation
- I would be interested in implementing this feature.
- Dominant language
- Python
- Stars
- 52
- Forks
- 75
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 3
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from creativecommons/quantifying
-
Link latest report from repository READMEPossibly taken @Priyankm23 claimed this 33 days ago. Open✨ goal: improvement 🏁 status: ready for work 💻 aspect: code 📖 aspect: docs good first issue help wanted 🟨 priority: medium
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
creativecommons/quantifying#311 · 3 comments ·
-
GCS free full web search ending 2026-12-31May be free again @TimidRobot claimed this 190 days ago, and no pull request is open. Open🏁 status: ready for work 💻 aspect: code 🔒 staff only 🛠 goal: fix 🟧 priority: high
creativecommons/quantifying#293 · 3 comments · 1 assignee ·
-
Consolidate completion checkPossibly taken @iamshravanijadhav claimed this 35 days ago. Open✨ goal: improvement 🏁 status: ready for work 💻 aspect: code help wanted 🟩 priority: low
Difficulty 4/5 3-5 days Newbie friendliness 38/100
creativecommons/quantifying#288 · 1 comment ·
-
Add a stacked horizontal bar plotMay be free again @oree-xx claimed this 287 days ago, and no pull request is open. Open✨ goal: improvement 💻 aspect: code 🚦 status: awaiting triage 🟩 priority: low
creativecommons/quantifying#274 · 1 assignee ·
-
Add Zenodo as Data Source for Commons QuantificationMay be free again A pull request for this issue was closed without being merged. Open✨ goal: improvement 🏁 status: ready for work 💻 aspect: code help wanted 🟩 priority: low
Difficulty 5/5 Over a week Newbie friendliness 35/100
creativecommons/quantifying#249 ·
All issues in creativecommons/quantifying
Similar issues
-
namespace
Difficulty 1/5 Under an hour Newbie friendliness 68/100
EclipseFdn/open-vsx.org#15604 ·
Maintainers usually reply within 1 day
-
app: scripts severity: low status: backlog type: bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
netbox-community/netbox-scripts#164 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Maintainers usually reply within 5 days
-
Public function to read participants.tsvPossibly taken @bruAristimunha claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Difficulty 1/5 1-3 hours Newbie friendliness 85/100
huggingface/tau#799 ·
Maintainers usually reply within 2 days