Make script for OSM data update, tools export, viz rebuild pipelines
@MoritzStefaner is already working on this.
Since Sep 16, 2026.
Assessment
This issue has not been assessed yet.
Description
Given cities.csv with columns cityid, name_en, country_en, nominatim_query (needs manual check), country_code, population:
(⚠️ are online steps that can timeout)
OSM data update pipeline pipeline_osm_data_update
- ⚠️ Get city boundaries via osmnx (take care of special cases like Copenhagen)
- Add centroid lat/lon columns to cities.csv
- ⚠️ Add wikidata_qid column to cities.csv
- ⚠️ curl all country pbfs from cities.csv from geofabrik, filenames: removing date + taking care of special country name mappings
- Create city pbfs, street_networks, bike_networks, growable_networks, streetbike_networks
- ⚠️ Fetch rail stations, schools via osmnx
- Recalculate meta/rec_table.csv and meta/network_metrics.csv
- Manual sanity checkpoint
- Update osm_data_date in meta/versions.csv
- ⚠️ Push new data version to zenodo
Tools export pipeline pipeline_tools_export
- Paralellizable: gbn offline export (~2 days)
- Paralellizable: fbn offline export (~1 day)
- Paralellizable: lbn offline export (~2 days)
- ⚠️ Paralellizable: sb online export (~3 days with 1000 sleep) - crashes & timeouts possible!
- Manual sanity checkpoint
- Update versions in meta/versions.csv
- ⚠️ Push new data versions to zenodo
- ⚠️ Push new data to server
Viz rebuild pipeline pipeline_viz_rebuild
To fill in with @MoritzStefaner whenever finalized
- Dominant language
- Jupyter Notebook
- Stars
- 2
- Forks
- 0
- Avg merge
- 9m
- Merged PRs (30d)
- 16
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from BikeNetKit/dataexport
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
BikeNetKit/dataexport#72 · 2 assignees ·
-
BikeNetKit/dataexport#69 · 3 comments · 1 assignee ·
-
Difficulty 3/5 1-2 days Newbie friendliness 35/100
BikeNetKit/dataexport#59 ·
-
BikeNetKit/dataexport#55 · 1 reaction · 2 assignees ·
-
BikeNetKit/dataexport#54 · 1 comment · 1 assignee ·