wb_download.py: use HTTPS, add a timeout and raise_for_status()

Open Beginner friendly
#608 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
1/5
Estimated time
1-3 hours
Newbie friendliness
88/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Quiet
Tech stack
python
Domain
api

Research direction

Open lectures/_static/lecture_specific/pandas/wb_download.py and inspect the fetch at lines 6-9. Update the request as described, then manually check that the HTTPS World Bank endpoint returns the expected workbook, since this static asset is not exercised by CI. Done means failures surface clearly and the workbook still supports the downstream read.

Written by the indexing model from the issue text.

Description

lectures/_static/lecture_specific/pandas/wb_download.py fetches the World Bank indicator over plain HTTP and writes the response body to disk without checking whether the request succeeded. Raised by Copilot while reviewing the downstream port at QuantEcon/lecture-python-programming.ml#6; it is upstream code, so it belongs here rather than in a translation edition.

Current lines 6-9:

wb_data_query = "http://api.worldbank.org/v2/en/indicator/gc.dod.totl.gd.zs?downloadformat=excel"
r = requests.get(wb_data_query)
with open('gd.xls', 'wb') as output:
    output.write(r.content)

Three points, in rough order of how much they matter:

No error check. On any non-200 response the error page body is written to gd.xls, and the failure only surfaces further down as an opaque pd.read_excel parse error rather than as the network problem it actually is. r.raise_for_status() turns that into a clear failure at the point of the fault.

Plain HTTP. api.worldbank.org serves HTTPS, so this is a free upgrade — and it is a file the lectures hold up as example code, which is the argument for getting it right beyond the MITM exposure itself.

No timeout. requests.get without timeout blocks indefinitely if the endpoint stops responding, which is an unpleasant failure mode inside a notebook build.

Suggested:

wb_data_query = "https://api.worldbank.org/v2/en/indicator/gc.dod.totl.gd.zs?downloadformat=excel"
r = requests.get(wb_data_query, timeout=30)
r.raise_for_status()
with open('gd.xls', 'wb') as output:
    output.write(r.content)

Note this script is a static asset rather than an executed cell, so nothing in CI exercises it — worth a manual check that the HTTPS endpoint returns the same workbook before merging.

Dominant language
JavaScript
Stars
72
Forks
31
Avg merge
2d 20h
Merged PRs (30d)
8

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from QuantEcon/lecture-python-programming

All issues in QuantEcon/lecture-python-programming

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.