inequality's data.ipynb raises under the pinned pandas 3, and nothing runs it to notice

Open Beginner friendly
#835 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
1/5
Estimated time
Under an hour
Newbie friendliness
92/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Quiet
Tech stack
jupyter-notebook, numpy, pandas, python
Domain
documentation

Research direction

Open lectures/_static/lecture_specific/inequality/data.ipynb and compare its line 88 with the already-fixed code at inequality.md:288. Run the notebook under the pinned environment and confirm that the shuffle completes without the read-only array exception; done means the served notebook executes past this step.

Written by the indexing model from the issue text.

Description

lectures/_static/lecture_specific/inequality/data.ipynb cannot run under this repo's own pinned environment. It is linked from inequality.md:618 as "This notebook can be used to compute this information over the full dataset", and it is served — so a reader who follows that link and runs it gets an exception.

The defect

Line 88 is y = np.asarray(y), and line 90 is rd.shuffle(y). Under pandas 3's copy-on-write default, np.asarray() of a Series returns a read-only view, so the shuffle raises. Reproduced against anaconda=2026.07 (pandas 3.0.5, numpy 2.5.2):

np.asarray(series).flags.writeable        = False
rd.shuffle(...)  ->  ValueError: assignment destination is read-only

np.asarray(series).copy().flags.writeable = True
rd.shuffle(...)  ->  OK

The same bug was already fixed in the lecture, ten weeks ago

inequality.md:288 reads y = np.asarray(y).copy() today — fixed by #776 on 2026-06-26 ("pandas 3.0 returns a read-only array from np.asarray() of a Series (Copy-on-Write), so rd.shuffle(y) raised … during the anaconda=2026.06 build"). The notebook carries the identical idiom and never received the fix, because the two copies of this code are not linked by anything.

Why nothing caught it

lectures/_config.yml:24-25 sets exclude_patterns: ['_static/*'], so the build never executes this notebook. It is a served artifact with zero execution coverage — the failure is only reachable by a reader.

That is the more interesting half of this issue: the lecture and the notebook contain the same algorithm, one is executed on every build and one is executed by nobody, and they have now silently diverged on a real API change. Any future environment bump can do this again.

Suggested fix

One line — y = np.asarray(y).copy() — matching the lecture.

Worth considering alongside it: whether this notebook should stay a hand-maintained twin of inequality.md's code at all, given nothing keeps them in step.

What this does not block

The dataset it builds, usa-gini-nwealth-tincome-lincome.csv, has migrated to QuantEcon/data-lectures, and a frozen copy of this notebook landed there as its provenance record (builders/usa-gini-nwealth-tincome-lincome.ipynb, builder_status: committed-frozen). That copy is deliberately not patched: it is committed verbatim as the record of what produced those bytes, and editing it would destroy the property that makes it worth keeping. The fix belongs here, in the copy that readers actually open.

Note also that the builder is non-deterministic independently of this bug — rd.shuffle is unseeded and qe.gini_coefficient accumulates in a prange loop, so re-running produces different low-order digits and cannot reproduce the committed CSV byte-for-byte. Fixing the ValueError makes it run; it does not make it reproducible. That is recorded in the manifest's integrity.upstream and is a separate question from this one.

Related: #776 (the original fix), #833 (unseeded RNG in heavy_tails and inequality, a neighbouring instance of the same reproducibility theme).

Dominant language
Jupyter Notebook
Stars
65
Forks
32
Avg merge
4d 14h
Merged PRs (30d)
6

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from QuantEcon/lecture-python-intro

All issues in QuantEcon/lecture-python-intro

Similar issues

More Documentation issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.