Generate the whole Consumer Trends package from its workbook, before the 22 December release

Đang mở
  • Có gì không ổn?
#390 1 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
4/5
Thời gian dự kiến
3-5 ngày
Mức phù hợp với người mới
48/100
Loại issue
Tính năng
Độ rõ ràng
Đặc tả rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
jupyter-notebook, python, yaml
Lĩnh vực
data-engineering, tooling

Hướng nghiên cứu

Start with scripts/build_uk_consumption_tax_packages.py and the pinned cpnsa.xlsx; trace how build_ons() combines workbook data with the seven legacy series and package metadata in YAML. Define the edition constants and generate the full package, preserving record order, ids, concepts, labels and notes. Verify that building without the YAML reproduces the committed file byte for byte, then run the five listed Consumer Trends tests and update the three named documentation sections.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

Summary

  • ons-consumer-trends-current-price-september-2026 (#384) cannot be built from its workbook alone.
  • build_ons() in scripts/build_uk_consumption_tax_packages.py reads the package's own YAML. From that file it takes the package skeleton and the 217 record sets of the seven series that predate #322.
  • From the workbook it only refreshes those series' column headers, CDID guards and shared dimensions, and regenerates every expanded_ record set.
  • ONS's next Consumer Trends release is 22 December 2026. Its package should come from the workbook, not from another renamed copy of this file.
  • Raised in review of #384: round 1, finding 1, and round 2.

How it came about

The workbook gives values, headers and series codes. It does not give Chronicle's naming choices for these series: concepts such as ons.household_expenditure.gas, record-set ids such as 04cn_04_5_2.cy2024, each series' labels and notes, and the "Q1 2020" quarter label. Those choices exist only in the YAML.

  • bfd88bd8 (9 September) committed the first Consumer Trends package directly as YAML, with no generator.
  • a362d1e8 (10 September) and d921b513 (15 September) edited that YAML to add quarters and the 04.5.1 to 04.5.4 subclasses.
  • 01e5316e (8 October, #322) added the first generator. It had to keep the existing facts' ids and concepts unchanged, because Microcosm vendors those rows. So it reads those record sets back from the YAML and generates only the new ones.
  • #384 made the September package by copying the June YAML and renaming its ids.

What the script takes from the YAML

The seven older series.

  • They are:
    • 04.5, electricity, gas and other fuels;
    • 04.5.1, electricity;
    • 04.5.2, gas;
    • 04.5.3, liquid fuels;
    • 04.5.4, solid fuels;
    • 07.2.2, fuels and lubricants for personal transport equipment;
    • 07.3.2, passenger transport by road.
  • Their record-set groups are 04cn, 04cn_04_5_1 to 04cn_04_5_4, 07cn_07_2_2 and 07cn_07_3_2.
    • Each group has 31 periods: calendar years 2020 to 2025 and quarters 2020 Q1 to 2026 Q1.
    • Each record set has one row and one measure.
  • They keep their specific ons.household_expenditure.* concepts, where the expanded series use ons.household_final_consumption_expenditure.
  • They come from earlier packages. #384 carried them from the June package to the September one by renaming their ids.

The package skeleton. schema_version, package_id, label, dimension_labels, dimension_value_labels and artifact.

With the YAML deleted, build_ons() fails with FileNotFoundError.

Requested

  • build_ons() builds the whole package from the pinned cpnsa.xlsx and constants in the script.
  • With the YAML deleted, it writes the committed file back byte for byte. The built facts are identical, including their aggregate and semantic keys.
  • The seven series keep their current shape, quirks included:
    • their quarterly source_period_label reads "Q1 2020", where the expanded record sets read "2020 Q1";
    • each series has its own measure label and concept_evidence_notes;
    • the record sets and their keys keep their order.
  • The edition is a set of constants: package directory, workbook, id prefix, vintage and last quarter. The December package is then a change of constants plus a new archive.
  • These now say the package is generated from the workbook:
    • the script's docstring and ONS_COMMENT header;
    • "Authoring and verification" in docs/uk-consumption-tax-facts.md;
    • section 2 of docs/data-sources/uk-excise-facts.md.

Out of scope.

  • The December release itself, and with it 2026 Q2, which waits for that refresh.
  • Any change to the seven series' concepts or ids.

Checks.

  • The Consumer Trends tests pass:
    • tests/test_source_uk_consumption_tax_facts.py;
    • tests/test_issue_254_transport_energy_sources.py;
    • tests/test_issue_257_transport_energy_followups.py;
    • tests/test_issue_270_need_ons_followups.py;
    • tests/test_issue_381_uk_excise_facts.py.
  • If the facts are identical, no bundle pin moves.

🤖 Generated with Claude Code

Ngôn ngữ chính
Python
Star
0
Fork
3
Merge trung bình
1 ngày 4 giờ
Pull request đã merge (30 ngày)
65

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của PolicyEngine/chronicle

Tất cả issue của PolicyEngine/chronicle

Issue tương tự

Thêm issue về Python

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.