Committee letters: the letterhead rules are precise but find only 35 of 17,519 — measure what they miss

Open
#153 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
22/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
python

Research direction

Start in build/committees.py: read classify() and the four positive rules, then run it over the 17,519-letter corpus and sample the rejects to hand-label what the rules turned away — that recall number is the deliverable. For the multi-committee gap, look at how classify() returns a single result and compare with the two-letterhead case in #125; for the letterhead-in-image case, check what ocr_tesseract.py stores against the text layer before deciding whether re-OCR is needed. Done means a measured recall figure with a labelled sample plus a written recommendation on the oversight-archive.waxman.house.gov question from #76.

Written by the indexing model from the issue text.

Description

enhancement

build/committees.py reads a committee letterhead out of a letter's text, and does it carefully — four positive rules, hand-checked at 65 of 65 sampled correct on 2026-11-13. It names its own limitation:

What it misses is letters whose text has no letterhead and does not open as the committee.

This is to measure that miss and decide what to do about it.

Measured on the PC's share

letters with text on this machine         17,519
identified as committee stationery            35

By committee: House Energy and Commerce 20, Ways and Means 18, Education and the Workforce 6, Senate Finance 6, House Oversight 2, and eleven others. By rule: majority/minority phones 10, opening sentence 12, opens with committee name 7, roster 6.

35 is very small. The Mac's corpus is larger and will find more, but the ratio is the thing to look at: the rules are precise and appear to have low recall.

Three ways a committee letter is missed today

1. No letterhead in the text. The letterhead is an image on many scanned letters, and the rules then fall back to the opening sentence — which only catches letters that announce themselves ("The Committee on Oversight … will be conducting oversight"). A committee letter that simply makes its request is invisible.

2. More than one committee. The AI-companies letter in #125 is on the letterhead of both the House Financial Services Committee and the House Foreign Affairs Committee, signed by Gottheimer and Lawler. classify() returns one committee per letter.

3. A committee archive on a member's host. oversight-archive.waxman.house.gov in #76 holds 5 letters and is titled "Committee on Oversight and Government Reform :: Chairman Henry A. Waxman". It is a committee archive, not a caucus and not a member's own site, and #76 proposed sending it here.

What to do

  • Measure recall, not just precision. Take a sample of letters the rules reject and read them: how many are committee letters? 65 of 65 accepted were right; nobody has checked what was turned away.
  • Allow more than one committee per letter, and record all of them. A joint letter from two committees is common and currently loses one.
  • Use the signature block as a second source. A letter signed by a chairman and a ranking member together, with their committee titles, is committee business whatever the letterhead shows — and the signature survives in 94% of letters with text (#41).
  • Decide oversight-archive.waxman.house.gov — 5 letters, the committee archive case from #76.
  • Re-OCR where the letterhead is an image. ocr_tesseract.py reads the first six pages at 300 dpi and the letterhead is on page one, so a letter with no letterhead in its text layer may have one in its OCR. Worth checking whether the two sources disagree.

Why it matters

A committee letter credited to the member whose site hosted it reads as a personal letter. Waxman writing to the FDA as Chairman of Oversight is a different act from Waxman writing as the member for Los Angeles, and the explorer currently cannot tell a reader which it was — except for the 35 where the letterhead survived.

Related: #125 (the two-committee AI letter), #76 (the Waxman oversight archive), #91, #23 (harvest committee sites), #41 (signature blocks), #152 (the same problem for caucuses).

Dominant language
Python
Stars
0
Forks
0
Avg merge
1h 11m
Merged PRs (30d)
413

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Robin-IA/Congressional-Letters-Dev

All issues in Robin-IA/Congressional-Letters-Dev

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.