Swallowing Critical GitHub API Errors (KeyError) in github_fetch.py
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 78/100
Research direction
Start in scripts/1-fetch/github_fetch.py at the GitHub API response parsing and the search_data["total_count"] lookup. Reproduce with a rate-limited request or invalid GH_TOKEN, then verify that the resulting failure exposes GitHub's returned error message or payload instead of only KeyError: 'total_count'.
Written by the indexing model from the issue text.
Description
Description
In scripts/1-fetch/github_fetch.py, the script fetches public repository counts for various Creative Commons tools. When parsing the GitHub API search response, the code expects to find "total_count" in the returned JSON object:
count = search_data["total_count"]
However, if the GitHub API returns an error response (such as "API Rate Limit Exceeded" or "Invalid Token" / "Bad credentials"), "total_count" will not exist in the response payload. As a result, the script encounters a KeyError on "total_count", which is caught by:
except KeyError as e:
raise shared.QuantifyingException(f"KeyError: {e}", 1)
This swallows the actual error details returned by the GitHub API (e.g., in the "message" field), making debugging extremely difficult without manually outputting the API response payload.
Reproduction
- Run
python scripts/1-fetch/github_fetch.pyusing a rate-limited network or an invalid/expiredGH_TOKEN. - The GitHub API returns a JSON response lacking the
"total_count"key (e.g., containing"message": "API rate limit exceeded..."or"message": "Bad credentials"). - The script throws
KeyError: 'total_count', which is converted intoQuantifyingException("KeyError: 'total_count'", 1). - Observe the script exiting with a generic
KeyError: 'total_count'exception, concealing the actual API rate limiting or authentication message.
Expectation
The script should check for GitHub API error messages (like "message" or other error indicators in the JSON response) before attempting to access "total_count", logging the specific error payload from GitHub to provide clarity on why the request failed.
- Dominant language
- Python
- Stars
- 48
- Forks
- 74
- PR merge metrics
- No merged PRs in 30d
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from creativecommons/quantifying
-
🏁 status: ready for work 💻 aspect: code 🔒 staff only 🛠 goal: fix 🟧 priority: high
creativecommons/quantifying#293 · 1 comment · 1 assignee ·
-
✨ goal: improvement 🏁 status: ready for work 💻 aspect: code help wanted 🟩 priority: low
Difficulty 4/5 3-5 days Newbie friendliness 38/100
creativecommons/quantifying#288 ·
-
✨ goal: improvement 💻 aspect: code 🚦 status: awaiting triage 🟩 priority: low
creativecommons/quantifying#274 · 1 assignee ·
-
✨ goal: improvement 🏁 status: ready for work 💻 aspect: code help wanted 🟩 priority: low
Difficulty 5/5 Over a week Newbie friendliness 35/100
creativecommons/quantifying#249 ·
-
✨ goal: improvement 🏁 status: ready for work 💻 aspect: code help wanted 🟩 priority: low
Difficulty 4/5 3-5 days Newbie friendliness 38/100
creativecommons/quantifying#248 · 4 comments ·
All issues in creativecommons/quantifying
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100