perf(api): Implement ETag conditional caching and request throttling to reduce GitHub API quota consumption
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 45/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- github, javascript
- Domain
- api, performance
Research direction
Start by reading src/services/github.js and src/services/cache.js to trace GitHub requests, IndexedDB entries, and cache-expiry handling. Map the existing repository, metrics, and contributor fetch paths before deciding how the proposed conditional requests, throttling, and rate-limit feedback fit together. Done means expired entries can revalidate, unchanged data refreshes its TTL, and request pressure and rate-limit headers are handled gracefully.
Written by the indexing model from the issue text.
Description
Problem Statement
Currently, OrgExplorer relies on GitHub REST API calls to fetch repository lists, metrics, and contributor data. While local IndexedDB caching is implemented, expired cache entries trigger full network re-fetches.
For unauthenticated users (limited to 60 requests/hour) and authenticated users (5,000 requests/hour), repeated re-fetching rapidly exhausts GitHub API rate limits, especially when exploring large organizations with dozens of repositories.
Proposed Solution
1. ETag-based Conditional Requests (If-None-Match)
- Store the
ETagheader along with cached payloads in IndexedDB (src/services/cache.js). - When a cached item expires, send a conditional request using the
If-None-Match: <etag>header. - If data has not changed, GitHub responds with
304 Not Modified, which costs 0 points against the GitHub rate limit quota. - Upon receiving a
304, refresh the local cache TTL and immediately return the stored data without re-downloading the payload.
2. Request Throttling & Batching
- Throttle sequential/parallel API calls to prevent triggering GitHub secondary rate limits when navigating through repository-heavy organizations.
3. Graceful Rate Limit Handling
- Read
x-ratelimit-remainingandx-ratelimit-resetheaders from responses to dynamically adjust UI feedback and throttle rates before hard failures happen.
Files to be Modified
src/services/github.jssrc/services/cache.js
Benefits
- Prevents unnecessary API quota consumption on recurring visits and stale-check cycles.
- Significantly speeds up page loads for returning visitors.
- Provides a smoother and more resilient user experience across the app.
- Dominant language
- JavaScript
- Stars
- 35
- Forks
- 98
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 9
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from AOSSIE-Org/OrgExplorer
-
[BUG]: "Clear All" in Settings doesn't clear the saved analysis, old data comes back after reload Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
AOSSIE-Org/OrgExplorer#253 · 3 comments · 1 assignee ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
AOSSIE-Org/OrgExplorer#251 · 2 comments · 1 assignee ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
AOSSIE-Org/OrgExplorer#245 · 1 assignee ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
AOSSIE-Org/OrgExplorer#230 · 2 comments ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
AOSSIE-Org/OrgExplorer#228 · 2 comments ·
All issues in AOSSIE-Org/OrgExplorer
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
area-deployment area-integrations triage:bot-seen
Difficulty 2/5 Half a day Newbie friendliness 86/100
-
Issue-Bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
sugarlabs/musicblocks#8924 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100
ArduPilot/ardupilot_wiki#8088 ·
-
[BUG] createTool tools cannot be registered with Mastra when exactOptionalPropertyTypes is enabled Opencustomer-eng status: needs triage
Difficulty 2/5 1-3 hours Newbie friendliness 84/100