Measure k: the per-page script-driven origin calls the offload figure counts on neither side
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 48/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- javascript
- Domain
- analytics, full-stack
Research direction
Start with the caveat in packages/console/README.md, then trace the browser response hook, postProcess.stripScripts, the plugin serve path, and console originLoad() described here. Done means the browser, plugin, and console stages tolerate missing peers, report saved/incurred/unknown and unspecified values, and the console suite passes with the plugin catalog guard.
Written by the indexing model from the issue text.
Description
Follow-up to #152. The console states net origin offload documents-only and says the fifth term is
missing from both sides of the ledger: the XHR/API calls a page's own scripts make when a rendering
crawler runs it. This issue measures that term so the console can count it instead of stating it.
The caveat is still in the tree (packages/console/README.md): "None of it passes through the plugin,
so the figure is documents-only on both sides … the true net offload for rendering crawlers is higher
than shown."
Status — updated 2026-09-16
Not shipped. #154 implemented all three stages and was closed unmerged; nothing from it is on
main (no hydration_calls, no uncacheableSubrequests, no scriptsStripped, no rendersJs). The
work below stands as designed; only the version plan needed renumbering, because the tags it reserved
were consumed by the #155–#159 release train.
| stage | was | now |
|---|---|---|
| browser | v1.22.0 | v1.25.0 (main is 1.24.0) |
| plugin | v0.65.0 | v0.68.0 (0.66.1 shipped #106; 0.67.0 is taken by #102's PR #162) |
| console | v0.13.0 | v0.14.0 (main is 0.13.0 — went to #159) |
One interaction to fold in before re-implementing: #158's endpoint switch drops the page's own
price/availability XHR from bot renders via a fleet block.urlPatterns entry. That shrinks the
renderer's own k on that deployment and raises the blocked count — which the design already
reports, and which is exactly why blocked is counted separately rather than silently lost.
The quantity
k(page) = same-origin subrequests a page load makes whose response no shared cache would serve —
the calls that reach the origin whoever runs the page. Then, per page-view by a crawler that executes
scripts:
| origin requests | |
|---|---|
| without prerender | 1 + k (document, then the page's calls) |
| cache-served, snapshot without scripts | 0 — k is saved |
| cache-served, snapshot with scripts | k — incurred |
| proxied origin page | 1 + k — incurred |
| our own render | 1 + k — incurred (the renderer runs the page too) |
Where each piece is measured
Browser (v1.25.0) — the renderer's response hook already sees every same-origin response and
already inspects cache headers for its own resource cache. Add a pure classifier over
(request, response) → uncacheable (explicit: non-GET, no-store/private/no-cache,
Set-Cookie, uncacheable status), cacheable (explicit positive freshness: s-maxage/max-age/
Expires), or unspecified (no freshness info — CDN-default dependent; reported, counted on neither
side). Also count same-origin requests our block list aborted (blocked), so the undercount is
visible. Carry the counts on the attempt and post them with the result as
subrequests: { sameOrigin, cacheable, uncacheable, unspecified, blocked }, plus
scriptsStripped: <postProcess.stripScripts> so the plugin knows whether the stored snapshot can
hydrate at all. Cost: string checks per response; ~5 ints on the wire.
Plugin (prerender-v0.68.0)
PrerenderedPagegainsuncacheableSubrequests: IntandscriptsStripped: Boolean(nullable; older
rows read as unknown). Written on store.rendergains seriessubrequests(method = kind) — a value per posted result, so Σ = mean × count;
this is the render fleet's ownkcost.- Registry:
analytics.bots[].rendersJs(boolean). Defaultstruefor the documented renderers only —
Googlebot, Google InspectionTool, Bingbot, Applebot, YandexBot. Every AI crawler in the registry runs
nothing and stays unflagged. - Serve path: for a request from a flagged crawler, emit
hydration_calls(new metric; it needs its own
three slots): path = side (saved|incurred|unknown), method = bot, type = source, value =k.
saved= cache serve of a script-stripped snapshot;incurred= cache serve of a snapshot with
scripts, or any origin serve;unknown= no page record, or a page that predates the upgrade. One
in-memory counter bump; ~30 combo rows/period/node.
Console (prerender-console-v0.14.0) — originLoad() becomes
net = 1 − (proxied + renders + Σk_renders + probes + sitemaps + Σk_incurred) ÷ (arrived + Σk_saved + Σk_incurred),
the exposure tile becomes "script calls: saved X · incurred Y", the unknown count is printed (it decays
over one render cycle after the plugin deploys), and unspecified is shown as the CDN-default caveat.
Order
Every stage tolerates the others being absent (the console falls back to v0.12.0's exposure count; the
plugin ignores fields an older fleet does not post). Browser first so k starts landing on pages;
plugin; console. Plugin + console land in one PR here (the console's catalog guard scans the plugin's
emit sites, so a plugin-only merge would fail the console suite), released as two tags.
Not resource-intensive
Measurement rides on hooks that already run per response; storage is two small nullable fields per page;
the serve path adds one counter bump for rendering-crawler serves. The cost is coordination and the
one-render-cycle warm-up.
- Dominant language
- JavaScript
- Stars
- 0
- Forks
- 0
- Avg merge
- 9h 10m
- Merged PRs (30d)
- 56
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from HarperFast/prerender-plugin
-
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
HarperFast/prerender-plugin#176 ·
-
Difficulty 4/5 3-5 days Newbie friendliness 48/100
HarperFast/prerender-plugin#189 ·
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
HarperFast/prerender-plugin#185 · 1 comment ·
-
Difficulty 5/5 Over a week Newbie friendliness 25/100
HarperFast/prerender-plugin#183 ·
-
Difficulty 5/5 Over a week Newbie friendliness 35/100
HarperFast/prerender-plugin#180 ·
All issues in HarperFast/prerender-plugin
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
docToolchain/docToolchain#1705 ·
-
Timezone select lists one option per character; duplicate "Failed" reason; shared tracker popover id Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
nightscout/nocturne#1414 ·
-
bug v2
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
modelcontextprotocol/inspector#2458 · 1 comment ·
-
Mend: dependency security vulnerability status: needs triage 🕵️♀️
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
carbon-design-system/ibm-products#9907 ·