Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Docs quiz: randomised questions generated from the documentation, by level

Open
  • Is something wrong?
#237 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
15/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Active
Tech stack
nuxt
Domain
documentation, frontend

Research direction

This is an open design proposal, and its final questions (quiz length, placement, topic tracks, whether Deployment is included) are still undecided. Before any code, read the Nuxt Content setup and the page frontmatter conventions to see how sections and hashes could be derived. Done means the maintainers agree on the open questions and the generation script pnpm quiz:generate is specified.

Written by the indexing model from the issue text.

Description

enhancement

A short quiz on the docs site that excites and intrigues developers, introduces CWA's ideas, and tests their knowledge a bit. Every answer links back to the docs section it came from. This is a proposal for discussion; the open questions are at the end.

What we've decided so far

  • Audience: developers. Questions come only from the documentation.
  • Where: a feature of the docs site.
  • Question types: multiple choice, "select the line of code that does X", and "spot the mistake".
  • Levels: the reader picks beginner, intermediate or advanced (and maybe a topic or two) before starting.
  • Variety without hand-writing: we generate the questions from the docs ourselves, and they're randomised, so each person's short quiz differs a lot and nobody has to write or code new questions.
  • Results: stay in the browser for now. No sign-in and no stored scores.

Proposed design

1. Map the docs into sections

Nuxt Content already parses every page into headings and code blocks. The unit for questions is a section: page path + heading anchor + a hash of its text. That gives each question:

  • a source to link to ("Read more" goes to /api/routes#route-generation);
  • a way to know it's stale (the section's hash changed).

Pages opt out with frontmatter quiz: false (releases, troubleshooting, the glossary maybe).

The default level comes from where the page lives, and a model can adjust it per question:

Docs section Default level
Getting Started, Key Ideas, Guides beginner
Core Concepts intermediate
API, Nuxt Module, Deployment advanced
2. Generate a question bank offline, not at runtime

A script (pnpm quiz:generate) sends each changed section to Claude and gets back questions in a fixed JSON schema. They're stored in the repo as a Nuxt Content data collection (content/quiz/<page>.yml) and prerendered with the site, so it costs nothing per visitor and needs no server.

Each question records:

  • type: choice, pick-line or spot-mistake
  • level, and concept tags
  • source: the page and anchor, plus the section hash it was generated from
  • quote: the exact sentence(s) or code from the section that prove the answer
  • answer, distractors, and a one or two sentence explanation

Only sections whose hash changed are regenerated, so a docs edit refreshes its questions in the same PR, and a reviewer reads the YAML diff. That's reading, not coding.

3. How each type comes from the docs
  • Multiple choice: a fact in the section. The distractors must be plausible, and the prompt forbids answers borrowed from other frameworks (the same rule as the chat persona in #1).
  • Select the line: take a real code block from the docs. The model picks one line and describes what it does; the reader clicks the line. The code is real by construction.
  • Spot the mistake: take a real code block and make one realistic mutation. Examples: a phantom method, .value on a plain boolean, getEntity() instead of getPage(), a uiComponent without its prefix, PUT where the docs say PATCH. The reader clicks the wrong line. Many of the best mutations are mistakes the docs audits actually found.
4. Grounding check before a question is kept

The docs audits showed how easily a confident, wrong claim gets in. So a second, separate pass gets only the section and the question, and must answer it on its own:

  • if it picks a different answer, or the quote isn't verbatim in the section, the question is dropped;
  • for spot-the-mistake, it must find the mutated line without being told.
5. Randomising each quiz

The browser picks about 10 questions for the chosen level and topics, spread across sections so one page can't dominate, and shuffles the options. The pick is seeded, so "share this quiz" can be a link with the seed, and a friend gets the same questions.

Each answer shows the explanation and a link to the section, which covers the "introduce the concepts" goal. A "report this question" link opens a pre-filled GitHub issue.

Cost

Generation is a one-off per changed section. At around 400 pages and 5–10 questions per section, a full first run is a few dollars on Haiku or Sonnet. After that, only edited sections cost anything. There's nothing at runtime.

Overlap with #1 (chat assistant)

Both need the docs split into sections with stable anchors and hashes. That step could be shared.

Open questions

  • How many questions in a quiz, and is there a timer?
  • Is there a score summary at the end ("you're intermediate on caching, beginner on fixtures")? That needs concept tags we trust.
  • Topic choice: free picks, or a few fixed tracks (Content editing, Building components, API, Deployment)?
  • Do Deployment questions belong? They're template-specific and change often.
  • Should generation run in CI on every docs PR, or locally before committing?
  • Where does it live: /quiz, a button in the header, or a card at the end of each guide ("Test yourself on this page")?
  • Should questions about versions ("since alpha.18") be excluded? They date quickly.
Dominant language
TypeScript
Stars
0
Forks
0
Avg merge
1h 20m
Merged PRs (30d)
1

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from components-web-app/docs

All issues in components-web-app/docs

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.