Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Fuzz every problem and language driver against the reference solutions

Open
  • Is something wrong?
#24 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Feature
Clarity
Clearly specified
Activity status
Active
Tech stack
javascript, node.js
Domain
backend, ci-cd, testing

Research direction

Start by reading backend/arena/runner/harness.js and the existing npm run validate:problems workflow to understand how reference solutions, problem metadata, and language drivers are run. Then inspect problem.function parameter types and the constraints referenced in issues #3 and #12. Done means the new seeded fuzzer supports the requested flags, shrinking, driver checks and CI run, and the stated validation commands pass.

Written by the indexing model from the issue text.

Description

enhancement help wanted

Every problem is validated by running its JavaScript and Python references against its hand-written tests (npm run validate:problems). That only proves the references agree on inputs someone thought of. Bugs hide at the edges: empty arrays, negative numbers, duplicate values, maximum sizes, unicode strings.

What to do

Add backend/arena/scripts/fuzz-problems.js:

  • Generate random inputs from each problem's parameter types in problem.function (all types listed in #3), respecting declared constraints where present (see #12).
  • Bias generation toward edge cases: empty, single element, all equal, sorted, reverse sorted, min and max values.
  • Run both reference solutions on each input and compare with the problem's compare mode.
  • On a mismatch, shrink the input to the smallest failing case and print it, ready to paste into the problem as a new test.
  • Run every language driver in backend/arena/runner/harness.js on a reference solution too, so driver bugs in serialization (big integers, floats, nested tuples, unicode) are caught. New language drivers (#1, #2, #3, #5, #6) should pass this before merging.
  • Deterministic by seed. --seed, --runs and --problem flags.

Definition of done

  • Runs over all 45 problems locally in a few minutes with CODEARENA_RUNNER=local.
  • Prove it works: break one reference on purpose and show the fuzzer finding and shrinking the failure in the pull request.
  • Any real disagreements found get their own issues or fixes, with the shrunk input added as a test.
  • A short fuzz run (fixed seed, small run count) added to CI.
  • npm test and node scripts/check-boundary.js pass.
Dominant language
JavaScript
Stars
0
Forks
2
No merged PRs in 30d

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from sennaicodes/codearena

All issues in sennaicodes/codearena

Similar issues

More JavaScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.