eval(routing): check_regression catches Exception more broadly than its calls raise

Open Beginner friendly
#448 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
84/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
testing

Research direction

Read evals/routing/check_regression.py, especially the handlers at lines 33, 47, and 67, and identify which exceptions the file reads and json.loads can raise. Narrow each handler to those errors, then check the file’s relevant tests or run the available regression check to confirm expected soft failures still work and unexpected errors are no longer hidden.

Written by the indexing model from the issue text.

Description

enhancement

evals/routing/check_regression.py (lines 33, 47 and 67) uses except Exception around file reads and json.loads. Those raise OSError, UnicodeDecodeError and json.JSONDecodeError; catching everything can hide a real bug as a soft failure. Narrow each handler to the errors its calls raise.

Found in review of #416 and left out of that PR to keep it small.

Dominant language
HTML
Stars
347
Forks
12
Avg merge
2d 16h
Merged PRs (30d)
95

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from coco-research/coco

All issues in coco-research/coco

Similar issues

More Testing & QA issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.