Integration test: grader-only files never reach the trial
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 74/100
- Issue type
- Feature
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- docker, python
- Domain
- ai-infra-agents, testing
Research direction
Start with src/meridian/runtime/payload.py and src/meridian/runtime/pytest_runner.py, then read the existing integration test setup to see how trials are launched. Add a trial whose agent lists the filesystem and check its output or final state for grader files. Done means an integration test fails if any hidden test file reaches the trial.
Written by the indexing model from the issue text.
Description
Context
A task's hidden tests ship as grader files that must never be visible to the agent. Unit tests cover the overlay at grading time, but nothing proves they are absent from the trial container.
What to do
Run a trial whose agent lists the filesystem, and assert no grader file appears in its output or final state.
Done when
Leaking a hidden test into a trial fails an integration test.
Where to look
src/meridian/runtime/payload.py, src/meridian/runtime/pytest_runner.py
New here? Read CONTRIBUTING.md and comment on this issue to say you are taking it.
Generated by Claude Code
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- Avg merge
- 1h 54m
- Merged PRs (30d)
- 14
Getting set up
- Ships a Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from patsypppe/meridian
-
area: security area: tests size: S tests
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Maintainers usually reply within 1 day
-
area: dashboard area: tests good first issue size: XS tests
Difficulty 1/5 1-3 hours Newbie friendliness 88/100
Maintainers usually reply within 1 day
-
area: dashboard area: tests good first issue size: S tests
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 1 day
-
area: dashboard area: tests good first issue size: XS tests
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
Maintainers usually reply within 1 day
-
area: gate area: tests good first issue size: S tests
Difficulty 1/5 1-3 hours Newbie friendliness 85/100
Maintainers usually reply within 1 day
All issues in patsypppe/meridian
Similar issues
-
request-theme
Difficulty 1/5 Under an hour Newbie friendliness 78/100
LizardByte/ThemerrDB#9883 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Project-MONAI/MONAI#9258 ·
Maintainers usually reply within 3 days
-
catalog:deprecation
Difficulty 1/5 Under an hour Newbie friendliness 88/100
surendranb/free-inference#31 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
fritzprix/libr-agent#2612 ·
Maintainers usually reply within 1 day
-
needs-triage presubmit-gate
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
gke-labs/kube-agents#5001 ·
Maintainers usually reply within 1 day