Amber artifact corpus: darnitdevorg/amber-corpus
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 65/100
- Issue type
- Documentation
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- yaml
- Domain
- data
Research direction
Read the screening excerpts in docs/ and the schema in entries/TEMPLATE.yaml first. Use them to fill an entry with the paper and artifact details, resource figures, claims, measurement protocol, and expected verdict described in the issue. Done means the entry follows the template and its details are supported by the screening material.
Written by the indexing model from the issue text.
Description
darnitdevorg/amber-corpus collects research artifacts from conference artifact-evaluation proceedings, where the Results Reproduced badge means an evaluator obtained the paper's main results from the submitted artifact.
Current state: mined 2325 artifacts across NDSS, USENIX Security, S&P, PETS, ACSAC, CHES, VehicleSec and WOOT; 774 carry the Reproduced badge and 271 are Functional-but-not-Reproduced, which is the negative pool. Filtered to 50 candidates needing no special hardware and shipping Docker.
Each entry records the paper, the archived artifact (DOI or pinned commit), resource figures quoted from the AE appendix, and the paper's claims with thresholds plus measurement_protocol, which is how the paper measured each metric and an expected_verdict.
Help wanted: filling in entries. The screening excerpts are in docs/, the schema in entries/TEMPLATE.yaml.
- Dominant language
- Python
- Stars
- 22
- Forks
- 27
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 35
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from darnitdevorg/darnit
-
needs triage
Difficulty 2/5 1-3 hours Newbie friendliness 83/100
darnitdevorg/darnit#587 ·
Maintainers usually reply within 1 day
-
needs triage
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
darnitdevorg/darnit#585 ·
Maintainers usually reply within 1 day
-
quality
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
darnitdevorg/darnit#523 ·
Maintainers usually reply within 1 day
-
documentation good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
darnitdevorg/darnit#516 ·
Maintainers usually reply within 1 day
-
good first issue module-core
Difficulty 1/5 1-3 hours Newbie friendliness 88/100
darnitdevorg/darnit#506 ·
Maintainers usually reply within 1 day
All issues in darnitdevorg/darnit
Similar issues
-
Difficulty 1/5 Under an hour Newbie friendliness 82/100
esphome/esphome.io#7657 ·
Maintainers usually reply within 1 day
-
comp/gateway comp/plugins duplicate P3 type/bug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
NousResearch/hermes-agent#137019 ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Kumo-Tabular weights unreachable behind restrictive egress proxies (Xet + plain HTTP both fail)Open
Difficulty 2/5 1-3 hours Newbie friendliness 67/100
NVIDIA/structured-data-models#1091 ·
Maintainers usually reply within 1 day
-
[work-item] adam-adae-worsening-emergence: conform spec.yaml to the specification style contract (REQ-1280)Possibly taken @muse-yamaa-bot claimed this today. Openwork-item
Difficulty 1/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day