TIGER-AI-Lab/ClawBench

Open-source benchmark for browser AI agents on daily tasks.

Stars: 795
Forks: 58
Open beginner issues: 0
Indexed issues: 30
Avg merge: 2d 21h
Merged PRs (30d): 18

Dominant language: Python
License: Apache-2.0
Last GitHub push: Sep 15, 2026
Latest indexed: Sep 20, 2026
Contributing guide: Contributing guide
Code of conduct: No code of conduct
Beginner labels: No beginner labels indexed

Beginner friendly Everything

Open All

30 open issues indexed

Proposal: isolated environmental-observation-retrieval suite with 12 distinct tasks Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 30/100

TIGER-AI-Lab/ClawBench#375 · 2 comments · Opened Sep 14, 2026
publicity: audit existing outreach PRs and make featured-in claims traceable Open

enhancement

Difficulty 4/5 3-5 days Newbie friendliness 52/100

TIGER-AI-Lab/ClawBench#372 · Opened Sep 13, 2026
publicity: prepare a reproducible next-release demo and bilingual announcement kit Open

enhancement

Difficulty 4/5 3-5 days Newbie friendliness 55/100

TIGER-AI-Lab/ClawBench#371 · Opened Sep 13, 2026
publicity: turn the model leaderboard call-to-action into an evidence-based submission guide Open

enhancement

Difficulty 3/5 1-2 days Newbie friendliness 72/100

TIGER-AI-Lab/ClawBench#370 · Opened Sep 13, 2026
Next release: data safety, scoring correctness, recovery and package validation checklist Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 25/100

TIGER-AI-Lab/ClawBench#366 · 3 comments · Opened Sep 13, 2026
[P2] CI: runtime-server changes select zero downstream harness image builds Open

bug

Difficulty 3/5 1-2 days Newbie friendliness 74/100

TIGER-AI-Lab/ClawBench#365 · Opened Sep 13, 2026
[P2] rescore: truncated judge cache is permanently skipped instead of retried Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 48/100

TIGER-AI-Lab/ClawBench#364 · Opened Sep 13, 2026
[P1] rescore: stale judge caches are silently attributed to a newly requested judge model Open

bug

Difficulty 4/5 3-5 days Newbie friendliness 68/100

TIGER-AI-Lab/ClawBench#363 · Opened Sep 13, 2026
[P1] reproduce: validate sample coverage and selected rubrics before reporting PASS/FAIL Open

bug

TIGER-AI-Lab/ClawBench#362 · 1 comment · 1 assignee · Opened Sep 13, 2026
Harbor upstream: agree live-site oracle evidence and parity acceptance protocol Open

Difficulty 5/5 Over a week Newbie friendliness 32/100

TIGER-AI-Lab/ClawBench#359 · Opened Sep 13, 2026
Harbor: preserve verifier evidence and scoring when Kernel cleanup fails Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

TIGER-AI-Lab/ClawBench#358 · Opened Sep 13, 2026
Harbor: register browser MCP tools for the default local runtime Open

Difficulty 4/5 3-5 days Newbie friendliness 66/100

TIGER-AI-Lab/ClawBench#357 · Opened Sep 13, 2026
Priority issues to work on for contributors Open

Difficulty 5/5 Over a week Newbie friendliness 25/100

TIGER-AI-Lab/ClawBench#342 · Opened Sep 8, 2026
Harbor: strict harbor/terminal-bench format compliance + e2b sandbox support Open

TIGER-AI-Lab/ClawBench#331 · 3 comments · 1 assignee · Opened Aug 31, 2026
Proposal: isolated academia-registry-v0 suite for evidence-grounded registry research Open

question

Difficulty 5/5 Over a week Newbie friendliness 35/100

TIGER-AI-Lab/ClawBench#317 · 2 comments · Opened Aug 20, 2026
feat(eval): support DeepSeek Harness and run a reproducible ClawBench benchmark Open

enhancement

TIGER-AI-Lab/ClawBench#309 · 1 comment · 1 assignee · Opened Aug 19, 2026
Stage-1 interceptor matching is duplicated between runtime-server and edgebench_judge, and the live copy has zero tests Open

Difficulty 4/5 3-5 days Newbie friendliness 64/100

TIGER-AI-Lab/ClawBench#301 · 1 comment · Opened Aug 18, 2026
judge_llm.py has diverged from judge.py (no Gemini support, wrong responses endpoint, 800-token cap) — collapse the duplication Open

Difficulty 3/5 1-2 days Newbie friendliness 70/100

TIGER-AI-Lab/ClawBench#300 · 1 comment · Opened Aug 18, 2026
batch --resume skips failed jobs (log-file heuristic) and overwrites batch-summary.json with 'skipped' rows Open

Difficulty 4/5 3-5 days Newbie friendliness 52/100

TIGER-AI-Lab/ClawBench#297 · 1 comment · Opened Aug 18, 2026
rescore: public reproducibility CLI defaults to a maintainer's home paths and silently no-ops elsewhere Open

Difficulty 3/5 1-2 days Newbie friendliness 68/100

TIGER-AI-Lab/ClawBench#296 · 1 comment · Opened Aug 18, 2026
Always report both Stage-1 (interception) and Stage-2 (judged); raw interception overcounts ~2× Open

enhancement

Difficulty 3/5 1-2 days Newbie friendliness 62/100

TIGER-AI-Lab/ClawBench#243 · Opened Jul 2, 2026
feat: adapter — run AssistantBench (realistic time-consuming live-web tasks) under the ClawBench harness Open

enhancement

Difficulty 4/5 3-5 days Newbie friendliness 48/100

TIGER-AI-Lab/ClawBench#188 · Opened May 20, 2026
feat: adapter — run Online-Mind2Web (live-web evolution of Mind2Web) under the ClawBench harness Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 35/100

TIGER-AI-Lab/ClawBench#189 · Opened May 20, 2026
feat: adapter — run WebVoyager (live-website, screenshot+LLM-judge browser agent) under the ClawBench harness Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 35/100

TIGER-AI-Lab/ClawBench#190 · Opened May 20, 2026
feat: adapter — run evolvent-ai/ClawMark (100 multi-day coworker-agent tasks, rule-based scoring) under the ClawBench harness Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 42/100

TIGER-AI-Lab/ClawBench#169 · Opened May 18, 2026
feat: adapter — run InternLM/WildClawBench (60 in-the-wild OpenClaw tasks) under the ClawBench harness Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 35/100

TIGER-AI-Lab/ClawBench#171 · Opened May 18, 2026
test(resilience): verify resume works — auto-restart on abort + alerting Open

Difficulty 5/5 Over a week Newbie friendliness 35/100

TIGER-AI-Lab/ClawBench#160 · Opened May 13, 2026
FEAT: support steel browser Open

Difficulty 4/5 3-5 days Newbie friendliness 35/100

TIGER-AI-Lab/ClawBench#105 · Opened Apr 29, 2026
FEAT: support more browser runtimes Open

TIGER-AI-Lab/ClawBench#103 · 1 assignee · Opened Apr 29, 2026
Support running other tasks suits under ClawBench's infrastructure using adaptor layers. Open

enhancement

Difficulty 5/5 Over a week Newbie friendliness 35/100

TIGER-AI-Lab/ClawBench#72 · 1 comment · Opened Apr 15, 2026

A short digest of beginner-friendly GitHub issues.