Independent single-GPU runtime measurement of Matrix-Game 2.0 — your 25 fps claim reproduces
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 20/100
- Issue type
- Documentation
- Clarity
- Needs clarification
- Activity status
- Active
- Tech stack
- python
- Domain
- machine-learning
Research direction
Start by reviewing the LiveFrame adapter, configuration, and evidence bundle linked in the issue, then compare them with Matrix-Game 2.0's published single-H100, 3-step setup. The issue asks for feedback or a suggested re-run configuration but does not identify repository files or define a code change or completion criteria.
Written by the indexing model from the issue text.
Description
Hi, big fan of Matrix-Game.
I've been building an open harness called LiveFrame that measures interactive world models' released pipelines on stated hardware, and binds every result to a digest ledger that anyone can re-verify.
I ran Matrix-Game 2.0 in the configuration your claim names (single H100, 3 steps) and measured 19.7 fps median across 24 cases, mean 19.6 with a std of 0.4. My harness has a disclosed delivery bias of about 8% and runs with conservative settings, so I read this as consistent with your published 25 fps. Of the four models I measured, yours was the only claim specified fully enough to test as published, and it held up.
Two things I'd value your eyes on:
- The adapter and config I used. Is there anything I should change to represent MG2 more fairly? It runs your pipeline with flash-attn per your pins, and the evidence bundle with the ledger is in the repo.
- If you'd ever want to cite an independent reproduction, or have me re-run with settings you recommend, I'm glad to.
Harness: https://github.com/kkjcodes/liveframe
Evidence bundle: https://github.com/kkjcodes/liveframe/tree/main/agent/world/evidence/m7-mg2-h100
Re-verify: liveframe world verify /ledger.json
Thanks for open-sourcing the model. Happy to answer anything.
- Dominant language
- Python
- Stars
- 2.7k
- Forks
- 285
- PR merge metrics
- No merged PRs in 30d
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from SkyworkAI/Matrix-Game
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
SkyworkAI/Matrix-Game#90 ·
-
Replace deprecated v2.ToTensor() in Matrix-Game-2/inference.py with v2.ToImage() + v2.ToDtype(scale=True)Possibly taken @xyf5432 claimed this 64 days ago. Open
Difficulty 1/5 Under an hour Newbie friendliness 90/100
SkyworkAI/Matrix-Game#82 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
SkyworkAI/Matrix-Game#80 ·
-
run logic error when finishPossibly taken @Arison591 claimed this 101 days ago. Open
Difficulty 1/5 Under an hour Newbie friendliness 88/100
SkyworkAI/Matrix-Game#77 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 20/100
SkyworkAI/Matrix-Game#93 ·
All issues in SkyworkAI/Matrix-Game
Similar issues
-
Difficulty 1/5 Under an hour Newbie friendliness 85/100
521xueweihan/HelloGitHub#4115 ·
-
get_lldp_neighbors_detail: query asks for 'max-address' (typo for 'mac-address'), so remote_chassis_id is always emptyPossibly taken @Jah-yee claimed this today. Open
Difficulty 1/5 Under an hour Newbie friendliness 88/100
-
lesson-submission pending-review
Difficulty 1/5 Under an hour Newbie friendliness 74/100
Ikalus1988/MisakaNet#3899 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
aws-samples/sample-per-model-bedrock#549 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 90/100