How easy it is to externalize the visibility of the benchmarks we already run continuously?
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 20/100
- Issue type
- Feature
- Clarity
- Needs clarification
- Activity status
- Stale
- Domain
- build-system, ci-cd
Research direction
Start with the linked TensorFlow PR comment and the TF2 benchmarks under the official model benchmark directory. Review how PerfZero runs the benchmarks internally and clarify which benchmarks, CI system, and integration path are intended; the issue does not define a concrete completion condition.
Written by the indexing model from the issue text.
Description
This a pinpoint to not lost the benchmarking subthread started at https://github.com/tensorflow/tensorflow/pull/33945#issuecomment-652471742
Quoting @zongweiz
We have [TF2 benchmarks] (https://github.com/tensorflow/models/tree/master/official/benchmark) from every official model garden model and they continuously running internally, using PerfZero framework. We should be able to select a set of benchmarks and hook it up with github CI (jenkins). Added @sganeshb to this thread for information and we will follow up.
Another thought is to start CI tests using a selected set of tf_benchmarks
/cc @alextp @naveenjha123 @sganeshb
- Dominant language
- Shell
- Stars
- 311
- Forks
- 139
- Avg merge
- 1h 28m
- Merged PRs (30d)
- 1
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from tensorflow/build
-
sig build dockerfiles
Difficulty 3/5 1-2 days Newbie friendliness 35/100
tensorflow/build#258 ·
-
sig build dockerfiles
Difficulty 2/5 1-3 hours Newbie friendliness 45/100
tensorflow/build#221 · 2 comments · 5 reactions ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
tensorflow/build#217 · 3 comments ·
-
Difficulty 5/5 Over a week Newbie friendliness 20/100
tensorflow/build#213 ·
-
sig build dockerfiles
Difficulty 3/5 1-2 days Newbie friendliness 35/100
tensorflow/build#210 · 1 comment ·
All issues in tensorflow/build
Similar issues
-
out-of-date
Difficulty 1/5 Under an hour Newbie friendliness 82/100
CachyOS/CachyOS-PKGBUILDS#1917 ·
Maintainers usually reply within 1 day
-
package-update
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
bug needs triage
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
EmbarkStudios/cargo-about#323 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
vllm-project/vllm-metal#841 ·
Maintainers usually reply within 1 day