Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

results summarizer is broken since 4.1.45

Open
  • Is something wrong?
#481 3 comments 0 reactions 1 assignee View on GitHub

@pgmpablo157321 is already working on this.

Since Oct 8, 2026.

Assessment

This issue has not been assessed yet.

Description

After compliance check adds scaling factors, the summarizer won't print out the summarized results since v4.1.44. Issue occurs after checking out branch at 4.1.45 commits or newer commits, but summarizer works as expected on the same results path when using the prior v4.1.44 commits as shown below.

This issue prevents any recent MLPerf Training results to display when checking submissions.

When running the mlperf logging verify script from v4.1.45 or later versions on the v6.0 results, you don't get the detailed summarized output, you only see the following:

logging]$ git checkout 782506cc6df92d08660e91d6734a086b5480c8da
Previous HEAD position was 77699df Increment version to 4.1.44
HEAD is now at 782506c Increment version to 4.1.45
logging]$ ./scripts/verify_for_v6.0_training.sh ../training_results_v6.0/Lambda

            ** Detailed log output is also at package_checker.log
             NOTICE: Applying scaling factor 1.0311762501465556 to dir ../training_results_v6.0/Lambda/results/gb300_n18_ngc25.09_nemo/gpt_oss_20b
             Strong Scaling Scores:
             INFO - Running repository checks.
             INFO - Running git-unfriendly file name checks.
             INFO - Running large file checks.
             INFO - ALL CHECKS PASSED.

When using the mlperf logging verify scripts from v4.1.44 on the v6.0 training results, you get this output for the Lamda results (which is expected):

logging]$ git checkout 77699dfa13df58fe9fa541f409b23b2b963b7be0
Previous HEAD position was 782506c Increment version to 4.1.45
HEAD is now at 77699df Increment version to 4.1.44
logging]$ ./scripts/verify_for_v6.0_training.sh ../training_results_v6.0/Lambda

               ** Detailed log output is also at package_checker.log
              NOTICE: Applying scaling factor 1.0311762501465556 to dir Lambda/results/gb300_n18_ngc25.09_nemo/gpt_oss_20b
              Strong Scaling Scores:
              division     availability submitter                          system  \
              0   closed  Available cloud    Lambda  Lambda-1-Click-Cluster_B200_n1
              1   closed  Available cloud    Lambda                Lambda_GB300_n18

              number_of_nodes       host_processor_model_name  host_processors_count  \
              0               1  Intel(R) Xeon(R) Platinum 8570                      2
              1              18                     Neoverse-V2                     36

              accelerator_model_name  accelerators_count  \
              0  NVIDIA Blackwell GPU (B200-SXM-180GB)                   8
              1     NVIDIA Blackwell Ultra GPU (GB300)                  72

              framework       notes  llama31_8b  dlrm_dcnv2  flux1  \
              0  PyTorch NVIDIA Release 25.04  B200 1000W   85.263156         NaN    NaN
              1  PyTorch NVIDIA Release 25.09               11.585940         NaN    NaN

              llama2_70b_lora  llama31_405b  gpt_oss_20b  deepseekv3_671b  \
              0              NaN           NaN    96.457092              NaN
              1              NaN           NaN    18.359156              NaN
                                                                                            details_url  \
              0    https://github.com/mlcommons/training_results_v6.0/blob/main/Lambda/systems/b200_n1_ngc25.09_nemo.json
              1  https://github.com/mlcommons/training_results_v6.0/blob/main/Lambda/systems/gb300_n18_ngc25.09_nemo.json

                                                                       code_url
              0  https://github.com/mlcommons/training_results_v6.0/blob/master/Lambda/benchmarks
              1  https://github.com/mlcommons/training_results_v6.0/blob/master/Lambda/benchmarks
              INFO - Running repository checks.
              INFO - Running git-unfriendly file name checks.
              INFO - Running large file checks.
              INFO - ALL CHECKS PASSED.
Dominant language
Python
Stars
43
Forks
60
Avg merge
2d 14h
Merged PRs (30d)
1

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from mlcommons/logging

All issues in mlcommons/logging

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.