Full test reproducibility appears to be failing on internal tests

Open
#482 10 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

A pull request for this has already been merged.

  • #485 by @gowerc — merged

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Tech stack
r
Domain
testing-qa

Research direction

Start with the linked check-rbmi.html artifact and test-reproducibility.R at line 75, then compare how the full test suite is run with the GitHub Actions pipelines. Reproduce the mismatch and identify whether the failure is limited to the full suite. Done means the reproducibility test passes consistently or the differing test coverage is documented.

Written by the indexing model from the issue text.

Description

bug

https://statistical-engineering.pages.roche.com/-/systems-integration-tests/-/jobs/77276548/artifacts/public/logs/check-rbmi.html

  ── Failure ('test-reproducibility.R:75:9'): Results are Reproducible ───────────
  `drawobj` (`actual`) not equal to `drawobj2` (`expected`).
  
      actual$samples[[1]]$beta | expected$samples[[1]]$beta    
  [1] 4.844                    - 5.504                      [1]
  [2] 8.212                    - 6.047                      [2]
  [3] 0.724                    - -0.615                     [3]
  [4] 0.448                    - -0.599                     [4]
  [5] 3.090                    - 3.010                      [5]
  [6] 2.247                    - 2.715                      [6]
  [7] -1.141                   - 1.856                      [7]
  [8] -0.486                   - 2.180                      [8]
  
  actual$samples[[1]]$sigma$A vs expected$samples[[1]]$sigma$A
                                             [,1]        [,2]        [,3]
  - actual$samples[[1]]$sigma$A[1, ]    3.8996258  1.81540872  0.18346780
  + expected$samples[[1]]$sigma$A[1, ]  4.9890982  1.04694767 -0.13357694
  - actual$samples[[1]]$sigma$A[2, ]    1.8154087  1.88882220 -0.04483718
  + expected$samples[[1]]$sigma$A[2, ]  1.0469477  1.02407796 -0.03493389
  - actual$samples[[1]]$sigma$A[3, ]    0.1834678 -0.04483718  0.37916568
  + expected$samples[[1]]$sigma$A[3, ] -0.1335769 -0.03493389  0.35201823

Haven't checked but I think this is from the FULL test suite that doesn't get checked via the GitHub actions pipelines

Dominant language
R
Stars
21
Forks
13
Avg merge
13h 22m
Merged PRs (30d)
7

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from openpharma/rbmi

All issues in openpharma/rbmi

Similar issues

More R issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.