Full local bin/test runs fail 60 s time limits that CI passes

Open
#412 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
48/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
swift
Domain
testing

Research direction

Start by reading bin/test and check how it sets SWT_EXPERIMENTAL_MAXIMUM_PARALLELIZATION_WIDTH; compare the Swift versions used locally and in CI described in the issue. Then run bin/test --all and inspect the listed BoardModel suites and ProcessTests.runsMoreCommandsAtOnceThanThereAreCPUs if time limits still fail. Done means the full run passes without reducing any test's coverage.

Written by the indexing model from the issue text.

Description

bug

A full local bin/test run fails tests that have a 60 s time limit, while CI passes them. The failing suites are the BoardModel suites with .timeLimit(.minutes(1)) (BoardModelAcknowledge, BoardModelPlan*, BoardModelStartNow, BoardModelSuggestions) and ProcessTests.runsMoreCommandsAtOnceThanThereAreCPUs. In a full parallel run, every app test reports 50 to 68 s, so the 60 s limits trip. One run had 49 such failures (seen on #329). A later run at load 10.8 had 76, and each of those tests passed when run alone.

Cause

xcode-select -p on this Mac points at /Applications/Xcode-26.0.0.app, whose Swift is 6.2. The Command Line Tools at /Library/Developer/CommandLineTools have Swift 6.4.2, and CI's macos-26 runner has Swift 6.3.3.

bin/test sets SWT_EXPERIMENTAL_MAXIMUM_PARALLELIZATION_WIDTH=2 so that only two tests run at once. Swift Testing reads that variable from Swift 6.3, and the 6.2 one ignores it, so a local run starts every test at once. Swift Testing's .timeLimit clock starts when the test starts, so it counts the time a @MainActor test waits for the main actor and the time an async test waits for a cooperative-pool thread. With every test started at once, those waits reach a minute.

Done when

A full local bin/test --all run passes without these time-limit failures, and no test checks less than it does now.

Dominant language
Swift
Stars
0
Forks
0
Avg merge
6h 15m
Merged PRs (30d)
308

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from haacked/switchboard

All issues in haacked/switchboard

Similar issues

More Swift issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.