[Bug]: Swift Testing JSONL counts assertion issues as failed tests

Bắt đầu với src/utils/swift-testing-line-parsers.ts, src/utils/xcodebuild-event-parser.ts và src/utils/xcodebuild-run-state.ts, tập trung vào parseSwiftTestingRunSummary, createXcodebuildEventParser và createTestSummaryFragment. Chạy reproduce.mjs được cung cấp trên bản build của package và xác minh rằng các chẩn đoán assertion vẫn tách biệt với số lượng test, với completed 2, failed 1, totalTests 2, passedTests 1 và failedTests 1.

Bug Description

XcodeBuildMCP 2.7.0 JSONL test progress and the terminal build summary count Swift Testing assertion issues as failed tests. A single failed test can contain several failed #expect assertions, so this produces failed > completed, invents extra tests in the summary, and can report zero passes despite emitting passing test-case results.

Confirmed by replaying the original Xcode output through the installed, unmodified upstream parser/run-state modules, and independently with the small parser reproduction below. The reproduction uses no OptionsWheelTracker harness, project, simulator, or LLM. Installed source-map contents match the v2.7.0 sources. The relevant files are unchanged on upstream main at e6ef59b49b44012c824f0a0de261c96142e37390.

Debug Output

Output from xcodebuildmcp doctor doctor --output json:

{
  "schema": "xcodebuildmcp.output.doctor-report",
  "schemaVersion": "2",
  "didError": false,
  "error": null,
  "data": {
    "serverVersion": "2.7.0",
    "checks": [
      {
        "name": "xcode",
        "status": "ok",
        "message": "Xcode 26.6 - Build version 17F113 (/Applications/Xcode.app/Contents/Developer)"
      },
      {
        "name": "process-tree",
        "status": "ok",
        "message": "Running under Xcode: No; 4 process entries"
      },
      {
        "name": "axe",
        "status": "ok",
        "message": "Available: Yes; UI automation: Yes; Video capture: Yes"
      },
      {
        "name": "xcodemake",
        "status": "ok",
        "message": "Enabled: No; Binary: No; Makefile: Not checked"
      },
      {
        "name": "mise",
        "status": "warning",
        "message": "Running under mise: No; Available: No"
      },
      {
        "name": "debugger-dap",
        "status": "ok",
        "message": "Selected backend: lldb-cli; lldb-dap available: Yes"
      },
      {
        "name": "manifest-tools",
        "status": "ok",
        "message": "Total tools: 82; Workflows: 15"
      },
      {
        "name": "runtime-registration",
        "status": "warning",
        "message": "Runtime registry unavailable."
      },
      {
        "name": "xcode-ide-bridge",
        "status": "ok",
        "message": "Workflow enabled: No; Connected: No; Proxied tools: 0"
      },
      {
        "name": "sentry",
        "status": "ok",
        "message": "Enabled: Yes"
      }
    ]
  }
}

Editor/Client

Codex desktop invoking the standalone Homebrew CLI. The isolated reproduction runs directly in Node.js.

MCP Server Version

2.7.0, Homebrew installation. This is also the latest published release at the time of this report.

LLM

Codex / GPT-6 during investigation. No model is involved in the reproduction.

Steps to Reproduce

Save the following as reproduce.mjs. It connects the shipped event parser to the shipped run-state accumulator, as the production pipeline does. The input contains two ordinary, non-parameterized tests: one passes; one fails with three assertion issues.

import { pathToFileURL } from 'node:url';

const moduleRoot = process.argv[2];
const { createXcodebuildEventParser } = await import(
  pathToFileURL(`${moduleRoot}/utils/xcodebuild-event-parser.js`)
);
const { createXcodebuildRunState } = await import(
  pathToFileURL(`${moduleRoot}/utils/xcodebuild-run-state.js`)
);
const events = [];
const state = createXcodebuildRunState({
  operation: 'TEST',
  onEvent: event => events.push(event),
});
const parser = createXcodebuildEventParser({
  operation: 'TEST',
  onEvent: event => state.push(event),
});
parser.onStdout([
  '✔ Test "passes" passed after 0.001 seconds.',
  '✘ Test "fails" recorded an issue at CountsTests.swift:10:3: Expectation failed: first',
  '✘ Test "fails" recorded an issue at CountsTests.swift:11:3: Expectation failed: second',
  '✘ Test "fails" recorded an issue at CountsTests.swift:12:3: Expectation failed: third',
  '✘ Test "fails" failed after 0.001 seconds with 3 issues.',
  '✘ Test run with 2 tests in 1 suite failed after 0.002 seconds with 3 issues.',
].join('\n') + '\n');
parser.flush();
state.finalize(false);
for (const { kind, fragment, ...data } of events) {
  if (['test-case-result', 'test-progress', 'build-summary'].includes(fragment)) {
    console.log(JSON.stringify({ event: `${kind}.${fragment}`, ...data }));
  }
}

For a Homebrew installation, run:

node reproduce.mjs "$(brew --prefix xcodebuildmcp)/libexec/build"

For another installation, pass the package's build directory instead.

The original simulator run used xcodebuildmcp simulator test ... --output jsonl with progress: true. Its raw Swift Testing summary was:

✘ Test run with 16 tests in 1 suite failed after 0.427 seconds with 22 issues.

The retained .xcresult summary independently reports 8 passed, 8 failed, 0 skipped. There are 16 named JSONL case results, also 8 passed and 8 failed. Some original tests were parameterized; these are named-test counts, not individual argument executions. The minimal reproduction avoids parameterization entirely.

Expected Behavior

For the minimal reproduction, retain both case-result events and all three failure diagnostics, but report progress completed: 2, failed: 1, skipped: 0 and terminal counts totalTests: 2, passedTests: 1, failedTests: 1, skippedTests: 0.

Assertion/diagnostic counts must remain distinct from test counts. Progress, case results, and the terminal summary should use a consistent counting unit.

Actual Behavior

The installed 2.7.0 modules emit:

{"event":"test-result.test-case-result","operation":"TEST","test":"passes","status":"passed","durationMs":1}
{"event":"test-result.test-progress","operation":"TEST","completed":1,"failed":0,"skipped":0}
{"event":"test-result.test-case-result","operation":"TEST","test":"fails","status":"failed","durationMs":1}
{"event":"test-result.test-progress","operation":"TEST","completed":2,"failed":3,"skipped":0}
{"event":"test-result.build-summary","operation":"TEST","status":"FAILED","totalTests":3,"passedTests":0,"failedTests":3,"skippedTests":0}

In the original run, progress was completed: 16, failed: 22. The terminal summary was totalTests: 25, passedTests: 0, failedTests: 25. Replaying its raw output plus XcodeBuildMCP's own xcresult-failure enrichment reproduces that exact summary: 22 parsed issue diagnostics become 25 unique diagnostic fragments after enrichment.

The owning code paths are:

• parseSwiftTestingRunSummary assigns the summary's issues count to failed.
• createXcodebuildEventParser suppresses native failed-result increments and then uses that summary value for failed progress.
• createTestSummaryFragment takes Math.max(state.failedTests, state.testFailures.length), conflating diagnostic fragments with failed tests again.

This report concerns the JSONL stream. The structured domain result has a separate xcresult-summary path; I am not claiming that every output mode reports these counts.

Error Messages

The downstream runner correctly rejected the original contradictory stream with:

test-result.build-summary: failed progress contradicts unique failed test-case results

That validation error is a consequence; the isolated upstream replay reproduces the wrong numbers before any downstream consumer runs.

Related history: #331 requested parser coverage including multiple expectations per test and was closed for inactivity; #374 documents aggregate parameterized-test entries; #389 changed progress reconciliation. I found no current issue reporting this concrete contradiction. Original downstream report: https://github.com/dpearson2699/ios-options-wheel-tracker/issues/1055.