JSON Schema/OpenAPI: AsyncAPI CLI `validate` over examples/ is a schema-examples check; `yarn test` → git-dependency program should be unknown; SchemaStore's tsconfig schemas taken as TS configs; library-driven validation harnesses

This issue bundles several classifier mistakes. Start with project_validator/ecosystems/javascript/analysis.py (TSCONFIG_FILES around lines 25-28), project_validator/ecosystems/openapi.py (ASYNCAPI_CLI near line 189) and json_schema.py's _validator, then the corpus expectations in corpus/projects.toml. Done means the corpus results for the SchemaStore, asyncapi/spec, OAI and JSON Schema repos match the expected outcomes listed, with the tests passing.

asyncapi/spec validates every document under examples/ with the AsyncAPI CLI on every PR (what the JSON Schema profile accepts as "a validator run on an examples directory") → WARN "no check of the schemas' examples". OAI/OpenAPI-Specification's yarn test expands to oai-spec-test, a program from a git dependency: the message says "no CI configuration runs … (or npm test)" when CI runs npm test — unknown, not FAIL. SchemaStore's src/schemas/json/tsconfig.json/jsconfig.json are the schemas for those files but type-checking credits "tsc (jsconfig.json, src/schemas/json/jsconfig.json, …) runs". All six JSON Schema repos and asyncapi/spec validate with a JSON Schema library driven by a project script (the dominant pattern of the ecosystem); a gap worth documenting.

Evidence from the corpus audit

SchemaStore/schemastore / type-checking

• kind: MESSAGE
• actual: PASS: tsc (jsconfig.json, src/schemas/json/jsconfig.json, src/schemas/json/tsconfig.json and 2 more) runs /
  consider enabling strict mode: "strict": true in the compilerOptions of tsconfig.json or deno check
• expected: PASS naming jsconfig.json only, with the strict-mode bonus: src/schemas/json/jsconfig.json and
  src/schemas/json/tsconfig.json are the JSON Schemas for those config files (and the "2 more" are
  src/test/tsconfig/tsconfig.json, src/test/jsconfig/jsconfig.json fixtures); the real config is strict.
• evidence: jsconfig.json:7: "strict": true; package.json:17: "typecheck": "tsc --project ./jsconfig.json",;
  .github/workflows/validate.yml:20: - run: 'npm run typecheck'; src/schemas/json/tsconfig.json:2-3:
  "$schema": "http://json-schema.org/draft-07/schema#", "$id": "https://json.schemastore.org/tsconfig.json" (a
  schema). project_validator/ecosystems/javascript/analysis.py:25,28: TSCONFIG_FILES = ("tsconfig.json", "jsconfig.json") / files=(*TSCONFIG_FILES, "tsconfig.*.json") — any file so named anywhere is a TypeScript
  setting, and one non-strict file among them cancels the bonus.
• repro: jsconfig.json with "strict": true, schemas/tsconfig.json being a JSON Schema (has $schema), CI runs
  tsc --project jsconfig.json; the PASS lists both and advises strict mode. Fixes: skip files whose content is a
  JSON Schema, and credit the config the --project/-p argument names.
• confidence: high

asyncapi/spec / schema-examples

• kind: FALSE_NEGATIVE
• actual: WARN: no check of the schemas' examples found in CI configuration / consider running in CI: spectral lint <description> (with a .spectral.yaml ruleset; …)
• expected: PASS: every PR runs the AsyncAPI CLI's validate over every document under examples/ (the repo's
  examples are AsyncAPI documents), which is what the JSON Schema profile accepts as "a validator run on an
  examples directory" (docs/ecosystems/json_schema.md, schema-examples); schema-validation already PASSes on
  the same command.
• evidence: .github/workflows/validate-examples.yml:22: find examples -type f \( -name "*.yml" -o -name "*.yaml" \) -not -path 'examples/social-media/common/*' | xargs -P 10 -L 1 asyncapi validate; ls examples | head →
  adeo-kafka-request-reply-asyncapi.yml, anyof-asyncapi.yml, …. Validator project_validator/ecosystems/openapi.py:189:
  ASYNCAPI_CLI = _linter("asyncapi", ASYNCAPI_ONLY, "asyncapi validate <description>", starts(("asyncapi", "validate"))) — no examples=True and no names_examples path (unlike json_schema.py's _validator).
• repro: workflow run: asyncapi validate examples/a.yaml with examples/a.yaml an AsyncAPI document.
• confidence: medium

OAI/OpenAPI-Specification / tests-run

• kind: MESSAGE
• actual: FAIL: JavaScript/TypeScript: tests found in 2 file(s), but no CI configuration runs jest, vitest, mocha, node --test, deno test, bun test or playwright (or npm test)
• expected: either PASS (CI runs the package's test script, which the message itself says counts) or an unknown:
  yarn test expands to oai-spec-test, a program from a git dependency the validator cannot read. Saying
  "no CI configuration runs … (or npm test)" when CI runs yarn test contradicts the evidence; vitest.config.mjs
  (with 100 % statement/line thresholds) shows the runner is vitest.
• evidence: .github/workflows/schema-tests.yaml:37: run: yarn test; package.json:25: "test": "oai-spec-test",;
  package.json:30: "@oai/build-infra": "git+https://github.com/OAI/build-infra.git#main";
  vitest.config.mjs:1-5: import { createVitestConfig } from "@oai/build-infra/vitest-config"; … thresholds: … statements: 100, lines: 100. Facts dump: "line": "oai-spec-test" with no Unknown.
• repro: package.json "test": "some-bin-from-a-dependency", CI yarn test: FAIL with a message naming npm test
  as acceptable. Either treat the package's test script as a run when its program is unknown (and say so), or
  record an Unknown("the program oai-spec-test") so strict mode errors.
• confidence: medium

all six JSON Schema repos, asyncapi/spec / schema-validation, schema-examples, tests-run (library-based harnesses)

• kind: UNSUPPORTED
• actual: FAIL schema-validation: no schema validation found in CI configuration and WARN schema-examples in
  SchemaStore, JSON-Schema-Test-Suite, compose-spec, CycloneDX, asyncapi/spec-json-schemas, OAI; FAIL/WARN
  tests-run where the harness is the test.
• expected: these repos do validate (and check valid/invalid examples) on every PR, but with a JSON Schema
  library driven by a project script, not a CLI the profiles know. Not a bug; the dominant pattern of the
  ecosystem. Tools and where they are used:
  • SchemaStore: node ./cli.js check (.github/workflows/validate.yml:22) runs ajv, ajv-draft-04 and
    @exodus/schemasafe over src/test/** and src/negative_test/** (package.json:28,32-33; cli.js:2110:
    check: Run all build checks).
  • JSON-Schema-Test-Suite: python -m tox → tox.ini:9: commands = {envpython} validation/check_validation_suite.py
    (jsonschema==4.19.0, validation/check_validation_suite.py:21: import jsonschema.validators); and
    .github/workflows/annotation-tests.yml:21: deno … annotations/annotation-tests.ts (:2:
    import { validate } from "npm:@hyperjump/json-schema/draft-07";).
  • asyncapi/spec-json-schemas: package.json:8: "test": "npm run build && vitest run && npm run validate:schemas",
    :17: "validate:schemas": "node scripts/validate-schemas.js" (ajv, ajv-draft-04 in devDependencies).
  • CycloneDX: tools/src/test/js/package.json:20-22 (node -- json-schema-lint-tests.js, ajv);
    tools/src/test/php/composer.json:8: "opis/json-schema": "2.6.0"; Java tests with networknt
    (JsonSchemaVerificationTest.java), XML Schema checks via JAXP; buf via docker (see the test.sh finding).
  • OAI: yarn test → oai-spec-test (hyperjump, vitest) over _archive_/schemas/v3.0/pass/*.yaml and fail/.
  • asyncapi/spec: .github/workflows/validate-embedded-examples.yml:22-23: run: npm run validate:examples /
    working-directory: ./scripts/validation/ → node embedded-examples-validation.js with @asyncapi/parser.
    Common pieces the profiles could learn: @hyperjump/json-schema and @exodus/schemasafe as JS validators (as
    dependencies imported by a *.test.* file that CI runs), the jsonschema/referencing Python libraries, and
    pass/+fail/ example directories (OAI, _archive_/schemas/v3.0/pass/) alongside valid//invalid/.
• evidence: as listed.
• repro: n/a (gap, not a bug)
• confidence: medium

Found by the regression-corpus audit (235 real repositories, one or more per ecosystem profile; corpus and expected results in corpus/). Evidence is quoted from the audited checkouts at the commits pinned in corpus/projects.toml; <checkouts>/<owner>__<repo> is such a checkout.

🤖 Generated with Claude Code

https://claude.ai/code/session_01ByTXNqDWdM7HzezrFtEvQt