Spec tracking check should validate the submitted SDK release, not first-release-after-spec
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 52/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Quiet
- Tech stack
- github, typescript
- Domain
- testing-qa
Research direction
Start in src/tier-check/checks/spec-tracking.ts and trace how the tiering submission currently supplies release and spec information. Define the inputs and validation flow needed for the submitted SDK release, then verify the release exists and that the measured gap uses the claimed spec version and 30-day threshold; the Java SDK assessment in issue #2301 provides the motivating case.
Written by the indexing model from the issue text.
Description
Problem
The current spec tracking check (src/tier-check/checks/spec-tracking.ts) finds the first SDK release published after the latest spec release and measures the time gap between them. This doesn't validate anything meaningful for tiering:
- It doesn't know which SDK version is being submitted for evaluation
- The "first release after spec" could be a totally unrelated bugfix/patch
- It's disconnected from the spec version the submission claims conformance against
For example, the Java SDK tiering submission shows "9d gap" — but that just means some SDK release happened 9 days after the latest spec release, not that the submitted version (1.0.0) was released within 30 days of the spec version (2025-06-18) it's being evaluated against.
Proposed Change
The spec tracking check should:
- Accept the submitted SDK version (or release tag) as input
- Accept the target spec version being claimed
- Verify the submitted SDK release exists on GitHub
- Measure the gap between the target spec release date and the submitted SDK release date
- Pass if the SDK release was within 30 days of that spec version's release
This ties the check to the actual submission rather than being a context-free scrape of GitHub releases.
Context
Came up during review of the Java SDK Tier 2 assessment (https://github.com/modelcontextprotocol/modelcontextprotocol/issues/2301).
- Dominant language
- TypeScript
- Stars
- 127
- Forks
- 101
- Avg merge
- 4d 7h
- Merged PRs (30d)
- 6
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from modelcontextprotocol/conformance
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
modelcontextprotocol/conformance#315 · 1 comment ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
modelcontextprotocol/conformance#312 · 1 comment ·
-
Difficulty 3/5 1-2 days Newbie friendliness 55/100
-
Difficulty 4/5 3-5 days Newbie friendliness 40/100
-
Difficulty 3/5 1-2 days Newbie friendliness 65/100
All issues in modelcontextprotocol/conformance
Similar issues
-
VerificationGate: ATTRIBUTION quote guard never matches a normal quotation (\b around the quote) Open
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
danielmiessler/LifeOS#2234 ·
-
T: Bug
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
-
Difficulty 1/5 Under an hour Newbie friendliness 85/100
-
Mend: dependency security vulnerability untriaged
Difficulty 2/5 1-3 hours Newbie friendliness 70/100