MCP spec conformance: 10 requirements violated (via @hasmcp/mcp-spec-test) — spec 2026-07-28

Open
#755 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
38/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Active
Tech stack
docker, python

Research direction

Start with examples/qr-server/server.py and reproduce the report using the Docker or npx @hasmcp/mcp-spec-test commands provided. Verify whether the stdio server exits before the MCP handshake in the intended runtime, separating environment or uv invocation failures from a server defect. Done means the cause is confirmed and the conformance result is documented or the server reaches a completed handshake.

Written by the indexing model from the issue text.

Description

Running the automated @hasmcp/mcp-spec-test conformance suite against the examples/qr-server/server.py stdio server (spec revision 2026-07-28) found that the server process exits before completing the MCP handshake, which causes 10 requirements to fail (and a further 22 to be skipped, since they depend on a completed handshake). This may be an environment/dependency artifact of running under uv run from a remote URL inside the test container rather than a real bug in the server itself — flagging it here so a maintainer with direct access to the intended runtime can confirm either way.

Conformance report

MCP 2026-07-28 conformance report

Verdict: not conformant — 10 requirements violated.

Target uv run https://raw.githubusercontent.com/modelcontextprotocol/ext-apps/HEAD/examples/qr-server/server.py --stdio
Transport stdio
Revision tested 2026-07-28
Revisions supported 2026-07-28, 2025-11-25
Passed 4
Failed 10
Not verified 22
Cases applied 36
Duration 27848ms
Generated 2026-08-24T09:21:16.771Z
Tool @hasmcp/mcp-spec-test 0.1.1

Failed (10)

the server deviates from the spec here

server/discover
  • server/discover is answered without a session or handshake
    expected 200, got 0: {"_raw":"server exited"}

0 !== 200

  • server/discover advertises the versions the server can serve
    Cannot read properties of undefined (reading 'supportedVersions')
  • server/discover is a CacheableResult with usable cache hints
    Cannot read properties of undefined (reading 'ttlMs')
  • server/discover reports server identity and capabilities
    Cannot read properties of undefined (reading 'capabilities')
  • server/discover is stable across calls within its own TTL
    Cannot read properties of undefined (reading 'supportedVersions')
  • server/discover advertises a revision this suite supports
    Cannot read properties of undefined (reading 'supportedVersions')
Version negotiation
  • a request with no version at all is served on the default
    a version-less request must be served, got 0: {"_raw":"server exited"}

0 !== 200

Official SDK interop
  • a stock official-SDK client completes the handshake
    MCP error -32000: Connection closed
  • the handshake settles on a revision inside the supported window
    initialize must return a protocolVersion, got undefined
  • a stock official-SDK client can list tools
    MCP error -32000: Connection closed

Not verified (22)

skipped; a skip is not a pass

Capability methods
  • tools/list returns schema-conformant tools
    cannot determine what the target supports — server/discover returned no result (status 0)
  • tools/call on an unknown tool is an error, not a crash
    cannot determine what the target supports — server/discover returned no result (status 0)
  • tools/call returns a schema-conformant CallToolResult
    cannot determine what the target supports — server/discover returned no result (status 0)
  • prompts/list returns schema-conformant prompts
    cannot determine what the target supports — server/discover returned no result (status 0)
  • prompts/get returns messages with a role and content
    cannot determine what the target supports — server/discover returned no result (status 0)
  • resources/list returns schema-conformant resources
    cannot determine what the target supports — server/discover returned no result (status 0)
  • resources/templates/list returns schema-conformant templates
    cannot determine what the target supports — server/discover returned no result (status 0)
  • resources/read returns contents for every sampled resource
    cannot determine what the target supports — server/discover returned no result (status 0)
  • resources/read on an unknown uri is an error
    cannot determine what the target supports — server/discover returned no result (status 0)
  • following nextCursor terminates and does not repeat a page
    cannot determine what the target supports — server/discover returned no result (status 0)
  • an invalid pagination cursor is rejected (SHOULD)
    cannot determine what the target supports — server/discover returned no result (status 0)
Version negotiation
  • a version declared in _meta is accepted
    cannot determine what the target supports — server/discover returned no result (status 0)
  • an unsupported version is rejected with the supported list
    cannot determine what the target supports — server/discover returned no result (status 0)
  • clientInfo is optional (SHOULD, not MUST)
    cannot determine what the target supports — server/discover returned no result (status 0)
Result envelope
  • every result carries the required resultType
    cannot determine what the target supports — server/discover returned no result (status 0)
  • cacheable list results carry the schema-required cache hints
    cannot determine what the target supports — server/discover returned no result (status 0)
  • results identify the server in _meta
    cannot determine what the target supports — server/discover returned no result (status 0)
  • a client on an older version receives no newer-revision fields
    cannot determine what the target supports — server/discover returned no result (status 0)
subscriptions/listen
  • subscriptions/listen acknowledges only the opted-in notification types
    cannot determine what the target supports — server/discover returned no result (status 0)
  • the acknowledgment carries the subscription id for correlation
    cannot determine what the target supports — server/discover returned no result (status 0)
  • a listen requesting no notification types is not a subscription to everything
    cannot determine what the target supports — server/discover returned no result (status 0)
  • a cancelled subscription ends with a conformant teardown result, if it sends one
    cannot determine what the target supports — server/discover returned no result (status 0)

Passed (4)

checked and conformant

server/discover
  • the suite is reading a schema that matches the features it selected
Result envelope
  • schema sanity: the envelope fields match the features selected
Official SDK interop
  • the official SDK does not yet implement the newest revision
subscriptions/listen
  • schema sanity: SubscriptionsListenResult requires _meta and resultType

How to reproduce

With Docker:

docker pull hasmcp/mcp-spec-test:latest
docker run --rm hasmcp/mcp-spec-test -c "uv run https://raw.githubusercontent.com/modelcontextprotocol/ext-apps/HEAD/examples/qr-server/server.py --stdio" --spec-version 2026-07-28

Without Docker (npx):

npx @hasmcp/mcp-spec-test@latest -c "uv run https://raw.githubusercontent.com/modelcontextprotocol/ext-apps/HEAD/examples/qr-server/server.py --stdio" --spec-version 2026-07-28

This report was generated as part of an automated conformance sweep across public MCP server repos. The server also supports spec revision 2025-11-25, which showed a different (smaller) set of failures and is reported separately. If this turns out to be a false positive specific to the test container's environment (e.g. missing uv/network access to fetch the remote script, or a uv run <url> invocation quirk rather than a real handshake bug), feel free to close this — happy to help re-verify with more detail if useful.

Dominant language
TypeScript
Stars
2.9k
Forks
387
Avg merge
3h 21m
Merged PRs (30d)
6

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from modelcontextprotocol/ext-apps

All issues in modelcontextprotocol/ext-apps

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.