SEP-2575: remaining conformance tests
メンテナーはふだん 7 日以内に返信
まだ誰も着手していません。
評価
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 初心者へのやさしさ
- 38/100
- issue の種類
- 機能追加
- 明瞭さ
- おおむね明確
- 活発さ
- 静か
- 技術スタック
- typescript
- 領域
- api, testing-qa
調査の方向性
まずトレーサビリティの更新と src/scenarios/client/request-metadata.ts にある既存のシナリオから始め、次に everythingClient.ts を調査して、現在の TypeScript SDK の制限を理解します。セクション B で提案されている 7 つのシナリオを既存の適合性パターンと照合し、新しい SUT 機能なしで実装できるものを特定します。実行可能なシナリオを追加するか、残りのチェックを除外または延期する理由を記録すれば完了です。
索引モデルが issue の本文から書いたものです。
説明
Status as of the 70b2800d traceability refresh (run against typescript-sdk@main). 8 of 26 declared requirements are tested, 18 are untested. Of the untested 18, 7 have a scenario already written but the reference SDK can't run it yet, and 11 have no scenario at all.
A. Scenario written — blocked on typescript-sdk support
src/scenarios/client/request-metadata.ts (#270) emits all 7 of these, but everythingClient.ts on typescript-sdk@main has no request-metadata handler — it logs Unknown scenario and exits before any HTTP request is made, so no checks are emitted. These will flip to tested once the SDK has a SEP-2575-aware conformance client, or we fix the checker to be more correct.
So no scenario action needed on these, this is a tracking bug.
| Check | Requirement | Level |
|---|---|---|
sep-2575-client-populates-meta |
Every client request includes protocolVersion, clientInfo, clientCapabilities in _meta |
MUST |
sep-2575-http-client-sends-version-header |
Every POST includes MCP-Protocol-Version header |
MUST |
sep-2575-http-version-header-matches-meta |
Header value matches _meta.…/protocolVersion |
MUST |
sep-2575-client-declares-roots-capability |
Roots-capable clients declare roots in _meta.…/clientCapabilities |
MUST (cond.) |
sep-2575-client-declares-sampling-capability |
Sampling-capable clients declare sampling |
MUST (cond.) |
sep-2575-client-declares-elicitation-capability |
Elicitation-capable clients declare elicitation |
MUST (cond.) |
sep-2575-client-retry-supported-version |
Client retries with a mutually-supported version after UnsupportedProtocolVersionError |
SHOULD |
B. No scenario yet — seems straightforward to write
| Check | Requirement | Level | Notes |
|---|---|---|---|
sep-2575-server-sends-subscription-ack |
notifications/subscriptions/acknowledged is the first message on a subscriptions/listen stream |
MUST | Open a listen stream, assert first frame |
sep-2575-server-tags-subscription-id |
Listen-stream notifications carry _meta.…/subscriptionId |
MUST | Open listen stream, trigger a notification, inspect _meta |
sep-2575-server-honors-notification-filter |
Server doesn't send notification types the client didn't request | MUST NOT | Open with a narrow filter, trigger out-of-filter event, assert silence; needs a trigger tool on the SUT server, or could be a lightweight check on recieved notification types |
sep-2575-http-server-no-independent-requests-on-stream |
Request stream contains only IncompleteResult, never independent JSON-RPC requests |
MUST NOT | Call a tool that needs sampling/elicitation, scan stream frames |
sep-2575-server-no-log-without-loglevel |
No notifications/message for requests that didn't set _meta.…/logLevel |
MUST NOT | Call a logging tool with no logLevel, assert no log frames. This one is a bit weird because it should be true on every request that you don't get logs, but having a tool that logs w/ capability but doesn't log w/o seems good. |
sep-2575-server-sends-prompts-list-changed-on-subscription |
List-changed-capable servers notify listen streams with promptsListChanged: true |
SHOULD | Needs the SUT server to expose a way to mutate its prompt list |
sep-2575-server-sends-tools-list-changed-on-subscription |
Same, for tools | SHOULD | Same. |
C. No scenario yet — observability / spec-wording questions
These describe internal server state rather than wire behavior, or depend on the SUT having a way to trigger the precondition. These we can likely exclude, or revisit their spec language if we want them to be testable. Maybe there are more creative scenarios than I can think of to test these, but I'm inclined to exclude them.
| Check | Requirement | Level | Concern |
|---|---|---|---|
sep-2575-server-stateless-no-prior-context |
Server doesn't treat connection/process identity as a session proxy | MUST NOT | Partially covered already: sep-2575-request-meta-invalid-missing-{meta,protocol-version,client-info,client-capabilities} verify the server rejects requests with incomplete _meta (i.e. won't fall back to remembered state) — but that's per-request, not cross-connection. A full scenario should also verify request #2 on a fresh connection succeeds with full _meta. Spec could phrase in terms of an observable consequence. |
sep-2575-server-stateless-no-connection-reuse-required |
Related operations work across separate connections | MUST NOT | Similar to above, this one seems difficult to test. We either should exclude it or update the spec language to make it clearer what we're |
sep-2575-http-server-disconnect-is-cancel |
Closing the SSE stream is treated as cancellation | MUST | "Treated as" is internal state. Once the stream closes there's no channel to observe it; only indirectly testable via …-stops-on-cancel. |
sep-2575-http-server-stops-on-cancel |
Server stops work and sends no further messages for the cancelled request | SHOULD + MUST NOT | "Stop work as soon as practical" is unobservable from a black-box harness. The "no further messages" half is testable only if the server might also fan out to a subscriptions/listen stream. |
- 主要言語
- TypeScript
- スター
- 130
- フォーク
- 107
- 平均マージ
- 8日 19時間
- マージ済み PR(30日)
- 2
環境構築
- Dockerfile・Docker Compose ファイルなし
- プルリクエストのテンプレートなし
- コントリビューションガイドを読む
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
modelcontextprotocol/conformance のほかの issue
-
難易度 2/5 1〜3時間 初心者へのやさしさ 88/100
modelcontextprotocol/conformance#531 · コメント 1 件 ·
メンテナーはふだん 7 日以内に返信
-
server-stateless: 500 ms whole-request deadline in no-log-without-loglevel reports slow servers as failures対応中かも @birbprophet が 9 日前に担当しました。 オープン
難易度 2/5 1〜3時間 初心者へのやさしさ 68/100
modelcontextprotocol/conformance#530 ·
メンテナーはふだん 7 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 75/100
modelcontextprotocol/conformance#519 ·
メンテナーはふだん 7 日以内に返信
-
bug
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
modelcontextprotocol/conformance#315 · コメント 1 件 ·
メンテナーはふだん 7 日以内に返信
-
bug
難易度 2/5 1〜3時間 初心者へのやさしさ 68/100
modelcontextprotocol/conformance#312 · コメント 1 件 ·
メンテナーはふだん 7 日以内に返信
modelcontextprotocol/conformance の issue をすべて見る
似ている issue
-
enhancement
難易度 2/5 1〜3時間 初心者へのやさしさ 74/100
メンテナーはふだん 1 日以内に返信
-
[Bug] remember() with special characters in namespace hangs until timeout instead of returning 400オープンbug
難易度 2/5 1〜3時間 初心者へのやさしさ 72/100
MystenLabs/MemWal#1133 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
bug user-priority/P2
難易度 1/5 1時間未満 初心者へのやさしさ 92/100
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1時間未満 初心者へのやさしさ 88/100
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 88/100
Effect-TS/effect#8881 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信