OAuth client sends discovery/registration requests with no User-Agent, so WAF-fronted servers (Cloudflare) 403 the whole flow

從 mcp/client/auth/utils.py 中的 create_oauth_metadata_request 和 create_client_registration_request 開始，接著追蹤 OAuthClientProvider.async_auth_flow 如何傳送它們的請求。使用 MockTransport 重現來將標頭與 client.get() 進行比較；完成的標準是 discovery 和 registration 請求不再遺失必要的 User-Agent，且透過 tests 或可比的本機 transport 驗證產生的 OAuth 流程。

Summary

OAuthClientProvider builds its OAuth discovery and registration requests as bare httpx.Request objects and sends them via client.send(). httpx merges a client's default headers only in build_request() (i.e. for client.get() / .post()), so these requests go out with no User-Agent and no Accept header.

Any MCP server behind a WAF that blocks user-agent-less traffic — Cloudflare's default bot management does — answers 403 to every one of them, making OAuth impossible to complete. The resulting error is also misattributed, which makes it hard to diagnose.

Reproduction

Against a Cloudflare-fronted MCP server (observed on https://mcp.services.biorender.com/mcp), with mcp==1.26.0:

POST /mcp                                        -> 401  (expected; carries www-authenticate)
GET  /.well-known/oauth-authorization-server     -> 403  <-- blocked
GET  /.well-known/oauth-protected-resource/mcp   -> 403  <-- blocked
GET  /.well-known/oauth-protected-resource       -> 403  <-- blocked
GET  /.well-known/oauth-authorization-server     -> 403  <-- blocked
POST /register                                   -> 403  <-- note the path

Isolating the trigger with curl against that same discovery URL:

normal curl (UA + Accept present)   -> 200
curl -H 'User-Agent:' -H 'Accept:'  -> 403
curl -H 'User-Agent:'               -> 403   # UA alone is the trigger
curl -H 'Accept:'                   -> 200   # Accept is not

And confirming the header loss is structural, not server-specific:

import httpx, asyncio

seen = {}
def handler(req):
    seen[req.url.path] = dict(req.headers)
    return httpx.Response(200, json={})

async def main():
    async with httpx.AsyncClient(transport=httpx.MockTransport(handler)) as c:
        await c.get('https://x.test/normal')                       # client.get()
        await c.send(httpx.Request('GET', 'https://x.test/bare'))   # what the SDK does

asyncio.run(main())
print('client.get() :', sorted(seen['/normal']))
print('client.send():', sorted(seen['/bare']))

client.get() : ['accept', 'accept-encoding', 'connection', 'host', 'user-agent']
client.send(): ['host']

Secondary problem: the failure is misreported

The 403 lands on metadata discovery, so context.oauth_metadata stays None. create_client_registration_request then falls back to urljoin(auth_base_url, "/register") — but this server's actual registration endpoint is /oauth/register, which its discovery document advertises correctly and which works fine when called with a User-Agent.

So the error surfaced to the user is Registration failed: 403 <cloudflare html>, pointing at a registration request to a path that was never the right one, when the real failure was four requests earlier. Anyone debugging this starts at the wrong end. (This is arguably worth addressing independently: a discovery failure could be reported as a discovery failure rather than silently degrading into a guessed-path registration.)

Affected code

• mcp/client/auth/utils.py:211 — create_oauth_metadata_request -> Request("GET", url, headers={MCP_PROTOCOL_VERSION: ...})
• mcp/client/auth/utils.py:215-227 — create_client_registration_request -> Request("POST", registration_url, json=..., headers={"Content-Type": "application/json"})

Both construct Request outside the client, so neither inherits client defaults.

Suggested fix

Set a default User-Agent (e.g. mcp-python-sdk/<version>) on the requests these helpers build, or have OAuthClientProvider.async_auth_flow stamp one onto each request it yields when absent. Adding Accept: application/json to discovery would also be reasonable, though it is not what triggers the block here.

Workaround

Downstream clients can pass an httpx_client_factory that attaches a request event hook, since hooks fire for every request the client sends, including the auth flow's bare ones:

async def _stamp_user_agent(request: httpx.Request) -> None:
    if "user-agent" not in request.headers:
        request.headers["user-agent"] = "my-app/1.0"

def factory(headers=None, timeout=None, auth=None):
    client = create_mcp_http_client(headers=headers, timeout=timeout, auth=auth)
    client.event_hooks = {"request": [_stamp_user_agent], "response": []}
    return client

Environment

• mcp 1.26.0, httpx 0.28.1, httpx-sse 0.4.1, anyio 4.11.0
• Python 3.11.13, Linux