Hacktoberfest 2026: những issue maintainer đã đánh dấu cho tháng Mười, đang mở và phù hợp người mới. Xem issue Hacktoberfest

Claude 5-generation models (adaptive_thinking: "required") never return readable reasoning text - only opaque envelope, despite billed reasoningTokens

Đang mở
#2,352 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Chưa có ai nhận issue này.

Đánh giá

Độ khó
4/5
Thời gian dự kiến
3-5 ngày
Mức phù hợp với người mới
42/100
Loại issue
Lỗi
Độ rõ ràng
Khá rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
github, vscode
Lĩnh vực
api, devtools

Hướng nghiên cứu

Bắt đầu với generated/rpc.d.ts và dist/types.d.ts để xem xét các kiểu reasoning và adaptive-thinking được cung cấp, sau đó so sánh đường đi của yêu cầu phiên với triển khai VS Code được mô tả trong issue. Xác minh cách các model bị ảnh hưởng được listModels() báo cáo và cách các sự kiện reasoning của assistant được cung cấp. Công việc được hoàn tất khi có thể thương lượng các model adaptive-thinking bắt buộc và reasoning có thể đọc được được cung cấp một cách đáng tin cậy, với các bài kiểm thử bao phủ đầu ra của sự kiện.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

enhancement

Summary

@github/copilot-sdk-driven sessions on Claude 5-generation models (claude-sonnet-5, claude-opus-5, claude-opus-4.8) never receive readable reasoning/thinking text (assistant.reasoning.content and assistant.message.reasoningText are always empty, assistant.reasoning_delta never fires), even though the model genuinely performs extended thinking (assistant.usage.reasoningTokens is real and substantial) and the request is not being blocked (contentFilterTriggered: false). Only the opaque/encrypted envelope (reasoningOpaque on assistant.message, byte-identical to reasoningId on assistant.reasoning) ever comes through.

The SDK exposes no configuration surface to influence this at all.

Environment

  • @github/copilot-sdk: 1.0.9
  • Copilot CLI (bundled runtime): 1.0.80, protocol 1.0.77
  • Models affected: claude-sonnet-5, claude-opus-5, claude-opus-4.8 (all report capabilities.supports.adaptive_thinking: "required" via listModels())
  • Models NOT affected: claude-sonnet-4.6 (adaptive_thinking: "optional") - not directly re-tested for this specific symptom, but flagged as a likely-safe comparison point; claude-haiku-4.5 (adaptive_thinking: "unsupported", and also doesn't support reasoningEffort at all)
  • GPT-family models (e.g. gpt-5.6-luna) are unaffected - reasoning streams and finalizes with full readable text correctly, confirmed live.

What we found, and how we verified it (no reverse engineering involved)

  1. listModels() reports adaptive_thinking: "required" for the affected models. Called client.listModels() directly and inspected the raw capabilities.supports object:

    {
      "id": "claude-sonnet-5",
      "capabilities": {
        "supports": {
          "adaptive_thinking": "required",
          "max_thinking_budget": 32000,
          "min_thinking_budget": 1024,
          ...
        }
      }
    }
    

    Per the SDK's own doc comment on AdaptiveThinkingSupport (generated/rpc.d.ts): "required" means "The model only accepts adaptive thinking and rejects thinking.type='enabled' with HTTP 400."

  2. There is no SDK-level way to request thinking: {type: 'adaptive'} (or any thinking mode) at all. A full-text search of the entire consumer-facing dist/types.d.ts for "thinking" or "adaptive" returns zero matches. SessionConfigBase.reasoningEffort and SessionConfigBase.reasoningSummary are the only reasoning-related session options exposed, and neither is documented as, or appears to actually be, a substitute for Anthropic's native thinking request parameter.

  3. The observed data is fully consistent with "adaptive thinking runs, but the runtime doesn't surface it as text for this negotiation path" rather than any kind of safety filtering:

    • assistant.usage.reasoningTokens is real and often substantial (26-495 tokens observed across many calls) - the model is genuinely spending tokens on extended thinking.
    • assistant.usage.contentFilterTriggered is false on every single affected call.
    • assistant.usage.apiEndpoint is /v1/messages (the Anthropic-native endpoint) on every call, both the ones that come back with readable text and the ones that don't - so it isn't a difference in which endpoint is used.
    • assistant.message.reasoningOpaque and assistant.reasoning.reasoningId are byte-for-byte identical for the same reasoning block - the same encrypted envelope is surfaced on two separate event fields, with no plaintext on either.
    • assistant.reasoning_delta never fires at all for these calls (confirmed via direct count, zero rows).
  4. We ruled out every other explanation we could think of via direct, controlled tests before landing here (documented in full in our own investigation notes, happy to share if useful): streaming session option, ephemeral-event/resume-replay behavior, sub-agent involvement, ephemeral vs. persisted event handling, enterprise/managed-settings policy (confirmed session.managed_settings_resolved/enforced never fire for any of our sessions), reasoning-token-count thresholds, and repository content/context (a controlled A/B test running an unrelated generic prompt inside the exact same repository that reliably showed 17/17 redacted results for real work came back fully clean, ruling out repo-specific policy or content-classification as the cause).

Why we believe this is specifically an SDK/runtime gap, not intended Anthropic behavior

The interactive copilot CLI TUI and the VS Code Copilot Chat extension both reliably show full reasoning text for the exact same account, models, and repository/content. VS Code's own (open source) implementation constructs the Anthropic /v1/messages request directly, explicitly setting thinking: { type: 'adaptive' } (or { type: 'enabled', budget_tokens } for non-required models) plus the interleaved-thinking-2025-05-14 beta header, and accumulates the raw thinking/signature deltas itself - it does not depend on any server-side summarization step. This strongly suggests the underlying model/API absolutely can and does return readable thinking text for these models when the request negotiates adaptive thinking correctly; the gap appears to be specific to how @github/copilot-sdk-driven sessions (and/or the CLI runtime backing them) negotiate or expose this for adaptive_thinking: "required" models.

Ask

  1. Could the SDK expose a session-level option to control Anthropic's thinking parameter directly (matching what VS Code's own client already sends), so consumers aren't dependent on an internal summarization step that doesn't appear to work for adaptive_thinking: "required" models?
  2. Alternatively/additionally, could assistant.reasoning/assistant.message reliably carry readable text for these models the way they already do for GPT-family reasoning models?

Happy to provide full raw event logs, or the request/response correlation IDs we've already captured on affected calls (interactionId, serviceRequestId / x-copilot-service-request-id, providerCallId / x-github-request-id, apiCallId) if useful for looking up server-side records.

Ngôn ngữ chính
Java
Star
10.5k
Fork
1.5k
Merge trung bình
1 ngày 9 giờ
Pull request đã merge (30 ngày)
130

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của github/copilot-sdk

Tất cả issue của github/copilot-sdk

Issue tương tự

Thêm issue về Java

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.