Add skill-level spans to OpenTelemetry traces
まだ誰も着手していません。
評価
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 初心者へのやさしさ
- 45/100
- issue の種類
- 機能追加
- 明瞭さ
- おおむね明確
- 活発さ
- 静か
- 技術スタック
- java
- 領域
- cli, observability
調査の方向性
まず、Copilot CLI v1.0.60 での skill 呼び出し中に execute_tool スパンがどこで作成されるかを追跡し、次に parentSpanId と skill コンテキストが skill ツールおよび preToolUse/postToolUse フックを通じてどのように受け渡されるかを調べます。トレースに skill.name を持つ execute_skill スパンと、正しくネストされた子ツールスパンが現れるか、ネストが実現できない場合にそれらのツールスパンが一貫して帰属されれば完了です。
索引モデルが issue の本文から書いたものです。
説明
Describe the feature or problem you'd like to solve
When a Copilot CLI agent invokes a project skill, the resulting tool calls (bash, glob, etc.) are emitted as flat children of the root invoke_agent span. There is no intermediate span representing the skill invocation itself, and no skill.name attribute on the child tool spans. This makes it impossible to attribute tool calls to the skill that triggered them using the OTEL trace data.
Current trace structure:
invoke_agent (root)
├── execute_tool glob ← no skill attribution
├── execute_tool bash ← no skill attribution
├── execute_tool bash ← no skill attribution
├── chat model
└── chat model
The skill name only appears in github.copilot.context.skills on the invoke_agent span, which lists all available skills — not the one that was actually invoked.
Proposed solution
Introduce an execute_skill (or similar) span that:
- Wraps the tool calls triggered by the skill invocation, so they become children of the skill span rather than the root agent span.
- Carries a
skill.nameattribute identifying which skill was executed. - Is a child of the
invoke_agentspan, preserving the existing hierarchy.
Example desired trace:
invoke_agent (root)
├── chat model
├── execute_skill "my-skill" ← NEW
│ ├── execute_tool glob ← child of skill span
│ ├── execute_tool bash ← child of skill span
│ └── execute_tool bash ← child of skill span
├── chat model
└── execute_tool bash ← not part of a skill
Alternatively (minimal version): if nesting is not feasible, adding a skill.name attribute to each execute_tool span that was triggered within a skill invocation context would also solve the problem.
How will it benefit GitHub Copilot CLI and its users?
- Skill-level latency measurement: Users can measure how long a skill takes end-to-end, rather than manually summing individual tool call durations.
- Tool call attribution: Clearly distinguish which tool calls belong to a skill invocation vs. general agent reasoning, enabling targeted debugging and optimization.
- Faster failure diagnosis: When a skill fails, users can immediately identify which child tool call failed without reading through command arguments.
- Aggregated dashboards: Teams can build per-skill usage and performance dashboards across sessions, which is essential for monitoring custom skill reliability at scale.
Example prompts or workflows
-
Debugging a failed skill: A custom skill fails intermittently. The user exports OTEL traces and filters for
execute_skillspans with error status. They drill into the childexecute_toolspans to see exactly which bash command failed — without reading every tool call in the session. -
Measuring skill performance over time: A team ships a custom skill and wants to track its p50/p95 latency across sessions. They query their trace backend for
execute_skillspans whereskill.name = "my-skill"and chart duration over time. Today this is impossible without manually parsing tool call arguments. -
Attributing token/tool usage to skills: A user runs a session where the agent invokes three different skills. They want to see how many tool calls each skill made and how much time each consumed. With skill-level spans, this is a simple trace query. Without them, all tool calls are indistinguishable siblings under
invoke_agent. -
Building an observability dashboard: A team sends OTEL traces to their backend and builds a dashboard showing skill invocation frequency, success rate, and latency. This requires a reliable
skill.nameattribute or dedicated span — inferring skill boundaries from bash command content is fragile and breaks when scripts change. -
Auditing skill usage in CI/automation: In cloud agent jobs, a team wants to verify that the correct skills were invoked and completed successfully. Skill-level spans would make this a simple trace query rather than log parsing.
Additional context
- Hooks (
preToolUse/postToolUse) also don't carry skill context —toolNameis"bash"/"glob"etc., with no reference to the parent skill. AddingskillNameto hook payloads would be a complementary improvement. - The
skilltool invocation itself may appear as anexecute_tool skillspan, but subsequent tool calls triggered by the skill are not linked to it viaparentSpanId. - Observed on Copilot CLI v1.0.60.
- 主要言語
- Java
- スター
- 10.5k
- フォーク
- 1.5k
- 平均マージ
- 1日 9時間
- マージ済み PR(30日)
- 131
コントリビューションガイド
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
github/copilot-sdk のほかの issue
-
agentic-workflows
難易度 2/5 1〜3時間 初心者へのやさしさ 68/100
github/copilot-sdk#2709 · コメント 1 件 ·
-
難易度 1/5 1時間未満 初心者へのやさしさ 78/100
github/copilot-sdk#2673 ·
-
bug testing
難易度 2/5 1〜3時間 初心者へのやさしさ 75/100
github/copilot-sdk#2628 ·
-
agentic-workflows
難易度 2/5 1〜3時間 初心者へのやさしさ 68/100
github/copilot-sdk#2627 · コメント 1 件 ·
-
agentic-workflows
難易度 2/5 1〜3時間 初心者へのやさしさ 74/100
github/copilot-sdk#2493 ·
github/copilot-sdk の issue をすべて見る
似ている issue
-
awaiting triage bug Causes friction Hop Gui P1 P2 Transforms
難易度 2/5 1〜3時間 初心者へのやさしさ 75/100
-
難易度 2/5 1〜3時間 初心者へのやさしさ 75/100
apache/flink-agents#1152 ·
-
難易度 2/5 1〜3時間 初心者へのやさしさ 75/100
-
難易度 2/5 1〜3時間 初心者へのやさしさ 70/100
jenkinsci/blueocean-plugin#5417 ·
-
難易度 2/5 1〜3時間 初心者へのやさしさ 75/100
objectionary/eo-graphs#75 ·