skill tool is almost never invoked; references/ content is effectively unreachable

Start by finding the CLI code paths for the skill tool, skill discovery, telemetry, and file reads under a skill's references/ directory. Reproduce the reported measurement approach with installed relevant skills, then add tracking for skill invocations and reference reads. Done means maintainers can see these rates over time and have enough data to decide whether references/ indirection should remain.

Description

The skill tool is effectively never invoked in practice, even in agent runs where relevant skills are installed and discoverable. Content that is pushed out of a skill's main body into its references/ directory (for "progressive disclosure") is essentially never retrieved either, so in practice it behaves as if that content had been deleted rather than deferred.

Steps to Reproduce

1. Instrument an agent's tool-call telemetry across a batch of independent sessions where one or more skills are installed and relevant to the task at hand.
2. Count skill tool invocations and file reads under each skill's references/ directory.
3. Compare pass/fail outcomes for tasks with a larger vs. smaller skill body resident in context.

Expected Behavior

• Skills that are relevant to a task should be invoked via the skill tool at a reasonable rate.
• When a skill's SKILL.md points to files under references/ for details, those files should actually get opened by the agent when needed — that is the entire premise of progressive disclosure.

Actual Behavior

Across two independent instrumented evaluation sets:

• 0 skill invocations across 40 sessions in one set.
• 6 skill invocations across 28 sessions in another set.

Related: files under a skill's references/ directory were opened only 4 times across 40 sessions. In one arm of the evaluation, not a single one of the six reference files that a shipped skill points to was ever opened.

This implies that content moved out of a skill body into references/ is effectively deleted rather than deferred, because the retrieval step that progressive disclosure depends on essentially never happens.

Supporting datapoint: in a controlled comparison, removing roughly 2,300 tokens of skill body text raised the task pass rate (45% → 70%), which is consistent with that content not being used productively even when it is resident in context.

Ask

• Instrument skill-invocation and skill-reference-read rates in telemetry so this can be tracked over time.
• Reconsider whether references/-based indirection is worth its cost versus just inlining a shorter skill body, given that the deferred content is apparently not being retrieved when needed.

Environment

• OS: macOS / Linux (evaluation ran across multiple platforms)
• CLI version: current