Foundry Toolkit Copilot model provider drops token usage and always reports zero context tokens

Abierto
  • ¿Algo no está bien?
#940 1 comentario 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
48/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Activo
Stack tecnológico
github, javascript, vscode
Área
developer-experience, observability

Línea de trabajo

The report identifies provideTokenCount() and provideLanguageModelResponse in the bundled dist/extension.js, and mentions convertOpenAIChunkToChatChunk as the usage conversion point. Start by locating these in the unminified extension source and tracing how response chunks reach Copilot. Done means actual input, output, and cached token usage is forwarded and token counting no longer always returns zero; the issue names no tests to run.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

needs attention

Title: Foundry Toolkit Copilot model provider drops token usage and always reports zero context tokens

Environment
  • VS Code: 1.142.0
  • Integrated Copilot extension: 0.70.0
  • Microsoft Foundry Toolkit: 1.6.19
  • OS: Windows x64
  • Model: Azure Foundry deployment gpt-5.6-sol
  • Integration: Foundry Toolkit language model provider in VS Code Copilot Chat
Problem

When using an Azure Foundry model through Foundry Toolkit in GitHub Copilot Chat, token usage is not reported correctly.

Observed behavior:

  • Context window indicator always displays 0 / 1M tokens.
  • Agent Debug Logs report 0 input and output tokens.
  • Copilot /usage reports AI Units but no Azure API token usage.
  • OpenTelemetry file export is enabled, but no chat token usage metrics are recorded.

Azure Foundry itself processes and bills the model requests normally. A separate direct API test confirms that the service returns valid usage information, including prompt_tokens, completion_tokens, and total_tokens.

Root cause identified through local extension analysis

1. Token counting always returns zero

In the installed Foundry Toolkit bundle:

dist/extension.js, around line 2520:

async provideTokenCount(i,a,o){return 0}

This explains why the Copilot context window indicator always displays zero.

2. API usage is not forwarded to Copilot

The internal response conversion handles the API usage information:

convertOpenAIChunkToChatChunk, around line 4988.

However, the language model provider implementation:

provideLanguageModelResponse, around line 2517

appears to forward text and tool-call content without forwarding the token usage data.

In the integrated Copilot extension, around line 7013, the consumer expects usage information as a LanguageModelDataPart. When it is missing, input, output, and cache token counts are set to zero.

These findings are based on inspection of the installed, bundled JavaScript code and have not been independently validated against the unminified source.

Expected behavior
  • Forward actual API input, output, and cached token usage to VS Code Copilot.
  • Implement meaningful token counting instead of returning zero.
  • Make token usage available to the Agent Debug Logs and supported telemetry consumers.
  • Allow developers using Azure Foundry BYOK deployments to monitor token consumption per model request.
Additional observation

OpenTelemetry file export works and produces telemetry data, but the inspected file contains no model-request token usage metrics. The missing model-turn spans may require separate investigation.

Impact

Without reliable token reporting, developers cannot monitor or optimize API token consumption within Copilot Chat.

In our environment, 296 Azure requests consumed approximately 3.28 million tokens, while the Copilot UI continued displaying zero usage.

Request

Could you confirm whether forwarding token usage from Foundry Toolkit language model providers to Copilot is currently supported?

If supported, please investigate the missing usage forwarding and the provideTokenCount() implementation.

If not yet supported, please consider implementing it so that users of Azure Foundry BYOK models can monitor their actual token consumption.

Lenguaje dominante
JavaScript
Estrellas
2.1k
Forks
260
Merge medio
3 h 18 min
PR fusionados (30 d)
43

Preparar el entorno

Este proyecto no incluye contenedor de desarrollo, Dockerfile ni guía de contribución, así que la configuración corre por tu cuenta: empieza por su README y consulta nuestra guía para la primera contribución para los pasos generales.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de microsoft/foundry-dev-tools

Todos los issues de microsoft/foundry-dev-tools

Issues similares

Más issues de JavaScript

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.