Hacktoberfest 2026: los issues que los mantenedores marcaron para octubre, abiertos y aptos para principiantes. Explorar issues de Hacktoberfest

[Bug][gh-copilot] DORA dashboard uses seat-assigning organization as organization adoption denominator

Abierto
#9,041 2 comentarios 0 reacciones 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
5/5
Tiempo estimado
Más de una semana
Aptitud para principiantes
35/100
Tipo de issue
Error
Claridad
Bastante claro
Estado de actividad
Tranquilo
Stack tecnológico
go, sql

Línea de trabajo

Comienza rastreando las 27 ubicaciones de consultas MySQL y las consultas PostgreSQL correspondientes en los dashboards incluidos de github-copilot-dora-correlation. Luego inspecciona CollectCopilotSeatAssignments, copilotSeatResponse y ExtractSeats para comprender los datos de asientos de enterprise y la atribución a organizaciones. La implementación terminada debe definir una población licenciada válida con alcance de organización para la adopción actual e histórica, sin clasificar la adopción NULL como >75%.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Search before asking
  • I searched the issues and found no issue covering this organization attribution and denominator problem.

Related, but not duplicates:

  • #8964 updates Copilot API endpoints and schemas.
  • #8960 adds AI adoption phases/cohorts.
  • #9002 implements parts of those changes.
What happened

The bundled github-copilot-dora-correlation dashboards calculate organization adoption as:

daily_active_users /
COUNT(_tool_copilot_seats WHERE organization = selected_scope.organization)

This assumes that Copilot seats assigned to members of an organization are also assigned by that organization.

GitHub explicitly supports assigning licenses through a central or "shell" organization. Its current attribution rules state that once a user has a Copilot seat anywhere in the enterprise, their usage is attributed to every organization in that enterprise where they are a member.

Therefore, an organization can have valid organization-scoped usage metrics while having zero seats assigned directly by that organization.

Observed result:

  • _tool_copilot_enterprise_daily_metrics contained daily active-user data for the selected organization scope.
  • _tool_copilot_seats contained no seats attributed to that organization.
  • The dashboard denominator was zero, so adoption was NULL.
  • Adoption trend, current adoption, correlation, and tier-based panels showed no or misleading data.
  • NULL adoption was classified as >75% by tier queries because their CASE expressions use ELSE '>75%'.

The MySQL DORA dashboard currently contains this seat-denominator pattern in 27 query locations. The PostgreSQL version appears to use the same assumption.

There is also a plugin-level enterprise seat attribution concern:

  • CollectCopilotSeatAssignments correctly calls the enterprise seats endpoint when an enterprise is configured.
  • copilotSeatResponse does not parse the response's organization field.
  • ExtractSeats sets every row's Organization to connection.Organization.
  • The enterprise endpoint can return multiple seat rows for the same user when access is granted through multiple organizations or enterprise teams.

This means enterprise seat records cannot reliably represent organization membership or organization-level license eligibility.

GitHub documentation:

https://docs.github.com/en/copilot/concepts/copilot-usage-metrics/copilot-metrics#how-are-metrics-attributed-across-organizations

Relevant behavior from the documentation:

If licenses are assigned in a dedicated "shell" organization for administrative purposes within the enterprise, users still appear in the metrics for all other organizations in the enterprise they belong to.

What do you expect to happen

Organization-scoped dashboards should use an organization-scoped licensed-user population rather than filtering enterprise seats by their assigning organization.

How to reproduce
  1. Create an enterprise containing:
    • a central organization used to assign Copilot seats
    • a separate product organization
  2. Assign Copilot seats to users through the central organization.
  3. Make those users active members of the product organization.
  4. Enable GitHub Copilot usage metrics for the enterprise.
  5. Configure the DevLake gh-copilot plugin with the enterprise and product organization.
  6. Collect Copilot reports and seat assignments.
  7. Select the product organization in the bundled GitHub Copilot + DORA Correlation dashboard.
  8. Confirm that organization-scoped active-user metrics exist.
  9. Observe that the dashboard denominator is zero or otherwise incorrect because it filters seats by assigning organization.
  10. Observe that NULL adoption weeks can be labeled >75%.

Diagnostic query:

SELECT
  day,
  daily_active_users
FROM _tool_copilot_enterprise_daily_metrics
WHERE connection_id = ?
  AND scope_id = ?
ORDER BY day DESC;

Current denominator pattern:

SELECT COUNT(*)
FROM _tool_copilot_seats s
WHERE s.connection_id = ?
  AND s.organization = ?;

The first query returns organization usage while the second returns no organization-assigned seats.

Anything else

Counting rows from _tool_copilot_user_daily_metrics is not a valid replacement denominator. Those reports represent daily user activity and produce weekday/weekend fluctuations rather than the complete licensed population.

The current-state seat table also cannot reconstruct historical weekly eligibility. A historical adoption trend needs either:

  • a daily licensed-population snapshot, or
  • an effective-dated license/membership source.

The dashboard should avoid silently applying today's seat state to historical dates.

Version

Observed with the gh-copilot plugin and bundled dashboard from the DevLake v1.0.3-beta10 generation. The same denominator pattern is present on the current main branch.

Are you willing to submit PR?
  • Yes, I am willing to submit a PR.
Code of Conduct
Lenguaje dominante
Go
Estrellas
3.1k
Forks
812
Merge medio
1 d 23 h
PR fusionados (30 d)
51

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de apache/devlake

Todos los issues de apache/devlake

Issues similares

Más issues de Go

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.