Run performance tests on the Willow RBAC authorization flows in openedx-authz and openedx-platform
Los mantenedores suelen responder en 1 día
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 5/5
- Tiempo estimado
- Más de una semana
- Aptitud para principiantes
- 35/100
- Tipo de issue
- Nueva funcionalidad
- Claridad
- Bien especificado
- Estado de actividad
- Activo
Línea de trabajo
Start by reading ADR 0014 and the seed_sandbox_data command and fixture in authz-sandbox-seeder; identify how to reproduce the existing visible-assignments profile. Then map the listed openedx-authz endpoints and Instructor Dashboard API v2 endpoints to their tests and data requirements. Done means committing reproducible scripts and instructions, reporting the requested latency, throughput, query, and enforcement-call metrics across dataset sizes and user profiles, and documenting ranked bottlenecks with follow-up issues.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Description
The RBAC Willow phases moved authorization decisions to openedx-authz and added enforcement in openedx-platform, most notably in the Instructor Dashboard API v2. Every one of those requests now evaluates one or more AuthZ permission checks, and each Casbin enforce() call has a measurable cost (20 to 80 ms in the profiling recorded in ADR 0014). Performance has only been checked ad hoc so far, and there is no repeatable baseline of the current state.
This task establishes that baseline, focused on the endpoints most likely to be affected, so regressions can be detected and optimization work can be prioritized with data.
Current Behavior
- There is no benchmark or load-test suite for
openedx-authzor for the Instructor Dashboard endpoints that enforce AuthZ permissions. - Only the visible-assignments hot path (
GET /api/authz/v1/assignments/) has been profiled and optimized. - The overhead of AuthZ checks in the Instructor Dashboard, with
authz.enable_course_authoringenabled versus disabled for a course, has not been measured.
Expected Behavior
A documented, reproducible performance baseline exists for the Willow RBAC flows, with measured latency, throughput and query counts, a comparison between the legacy and AuthZ code paths, and a ranked list of bottlenecks with follow-up issues.
Acceptance Criteria
- A reproducible dataset is defined and documented (see Test Dataset), including its size tiers and any data the seeder does not create.
- Priority
openedx-authzendpoints are tested (see below), with results reported per endpoint. - Priority Instructor Dashboard endpoints in
openedx-platformare tested (see below), with results reported per endpoint. - Each Instructor Dashboard scenario is run with
authz.enable_course_authoringenabled and disabled for the course, and the overhead of the AuthZ path over the legacy path is reported. - Each scenario is run with at least three user profiles: global or org-wide admin (glob scope), course-level role, and a user without access.
- Results report p50, p95 and p99 latency, requests per second, error rate, and database query count per request, for several dataset sizes (small, medium, large).
- Number of AuthZ
enforce()calls per request is recorded for each Instructor Dashboard endpoint. - The test environment (hardware, Python and Django versions, database, cache backend, workers) is recorded so results are comparable.
- Bottlenecks are listed and ranked, and a follow-up issue is opened for each significant one.
- Scripts and instructions to re-run the tests are committed or linked, and results are published in the repository docs or linked from this issue.
- Optional endpoints are tested only after all priority ones are complete, and are clearly marked as optional in the results.
Priority endpoints: openedx-authz
These are on frequently used paths or scale with the number of assignments, so they have the highest potential performance impact.
-
GET /api/authz/v1/permissions/validate/me(permission checks, called by the frontends on page load) -
GET /api/authz/v1/assignments/(visible assignments with filters; already profiled in ADR 0014, use as a regression reference) -
GET /api/authz/v1/users/(team members listing) -
GET /api/authz/v1/roles/users/(users by role) -
is_user_allowedinopenedx_authz.api, called directly, as the baseline cost of a single permission check
Priority endpoints: openedx-platform (Instructor Dashboard API v2)
Base path /api/instructor/v2/courses/<course_id>/. These endpoints either enforce AuthZ permissions directly or perform work that grows with the number of learners, so they are the most likely to show an impact.
-
GET /(course metadata): loaded every time the dashboard opens and checks whether the user holds any of the instructor dashboard permissions, which can trigger several permission checks in a single request. -
GET enrollments(paginated enrollments list): scales with the number of enrolled learners.
Optional endpoints
Lower expected impact on performance. Test them only if time allows, and only if the priority scenarios are complete.
openedx-authz
-
GET /api/authz/v1/roles/ -
GET /api/authz/v1/scopes/ -
GET /api/authz/v1/orgs/ -
POST /api/authz/v1/users/validate/ - The course authoring waffle flag states endpoint
- Role assignment and unassignment through the REST API and Python API
openedx-platform (Instructor Dashboard API v2)
-
courses/<course_id>/team,team/rolesandteam/<email_or_username> -
instructor_tasks,tasks/<task_id>and report downloads -
GET grading-config. -
GET problems/<location>(problem details). -
GET learners/<email_or_username>(learner detail with grading data). - Beta testers and the remaining enrollment modification endpoints
Test Dataset
The seed_sandbox_data management command from authz-sandbox-seeder can be used as the base for the dataset, with some limitations to cover.
What it provides:
- It creates organizations, courses, users and AuthZ role assignments from a JSON fixture, and is idempotent (
--resetand--reset-onlyallow clean re-runs). - Role assignments are created through
openedx_authz.api.users.assign_role_to_user_in_scope, so the data goes through the same path as production assignments. - Its bundled
sandbox_seed_data_bulk.jsonfixture (now the default) is meant for load testing: 10 organizations, 20 courses, 100 users, with a mix of users with no roles, one role and several roles, and a mix of course-level and org/platform glob scopes. - It accepts a custom fixture through
--data-file, so larger tiers can be generated.
What it does not cover and must be added:
-
Size: the bundled fixture is small for the large tier. Larger custom fixtures (for example hundreds of courses and thousands of assignments) need to be generated.
-
Learners: it does not enroll users in courses, and the Instructor Dashboard endpoints (
enrollments,learners/<id>,grading-config,problems/<location>) need enrolled learners, course content and grade data. -
Feature flag: it does not enable
authz.enable_course_authoring, which must be set explicitly per course for each scenario. -
Decide whether to extend the seeder (learner enrollment and grades) or to add a separate script for the data it does not create.
Out of Scope
- Implementing performance optimizations (tracked in follow-up issues).
- Policy cache loading and reload behavior.
- Content libraries and Studio course authoring endpoints, unless a specific test is justified and added explicitly.
- Performance of the admin console MFE and other frontend applications.
Dependencies
- Related to the Willow RBAC phases tracked for
openedx-authzandopenedx-platform.
- Lenguaje dominante
- Python
- Estrellas
- 0
- Forks
- 10
- Merge medio
- 5 d 11 h
- PR fusionados (30 d)
- 25
Preparar el entorno
- Sin Dockerfile ni archivo de Docker Compose
- Tiene una plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de openedx/openedx-authz
-
Add a course permission check to ContainerChildrenViewPosiblemente ocupada @BryanttV la tomó hace 7 días. Abiertowillow
Dificultad 2/5 1-3 horas Aptitud para principiantes 76/100
openedx/openedx-authz#604 · 1 asignado ·
Los mantenedores suelen responder en 1 día
-
Allow Course Editors to import YouTube transcripts and surface errorsPosiblemente ocupada @efortish la tomó hace 7 días. Abiertowillow
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
openedx/openedx-authz#597 · 1 asignado ·
Los mantenedores suelen responder en 1 día
-
willow
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
openedx/openedx-authz#579 ·
Los mantenedores suelen responder en 1 día
-
Add `hidden` property to permission categories and permissionsPosiblemente ocupada @BryanttV la tomó hace 2 días. Abierto
openedx/openedx-authz#632 · 1 asignado ·
Los mantenedores suelen responder en 1 día
-
Dificultad 5/5 Más de una semana Aptitud para principiantes 35/100
openedx/openedx-authz#631 ·
Los mantenedores suelen responder en 1 día
Todos los issues de openedx/openedx-authz
Issues similares
-
CI flake: test-windows times out spawning Node in tests/test_antigravity_node_hooks.py (develop red)Abiertoarea/ci P2
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
Los mantenedores suelen responder en 2 días
-
benchmark
Dificultad 1/5 1-3 horas Aptitud para principiantes 72/100
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 79/100
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 77/100
mozilla-ai/cq#601 ·
Los mantenedores suelen responder en 1 día
-
Bump Python to 3.14 or 3.15Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
Los mantenedores suelen responder en 1 día