Run performance tests on the Willow RBAC authorization flows in openedx-authz and openedx-platform

Đang mở
#630 0 bình luận 0 reaction 0 người được giao Xem trên GitHub

Maintainer thường phản hồi trong vòng 1 ngày

Chưa có ai nhận issue này.

Đánh giá

Độ khó
5/5
Thời gian dự kiến
Hơn một tuần
Mức phù hợp với người mới
35/100
Loại issue
Tính năng
Độ rõ ràng
Đặc tả rõ ràng
Mức độ hoạt động
Sôi nổi
Công nghệ
django, python

Hướng nghiên cứu

Start by reading ADR 0014 and the seed_sandbox_data command and fixture in authz-sandbox-seeder; identify how to reproduce the existing visible-assignments profile. Then map the listed openedx-authz endpoints and Instructor Dashboard API v2 endpoints to their tests and data requirements. Done means committing reproducible scripts and instructions, reporting the requested latency, throughput, query, and enforcement-call metrics across dataset sizes and user profiles, and documenting ranked bottlenecks with follow-up issues.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Mô tả

Description

The RBAC Willow phases moved authorization decisions to openedx-authz and added enforcement in openedx-platform, most notably in the Instructor Dashboard API v2. Every one of those requests now evaluates one or more AuthZ permission checks, and each Casbin enforce() call has a measurable cost (20 to 80 ms in the profiling recorded in ADR 0014). Performance has only been checked ad hoc so far, and there is no repeatable baseline of the current state.

This task establishes that baseline, focused on the endpoints most likely to be affected, so regressions can be detected and optimization work can be prioritized with data.

Current Behavior
  • There is no benchmark or load-test suite for openedx-authz or for the Instructor Dashboard endpoints that enforce AuthZ permissions.
  • Only the visible-assignments hot path (GET /api/authz/v1/assignments/) has been profiled and optimized.
  • The overhead of AuthZ checks in the Instructor Dashboard, with authz.enable_course_authoring enabled versus disabled for a course, has not been measured.
Expected Behavior

A documented, reproducible performance baseline exists for the Willow RBAC flows, with measured latency, throughput and query counts, a comparison between the legacy and AuthZ code paths, and a ranked list of bottlenecks with follow-up issues.

Acceptance Criteria
  • A reproducible dataset is defined and documented (see Test Dataset), including its size tiers and any data the seeder does not create.
  • Priority openedx-authz endpoints are tested (see below), with results reported per endpoint.
  • Priority Instructor Dashboard endpoints in openedx-platform are tested (see below), with results reported per endpoint.
  • Each Instructor Dashboard scenario is run with authz.enable_course_authoring enabled and disabled for the course, and the overhead of the AuthZ path over the legacy path is reported.
  • Each scenario is run with at least three user profiles: global or org-wide admin (glob scope), course-level role, and a user without access.
  • Results report p50, p95 and p99 latency, requests per second, error rate, and database query count per request, for several dataset sizes (small, medium, large).
  • Number of AuthZ enforce() calls per request is recorded for each Instructor Dashboard endpoint.
  • The test environment (hardware, Python and Django versions, database, cache backend, workers) is recorded so results are comparable.
  • Bottlenecks are listed and ranked, and a follow-up issue is opened for each significant one.
  • Scripts and instructions to re-run the tests are committed or linked, and results are published in the repository docs or linked from this issue.
  • Optional endpoints are tested only after all priority ones are complete, and are clearly marked as optional in the results.
Priority endpoints: openedx-authz

These are on frequently used paths or scale with the number of assignments, so they have the highest potential performance impact.

  • GET /api/authz/v1/permissions/validate/me (permission checks, called by the frontends on page load)
  • GET /api/authz/v1/assignments/ (visible assignments with filters; already profiled in ADR 0014, use as a regression reference)
  • GET /api/authz/v1/users/ (team members listing)
  • GET /api/authz/v1/roles/users/ (users by role)
  • is_user_allowed in openedx_authz.api, called directly, as the baseline cost of a single permission check
Priority endpoints: openedx-platform (Instructor Dashboard API v2)

Base path /api/instructor/v2/courses/<course_id>/. These endpoints either enforce AuthZ permissions directly or perform work that grows with the number of learners, so they are the most likely to show an impact.

  • GET / (course metadata): loaded every time the dashboard opens and checks whether the user holds any of the instructor dashboard permissions, which can trigger several permission checks in a single request.
  • GET enrollments (paginated enrollments list): scales with the number of enrolled learners.
Optional endpoints

Lower expected impact on performance. Test them only if time allows, and only if the priority scenarios are complete.

openedx-authz

  • GET /api/authz/v1/roles/
  • GET /api/authz/v1/scopes/
  • GET /api/authz/v1/orgs/
  • POST /api/authz/v1/users/validate/
  • The course authoring waffle flag states endpoint
  • Role assignment and unassignment through the REST API and Python API

openedx-platform (Instructor Dashboard API v2)

  • courses/<course_id>/team, team/roles and team/<email_or_username>
  • instructor_tasks, tasks/<task_id> and report downloads
  • GET grading-config.
  • GET problems/<location> (problem details).
  • GET learners/<email_or_username> (learner detail with grading data).
  • Beta testers and the remaining enrollment modification endpoints
Test Dataset

The seed_sandbox_data management command from authz-sandbox-seeder can be used as the base for the dataset, with some limitations to cover.

What it provides:

  • It creates organizations, courses, users and AuthZ role assignments from a JSON fixture, and is idempotent (--reset and --reset-only allow clean re-runs).
  • Role assignments are created through openedx_authz.api.users.assign_role_to_user_in_scope, so the data goes through the same path as production assignments.
  • Its bundled sandbox_seed_data_bulk.json fixture (now the default) is meant for load testing: 10 organizations, 20 courses, 100 users, with a mix of users with no roles, one role and several roles, and a mix of course-level and org/platform glob scopes.
  • It accepts a custom fixture through --data-file, so larger tiers can be generated.

What it does not cover and must be added:

  • Size: the bundled fixture is small for the large tier. Larger custom fixtures (for example hundreds of courses and thousands of assignments) need to be generated.

  • Learners: it does not enroll users in courses, and the Instructor Dashboard endpoints (enrollments, learners/<id>, grading-config, problems/<location>) need enrolled learners, course content and grade data.

  • Feature flag: it does not enable authz.enable_course_authoring, which must be set explicitly per course for each scenario.

  • Decide whether to extend the seeder (learner enrollment and grades) or to add a separate script for the data it does not create.

Out of Scope
  • Implementing performance optimizations (tracked in follow-up issues).
  • Policy cache loading and reload behavior.
  • Content libraries and Studio course authoring endpoints, unless a specific test is justified and added explicitly.
  • Performance of the admin console MFE and other frontend applications.
Dependencies
  • Related to the Willow RBAC phases tracked for openedx-authz and openedx-platform.
Ngôn ngữ chính
Python
Star
0
Fork
10
Merge trung bình
5 ngày 11 giờ
Pull request đã merge (30 ngày)
25

Chuẩn bị môi trường

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Issue khác của openedx/openedx-authz

Tất cả issue của openedx/openedx-authz

Issue tương tự

Thêm issue về Python

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.