oppia/oppia

[Feature Request]: Migrate LoggedInLearner and LoggedOutLearner acceptance tests to Playwright

オープン

#26,343 opened on 2026/06/09

 (2 件のコメント) (0 件のリアクション) (1 人の担当者)Python (3,534 件のフォーク)batch import
enhancementgood first issue

Repository metrics

Stars
 (5,292 個のスター)
PR merge metrics
 (平均マージ 20d 19h) (30d で 71 merged PRs)

説明

Is your feature request related to a problem? Please describe.

As part of the rescoped GSoC project (#24715), the migration effort will focus on the LoggedInLearner and LoggedOutLearner user acceptance tests. The existing Puppeteer-based tests are flaky and difficult to maintain.

Describe the solution (or solutions) you'd like

Migrate all LoggedInLearner (except one) and LoggedOutLearner acceptance tests from Puppeteer to Playwright, following the established page object model architecture.

Proposal: https://docs.google.com/document/d/14o3v2iAHVBEGXp7N2yIBYE5d0YQN3Wjf/edit

Describe alternatives you've considered and rejected

No response

Additional context

Progress will be tracked across two milestones:

Project Milestones

Milestone 1

Key objective:

  • Initialize Playwright infrastructure alongside Puppeteer enabling dual framework execution.
  • Migrate logged-in-learner user test specs from Puppeteer to Playwright while maintaining CI stability.
  • Migrate 2 e2e tests to puppeteer acceptance tests.
No. Description of PR / Action Sub-tasks (1–2 day breakdown) Prerequisite Steps PR Link Target PR Creation Target PR Merge Current Status
1 Install isolated Node 20 runtime under oppia_tools; add PLAYWRIGHT_NODE_PATH, PLAYWRIGHT_NPM_BIN_PATH, PLAYWRIGHT_NPX_BIN_PATH, PLAYWRIGHT_NODE_VERSION constants to scripts/common.py; add install_playwright_node() to install_third_party_libs.py; Add unit tests to common_test.py and install_third_party_libs_test.py Day 1 (May 24): Add constants to common.py; implement install_playwright_node() in install_third_party_libs.pyDay 2 (May 25): Write unit tests; verify existing CI workflows remain stable None https://github.com/oppia/oppia/pull/26192 May 25 May 26 ✅ Merged
2 Extend acceptance.json schema to include framework field; Add install_playwright_dependencies() and run_acceptance_tests.py; add framework-aware compilation guard for compile_test_ts_files() (skip for Playwright suites); inline framework reading logic in run_tests(); add unit tests. Day 1 (May 26): Extend acceptance.json, Implement install_playwright_dependencies() and compilation guard. Write unit tests in run_acceptance_tests_test.py Node 20 runtime installed https://github.com/oppia/oppia/pull/26226 May 27 Jun 2 ✅ Merged
3 Implement framework-aware routing in servers.py using the local node_modules/.bin/playwright binary; integrate with run_acceptance_tests.py; add unit tests in servers_test.py Day 1 (May 26): Implement routing logic in servers.py; integrate with run_acceptance_tests.py. Write unit tests in servers_test.py Configuration schema updated https://github.com/oppia/oppia/pull/26226 May 27 Jun 2 ✅ Merged
4 Update check_tests_are_captured_in_ci.py to scan both spec directories; add unit tests Day 1 (May 27): Extend schema; update scanning logic in check_tests_are_captured_in_ci.py; write unit tests in check_tests_are_captured_in_ci_test.py Playwright setup complete https://github.com/oppia/oppia/pull/26226 May 27 Jun 2 ✅ Merged
5 Update run-a-specific-acceptance-test/action.yml to add Playwright artifact upload steps to the existing Puppeteer video upload step; support dual-framework artifact uploads Day 1 (May 28): Update action.yml; add Playwright artifact upload steps; test on a branch Routing logic implemented https://github.com/oppia/oppia/pull/26262 May 29 Jun 3 ✅ Merged
6 Migrate 2 e2e tests to acceptance tests using Puppeteer. The remaining 5 e2e tests have open PRs that are either marked LGTM or being actively worked on by members of dev-workflow team. Will help them solve CI errors and get their PRs merged. Day 1–2 per webdriverio suite: migrate spec to Puppeteer CUJ structure; verify locally and in CI No prerequisites May 31 Jun 6
6.1 Migrate e2e test navigation.js - issue https://github.com/oppia/oppia/issues/24606 Day 1 (May 30): Migrate the test. Verify stability using stress tests. No prerequisites https://github.com/oppia/oppia/pull/26269 May 30 Jun 4 ✅ Merged
6.2 Migrate e2e test publicationAndLibrary.js - issue https://github.com/oppia/oppia/issues/25060 Day 1 (May 31): Migrate the test.Day 2 (Jun 1): Verify stability using stress tests. No prerequisites https://github.com/oppia/oppia/pull/26276 Jun 1 Jun 5 ✅ Merged
Checkpoint: Dual-framework execution working (Puppeteer + Playwright) with stable CIDemo to PMs/stakeholders:- Acceptance tests can run using both Puppeteer and Playwright- Framework-based routing via acceptance.json works- CI successfully executes both frameworks- All e2e tests migrated to acceptance tests CI and routing setup Jun 6
7 Start migrating Puppeteer acceptance tests to Playwright acceptance tests. Create Playwright project folder under core/tests/playwright-acceptance-tests/ with package.json, playwright.config.ts, .gitignore; install dependencies Playwright migration workflow established
7.1 Migrate logged-in-learner/add-and-remove-exploration-from-play-later Day 1 (Jun 4): Migrate the test; verify locally. Run in CI; fix any failures. https://github.com/oppia/oppia/pull/26300 Jun 4 Jun 8 ✅ Merged
7.2 Migrate logged-in-learner/completes-the-exploration-and-decides-what-to-do-next Day 1 (Jun 5): Migrate the test; verify locally. Run in CI; fix any failures. https://github.com/oppia/oppia/pull/26321 Jun 5 Jun 9 ✅ Merged
7.3 Migrate logged-in-learner/sets-goal-on-learner-dashboard Day 1 (Jun 6): Migrate the test; verify locally. Run in CI; fix any failures. https://github.com/oppia/oppia/pull/26329 Jun 6 Jun 10 ✅ Merged
7.4 Migrate logged-in-learner/provide-feedback-on-the-lesson-or-report-it-from-the-lesson-player Day 1 (Jun 7): Migrate the test; verify locally. Run in CI; fix any failures. https://github.com/oppia/oppia/pull/26332 Jun 7 Jun 11 ✅ Merged
7.5 Migrate logged-in-learner/access-pages-that-require-higher-privileges and logged-in-learner/goes-through-the-sign-in-flow Day 1 (Jun 8): Migrate both the tests; verify locally. Run in CI; fix any failures. https://github.com/oppia/oppia/pull/26339 Jun 8 Jun 12 ✅ Merged
7.6 Migrate logged-in-learner/reaches-a-checkpoint-and-saves-their-progress Day 1 (Jun 9): Migrate both the tests; verify locally. Run in CI; fix any failures. https://github.com/oppia/oppia/pull/26344 Jun 9 Jun 13 ✅ Merged
7.7 Migrate logged-in-learner/export-and-delete-account and logged-in-learner/changes-site-language-to-rtl Day 1 (Jun 10): Migrate both the tests; verify locally. Run in CI; fix any failures. https://github.com/oppia/oppia/pull/26359 Jun 10 Jun 14 ✅ Merged
7.8 Migrate logged-in-learner/edit-the-profile Day 1 (Jun 11): Migrate the test; verify locally. Run in CI; fix any failures. https://github.com/oppia/oppia/pull/26375 Jun 11 Jun 15 ✅ Merged
7.9 Migrate logged-in-learner/choose-new-lesson-to-play-from-learner-dashboard Day 1 (Jun 12): Migrate the test; verify locally. Run in CI; fix any failures. https://github.com/oppia/oppia/pull/26382 Jun 12 Jun 16 ✅ Merged
7.10 Migrate logged-in-learner/manage-classroom-progress-in-home-learner-dashboard Day 1 (Jun 13): Migrate the test; verify locally.Day 2 (Jun 14): Run in CI; fix any failures. https://github.com/oppia/oppia/pull/26391 Jun 14 Jun 18 ✅ Merged
7.11 Migrate logged-in-learner/manage-in-progress-and-completed-lessons-and-skill Day 1 (Jun 15): Migrate the test; verify locally.Day 2 (Jun 16): Run in CI; fix any failures. https://github.com/oppia/oppia/pull/26426 Jun 16 Jun 20 ✅ Merged
7.12 Migrate logged-in-learner/manage-in-progress-and-completed-community-lessons-and-skill Day 1 (Jun 17): Migrate the test; verify locally.Day 2 (Jun 18): Run in CI; fix any failures. Jun 18 Jun 22
7.13 Migrate logged-in-learner/manage-goals-in-learner-dashboard Day 1 (Jun 19): Migrate the test; verify locally. https://github.com/oppia/oppia/pull/26449 Jun 19 Jun 23 ✅ Merged
7.14 Migrate logged-in-learner/interact-with-goals-in-learner-dashboard Day 1 (Jun 20): Migrate the test; verify locally. https://github.com/oppia/oppia/pull/26449 Jun 20 Jun 24 ✅ Merged
7.15 Migrate logged-in-learner/manage-community-lesson-progress-in-home-learner-dashboard Day 1 (Jun 21): Migrate the test; verify locally. Run in CI; fix any failures. https://github.com/oppia/oppia/pull/26449 Jun 21 Jun 25 ✅ Merged
7.16 Migrate logged-in-learner/starts-from-beginning-after-completing-a-lesson Day 1 (Jun 22): Migrate the test; verify locally. https://github.com/oppia/oppia/pull/26449 Jun 22 Jun 26 ✅ Merged
8 Replace waitForSelector with common wrappers (like expectElementToBeVisible) and use Playwright API calls Day 1 (Jun 23): Open the PR PRs till 7.16 are approved https://github.com/oppia/oppia/pull/26627 Jun 23 Jun 27 ✅ Merged
9.1 Add --update-snapshots flag to playwright acceptance tests - issue https://github.com/oppia/oppia/issues/26465. And add a workflow for updating all the snapshots in all acceptance tests. Day 1 (Jun 26): Open the PR https://github.com/oppia/oppia/pull/26469 and https://github.com/oppia/oppia/pull/26647 Jun 26 Jul 1 ✅ Merged
9.2 Add playwright acceptance tests to the merge queue. Day 1 (Jun 27): Open the PR https://github.com/oppia/oppia/pull/26596 Jun 27 Jun 29 ✅ Merged
Demo to PMs/stakeholders: The above-mentioned tests of logged-in-learner users run through Playwright. Complete migration of above tests. Jun 26

Milestone 2

Key objective:

  • Migrate logged-out-learner user tests specs from Puppeteer to Playwright while maintaining CI stability.
  • Fix method overriding issue and snapshot non-fast fail behaviour.
  • Document debugging guides.
No. Description of PR / Action Sub-tasks (1–2 day breakdown) Prerequisite Steps PR Link Target PR Creation Target PR Merge Current Status
1 Migrate logged-out-learner/play-a-complete-community-lesson Day 1 (Jul 17): Migrate the test; verify locally.Day 2 (Jul 18): Run in CI; fix failures. https://github.com/oppia/oppia/pull/26738 Jul 18 Jul 20 ✅ Merged
2 Migrate logged-out-learner/listen-to-voiceovers-of-the-lessons-in-the-lesson-player Day 1 (Jul 19): Migrate the test; verify locally.Day 2 (Jul 20): Run in CI; fix failures. https://github.com/oppia/oppia/pull/26738 Jul 20 Jul 22 ✅ Merged
3 Migrate logged-out-learner/complete-some-practice-question Day 1 (Jul 21): Migrate the test; verify locally. Run in CI; fix any failures. https://github.com/oppia/oppia/pull/26775 Jul 21 Jul 23 ✅ Merged
4 Migrate logged-out-learner/reaches-a-checkpoint-and-saves-their-progress Day 1 (Jul 22): Migrate the test; verify locally. Run in CI; fix any failures. https://github.com/oppia/oppia/pull/26775 Jul 22 Jul 24 ✅ Merged
5 Migrate logged-out-learner/use-the-feedback-and-help-cards Day 1 (Jul 23): Migrate the test; verify locally. Run in CI; fix any failures. https://github.com/oppia/oppia/pull/26852 Jul 23 Jul 25 ✅ Merged
6 Migrate logged-out-learner/share-the-lesson-from-the-lesson-player Day 1 (Jul 24): Migrate the test; verify locally. Run in CI; fix any failures. https://github.com/oppia/oppia/pull/26852 Jul 24 Jul 26 ✅ Merged
7 Migrate logged-out-learner/give-feedback-on-the-lesson-from-the-lesson-player Day 1 (Jul 25): Migrate the test; verify locally. Run in CI; fix any failures. https://github.com/oppia/oppia/pull/26920 Jul 25 Jul 27
8 Migrate logged-out-learner/discover-the-website-and-navigate-to-math-classroom Day 1 (Jul 26): Migrate the test; verify locally. Run in CI; fix any failures. https://github.com/oppia/oppia/pull/26920 Jul 26 Jul 28
9 Migrate logged-out-learner/pick-a-lesson-to-learn Day 1 (Jul 27): Migrate the test; verify locally. Run in CI; fix any failures. https://github.com/oppia/oppia/pull/26939 Jul 27 Jul 29
10 Migrate logged-out-learner/search-for-a-specific-exploration Day 1 (Jul 28): Migrate the test; verify locally. Run in CI; fix any failures. https://github.com/oppia/oppia/pull/26939 Jul 28 Jul 30
11 Migrate logged-out-learner/visits-the-community-library-and-look-around Day 1 (Jul 29): Migrate the test; verify locally. Run in CI; fix any failures. https://github.com/oppia/oppia/pull/26972 Jul 29 Jul 31
12 Migrate logged-out-learner/access-the-lesson-player Day 1 (Jul 30): Migrate the test; verify locally. Run in CI; fix any failures. https://github.com/oppia/oppia/pull/26972 Jul 30 Aug 1
13 Migrate logged-out-learner/complete-the-embedded-lesson Day 1 (Jul 31): Migrate the test; verify locally. Run in CI; fix any failures. https://github.com/oppia/oppia/pull/26972 Jul 31 Aug 2
14 Fix method overriding issue in UserFactory (detailed in Section 13.9) Day 1 (Aug 8,9): Implement duplicate method detection; fix any existing violations.Day 2 (Aug 10): Write unit tests Stable Playwright-based test execution Aug 10 Aug 14
15 Create debugging guide in oppia-web-developer-docs covering: running tests locally, Playwright Inspector, Trace Viewer, CI artifacts, stress test workflow, updating snapshots (tracking issues #16136, #22319, #422) Day 1 (Aug 15): Write sections on local debugging, Playwright Inspector, Trace Viewer.Day 2 (Aug 16): Write sections on CI artifacts, stress test workflow, snapshot updates; submit PR for review CI artifacts and tracing enabled Aug 16 Aug 18
Demo to PMs/stakeholders:- All logged-out-learner tests mentioned above migrated to Playwright- Debugging guide published Stable Playwright execution, debugging setup Aug 19

Meeting minutes: https://docs.google.com/document/d/1TdWroVK0cgEEZoCbUvpTHRKzzakOENb2xD-Gh34lv4Y/edit Journal: https://medium.com/@mohak14062005

コントリビューターガイド