[Feature Request]: Migrate LoggedInLearner and LoggedOutLearner acceptance tests to Playwright
#26,343 opened on 2026/06/09
Repository metrics
- Stars
- (5,292 個のスター)
- PR merge metrics
- (平均マージ 20d 19h) (30d で 71 merged PRs)
説明
Is your feature request related to a problem? Please describe.
As part of the rescoped GSoC project (#24715), the migration effort will focus on the LoggedInLearner and LoggedOutLearner user acceptance tests. The existing Puppeteer-based tests are flaky and difficult to maintain.
Describe the solution (or solutions) you'd like
Migrate all LoggedInLearner (except one) and LoggedOutLearner acceptance tests from Puppeteer to Playwright, following the established page object model architecture.
Proposal: https://docs.google.com/document/d/14o3v2iAHVBEGXp7N2yIBYE5d0YQN3Wjf/edit
Describe alternatives you've considered and rejected
No response
Additional context
Progress will be tracked across two milestones:
Project Milestones
Milestone 1
Key objective:
- Initialize Playwright infrastructure alongside Puppeteer enabling dual framework execution.
- Migrate logged-in-learner user test specs from Puppeteer to Playwright while maintaining CI stability.
- Migrate 2 e2e tests to puppeteer acceptance tests.
| No. | Description of PR / Action | Sub-tasks (1–2 day breakdown) | Prerequisite Steps | PR Link | Target PR Creation | Target PR Merge | Current Status |
|---|---|---|---|---|---|---|---|
| 1 | Install isolated Node 20 runtime under oppia_tools; add PLAYWRIGHT_NODE_PATH, PLAYWRIGHT_NPM_BIN_PATH, PLAYWRIGHT_NPX_BIN_PATH, PLAYWRIGHT_NODE_VERSION constants to scripts/common.py; add install_playwright_node() to install_third_party_libs.py; Add unit tests to common_test.py and install_third_party_libs_test.py |
Day 1 (May 24): Add constants to common.py; implement install_playwright_node() in install_third_party_libs.pyDay 2 (May 25): Write unit tests; verify existing CI workflows remain stable |
None | https://github.com/oppia/oppia/pull/26192 | May 25 | May 26 | ✅ Merged |
| 2 | Extend acceptance.json schema to include framework field; Add install_playwright_dependencies() and run_acceptance_tests.py; add framework-aware compilation guard for compile_test_ts_files() (skip for Playwright suites); inline framework reading logic in run_tests(); add unit tests. |
Day 1 (May 26): Extend acceptance.json, Implement install_playwright_dependencies() and compilation guard. Write unit tests in run_acceptance_tests_test.py |
Node 20 runtime installed | https://github.com/oppia/oppia/pull/26226 | May 27 | Jun 2 | ✅ Merged |
| 3 | Implement framework-aware routing in servers.py using the local node_modules/.bin/playwright binary; integrate with run_acceptance_tests.py; add unit tests in servers_test.py |
Day 1 (May 26): Implement routing logic in servers.py; integrate with run_acceptance_tests.py. Write unit tests in servers_test.py |
Configuration schema updated | https://github.com/oppia/oppia/pull/26226 | May 27 | Jun 2 | ✅ Merged |
| 4 | Update check_tests_are_captured_in_ci.py to scan both spec directories; add unit tests |
Day 1 (May 27): Extend schema; update scanning logic in check_tests_are_captured_in_ci.py; write unit tests in check_tests_are_captured_in_ci_test.py |
Playwright setup complete | https://github.com/oppia/oppia/pull/26226 | May 27 | Jun 2 | ✅ Merged |
| 5 | Update run-a-specific-acceptance-test/action.yml to add Playwright artifact upload steps to the existing Puppeteer video upload step; support dual-framework artifact uploads |
Day 1 (May 28): Update action.yml; add Playwright artifact upload steps; test on a branch |
Routing logic implemented | https://github.com/oppia/oppia/pull/26262 | May 29 | Jun 3 | ✅ Merged |
| 6 | Migrate 2 e2e tests to acceptance tests using Puppeteer. The remaining 5 e2e tests have open PRs that are either marked LGTM or being actively worked on by members of dev-workflow team. Will help them solve CI errors and get their PRs merged. | Day 1–2 per webdriverio suite: migrate spec to Puppeteer CUJ structure; verify locally and in CI | No prerequisites | — | May 31 | Jun 6 | |
| 6.1 | Migrate e2e test navigation.js - issue https://github.com/oppia/oppia/issues/24606 |
Day 1 (May 30): Migrate the test. Verify stability using stress tests. | No prerequisites | https://github.com/oppia/oppia/pull/26269 | May 30 | Jun 4 | ✅ Merged |
| 6.2 | Migrate e2e test publicationAndLibrary.js - issue https://github.com/oppia/oppia/issues/25060 |
Day 1 (May 31): Migrate the test.Day 2 (Jun 1): Verify stability using stress tests. | No prerequisites | https://github.com/oppia/oppia/pull/26276 | Jun 1 | Jun 5 | ✅ Merged |
| — | Checkpoint: Dual-framework execution working (Puppeteer + Playwright) with stable CIDemo to PMs/stakeholders:- Acceptance tests can run using both Puppeteer and Playwright- Framework-based routing via acceptance.json works- CI successfully executes both frameworks- All e2e tests migrated to acceptance tests |
— | CI and routing setup | — | Jun 6 | — | |
| 7 | Start migrating Puppeteer acceptance tests to Playwright acceptance tests. Create Playwright project folder under core/tests/playwright-acceptance-tests/ with package.json, playwright.config.ts, .gitignore; install dependencies |
— | Playwright migration workflow established | — | — | — | |
| 7.1 | Migrate logged-in-learner/add-and-remove-exploration-from-play-later |
Day 1 (Jun 4): Migrate the test; verify locally. Run in CI; fix any failures. | — | https://github.com/oppia/oppia/pull/26300 | Jun 4 | Jun 8 | ✅ Merged |
| 7.2 | Migrate logged-in-learner/completes-the-exploration-and-decides-what-to-do-next |
Day 1 (Jun 5): Migrate the test; verify locally. Run in CI; fix any failures. | — | https://github.com/oppia/oppia/pull/26321 | Jun 5 | Jun 9 | ✅ Merged |
| 7.3 | Migrate logged-in-learner/sets-goal-on-learner-dashboard |
Day 1 (Jun 6): Migrate the test; verify locally. Run in CI; fix any failures. | — | https://github.com/oppia/oppia/pull/26329 | Jun 6 | Jun 10 | ✅ Merged |
| 7.4 | Migrate logged-in-learner/provide-feedback-on-the-lesson-or-report-it-from-the-lesson-player |
Day 1 (Jun 7): Migrate the test; verify locally. Run in CI; fix any failures. | — | https://github.com/oppia/oppia/pull/26332 | Jun 7 | Jun 11 | ✅ Merged |
| 7.5 | Migrate logged-in-learner/access-pages-that-require-higher-privileges and logged-in-learner/goes-through-the-sign-in-flow |
Day 1 (Jun 8): Migrate both the tests; verify locally. Run in CI; fix any failures. | — | https://github.com/oppia/oppia/pull/26339 | Jun 8 | Jun 12 | ✅ Merged |
| 7.6 | Migrate logged-in-learner/reaches-a-checkpoint-and-saves-their-progress |
Day 1 (Jun 9): Migrate both the tests; verify locally. Run in CI; fix any failures. | — | https://github.com/oppia/oppia/pull/26344 | Jun 9 | Jun 13 | ✅ Merged |
| 7.7 | Migrate logged-in-learner/export-and-delete-account and logged-in-learner/changes-site-language-to-rtl |
Day 1 (Jun 10): Migrate both the tests; verify locally. Run in CI; fix any failures. | — | https://github.com/oppia/oppia/pull/26359 | Jun 10 | Jun 14 | ✅ Merged |
| 7.8 | Migrate logged-in-learner/edit-the-profile |
Day 1 (Jun 11): Migrate the test; verify locally. Run in CI; fix any failures. | — | https://github.com/oppia/oppia/pull/26375 | Jun 11 | Jun 15 | ✅ Merged |
| 7.9 | Migrate logged-in-learner/choose-new-lesson-to-play-from-learner-dashboard |
Day 1 (Jun 12): Migrate the test; verify locally. Run in CI; fix any failures. | — | https://github.com/oppia/oppia/pull/26382 | Jun 12 | Jun 16 | ✅ Merged |
| 7.10 | Migrate logged-in-learner/manage-classroom-progress-in-home-learner-dashboard |
Day 1 (Jun 13): Migrate the test; verify locally.Day 2 (Jun 14): Run in CI; fix any failures. | — | https://github.com/oppia/oppia/pull/26391 | Jun 14 | Jun 18 | ✅ Merged |
| 7.11 | Migrate logged-in-learner/manage-in-progress-and-completed-lessons-and-skill |
Day 1 (Jun 15): Migrate the test; verify locally.Day 2 (Jun 16): Run in CI; fix any failures. | — | https://github.com/oppia/oppia/pull/26426 | Jun 16 | Jun 20 | ✅ Merged |
logged-in-learner/manage-in-progress-and-completed-community-lessons-and-skill |
— | — | |||||
| 7.13 | Migrate logged-in-learner/manage-goals-in-learner-dashboard |
Day 1 (Jun 19): Migrate the test; verify locally. | — | https://github.com/oppia/oppia/pull/26449 | Jun 19 | Jun 23 | ✅ Merged |
| 7.14 | Migrate logged-in-learner/interact-with-goals-in-learner-dashboard |
Day 1 (Jun 20): Migrate the test; verify locally. | — | https://github.com/oppia/oppia/pull/26449 | Jun 20 | Jun 24 | ✅ Merged |
| 7.15 | Migrate logged-in-learner/manage-community-lesson-progress-in-home-learner-dashboard |
Day 1 (Jun 21): Migrate the test; verify locally. Run in CI; fix any failures. | — | https://github.com/oppia/oppia/pull/26449 | Jun 21 | Jun 25 | ✅ Merged |
| 7.16 | Migrate logged-in-learner/starts-from-beginning-after-completing-a-lesson |
Day 1 (Jun 22): Migrate the test; verify locally. | — | https://github.com/oppia/oppia/pull/26449 | Jun 22 | Jun 26 | ✅ Merged |
| 8 | Replace waitForSelector with common wrappers (like expectElementToBeVisible) and use Playwright API calls |
Day 1 (Jun 23): Open the PR | PRs till 7.16 are approved | https://github.com/oppia/oppia/pull/26627 | Jun 23 | Jun 27 | ✅ Merged |
| 9.1 | Add --update-snapshots flag to playwright acceptance tests - issue https://github.com/oppia/oppia/issues/26465. And add a workflow for updating all the snapshots in all acceptance tests. | Day 1 (Jun 26): Open the PR | https://github.com/oppia/oppia/pull/26469 and https://github.com/oppia/oppia/pull/26647 | Jun 26 | Jul 1 | ✅ Merged | |
| 9.2 | Add playwright acceptance tests to the merge queue. | Day 1 (Jun 27): Open the PR | https://github.com/oppia/oppia/pull/26596 | Jun 27 | Jun 29 | ✅ Merged | |
| — | Demo to PMs/stakeholders: The above-mentioned tests of logged-in-learner users run through Playwright. | — | Complete migration of above tests. | — | Jun 26 | — |
Milestone 2
Key objective:
- Migrate logged-out-learner user tests specs from Puppeteer to Playwright while maintaining CI stability.
- Fix method overriding issue and snapshot non-fast fail behaviour.
- Document debugging guides.
| No. | Description of PR / Action | Sub-tasks (1–2 day breakdown) | Prerequisite Steps | PR Link | Target PR Creation | Target PR Merge | Current Status |
|---|---|---|---|---|---|---|---|
| 1 | Migrate logged-out-learner/play-a-complete-community-lesson |
Day 1 (Jul 17): Migrate the test; verify locally.Day 2 (Jul 18): Run in CI; fix failures. | — | https://github.com/oppia/oppia/pull/26738 | Jul 18 | Jul 20 | ✅ Merged |
| 2 | Migrate logged-out-learner/listen-to-voiceovers-of-the-lessons-in-the-lesson-player |
Day 1 (Jul 19): Migrate the test; verify locally.Day 2 (Jul 20): Run in CI; fix failures. | — | https://github.com/oppia/oppia/pull/26738 | Jul 20 | Jul 22 | ✅ Merged |
| 3 | Migrate logged-out-learner/complete-some-practice-question |
Day 1 (Jul 21): Migrate the test; verify locally. Run in CI; fix any failures. | — | https://github.com/oppia/oppia/pull/26775 | Jul 21 | Jul 23 | ✅ Merged |
| 4 | Migrate logged-out-learner/reaches-a-checkpoint-and-saves-their-progress |
Day 1 (Jul 22): Migrate the test; verify locally. Run in CI; fix any failures. | — | https://github.com/oppia/oppia/pull/26775 | Jul 22 | Jul 24 | ✅ Merged |
| 5 | Migrate logged-out-learner/use-the-feedback-and-help-cards |
Day 1 (Jul 23): Migrate the test; verify locally. Run in CI; fix any failures. | — | https://github.com/oppia/oppia/pull/26852 | Jul 23 | Jul 25 | ✅ Merged |
| 6 | Migrate logged-out-learner/share-the-lesson-from-the-lesson-player |
Day 1 (Jul 24): Migrate the test; verify locally. Run in CI; fix any failures. | — | https://github.com/oppia/oppia/pull/26852 | Jul 24 | Jul 26 | ✅ Merged |
| 7 | Migrate logged-out-learner/give-feedback-on-the-lesson-from-the-lesson-player |
Day 1 (Jul 25): Migrate the test; verify locally. Run in CI; fix any failures. | — | https://github.com/oppia/oppia/pull/26920 | Jul 25 | Jul 27 | |
| 8 | Migrate logged-out-learner/discover-the-website-and-navigate-to-math-classroom |
Day 1 (Jul 26): Migrate the test; verify locally. Run in CI; fix any failures. | — | https://github.com/oppia/oppia/pull/26920 | Jul 26 | Jul 28 | |
| 9 | Migrate logged-out-learner/pick-a-lesson-to-learn |
Day 1 (Jul 27): Migrate the test; verify locally. Run in CI; fix any failures. | — | https://github.com/oppia/oppia/pull/26939 | Jul 27 | Jul 29 | |
| 10 | Migrate logged-out-learner/search-for-a-specific-exploration |
Day 1 (Jul 28): Migrate the test; verify locally. Run in CI; fix any failures. | — | https://github.com/oppia/oppia/pull/26939 | Jul 28 | Jul 30 | |
| 11 | Migrate logged-out-learner/visits-the-community-library-and-look-around |
Day 1 (Jul 29): Migrate the test; verify locally. Run in CI; fix any failures. | — | https://github.com/oppia/oppia/pull/26972 | Jul 29 | Jul 31 | |
| 12 | Migrate logged-out-learner/access-the-lesson-player |
Day 1 (Jul 30): Migrate the test; verify locally. Run in CI; fix any failures. | — | https://github.com/oppia/oppia/pull/26972 | Jul 30 | Aug 1 | |
| 13 | Migrate logged-out-learner/complete-the-embedded-lesson |
Day 1 (Jul 31): Migrate the test; verify locally. Run in CI; fix any failures. | — | https://github.com/oppia/oppia/pull/26972 | Jul 31 | Aug 2 | |
| 14 | Fix method overriding issue in UserFactory (detailed in Section 13.9) |
Day 1 (Aug 8,9): Implement duplicate method detection; fix any existing violations.Day 2 (Aug 10): Write unit tests | Stable Playwright-based test execution | — | Aug 10 | Aug 14 | |
| 15 | Create debugging guide in oppia-web-developer-docs covering: running tests locally, Playwright Inspector, Trace Viewer, CI artifacts, stress test workflow, updating snapshots (tracking issues #16136, #22319, #422) |
Day 1 (Aug 15): Write sections on local debugging, Playwright Inspector, Trace Viewer.Day 2 (Aug 16): Write sections on CI artifacts, stress test workflow, snapshot updates; submit PR for review | CI artifacts and tracing enabled | — | Aug 16 | Aug 18 | |
| — | Demo to PMs/stakeholders:- All logged-out-learner tests mentioned above migrated to Playwright- Debugging guide published | — | Stable Playwright execution, debugging setup | — | Aug 19 | — |
Meeting minutes: https://docs.google.com/document/d/1TdWroVK0cgEEZoCbUvpTHRKzzakOENb2xD-Gh34lv4Y/edit Journal: https://medium.com/@mohak14062005