[🚀 Feature]: Remove dependencies on public URLs from tests
I maintainer di solito rispondono entro 1 giorno
@yashp676 ci sta già lavorando.
Dal 6/10/2026.
Valutazione
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Idoneità per principianti
- 35/100
- Tipo di issue
- Refactoring
- Chiarezza
- Abbastanza chiara
- Stato di attività
- Attiva
- Ambito
- devtools, networking, testing-qa
Direzione di ricerca
Start with the failing Python targets in py/test/selenium/webdriver/common/bidi/browser_tests.py and _bidi/browser_tests.py, then review the other files listed in the issue by binding. Replace public test dependencies with local or reserved-host equivalents, remove the identified dead tests, and verify the affected suites run without public network access. Resolve the Rust mirror decision and audit requires-network tags before considering the cleanup complete.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Description
Several tests navigate to or connect to real public sites. These tests fail or flake when the site changes, is slow, is blocked by CI networking, or when the browser changes behavior on untrusted networks (we already have xfail_chrome markers for HTTP to HTTPS auto-upgrade on http://example.com). They also prevent running the suites with network access disabled.
A larger set of tests uses real registrable domains (example.com, google.com, test.com, etc.) purely as string data. Those never hit the network, but they are inconsistent and make it hard to audit which tests actually need network access.
This is breaking trunk CI now
Python / Browser Tests (chrome-bidi, windows) failed on trunk (4732af16c5). The only failures in the job are the two copies of one test, each failing all 3 attempts, so this is deterministic and not a flake:
//py:test/selenium/webdriver/common/bidi/browser_tests-chrome-bidi//py:test/selenium/webdriver/common/_bidi/browser_tests-chrome-bidi
FAILED py\test\selenium\webdriver\common\bidi\browser_tests.py::test_create_user_context_with_direct_proxy[chrome]
AssertionError: assert 'example domain' in 'this domain is for use in documentation examples without needing permission. this is not a service, avoid relying on it for testing and monitoring purposes.\nlearn more'
Underlying cause: nothing changed in Selenium, the browser, or the driver. test_create_user_context_with_direct_proxy loads http://example.com/ and asserts the body text contains "example domain". IANA changed the page (Last-Modified: Mon, 28 Sep 2026 16:19:23 GMT, the day before the failing run) and removed the <h1>Example Domain</h1>. The body is now:
<body><p>This domain is for use in documentation examples without needing permission. This is not a service, avoid relying on it for testing and monitoring purposes.</p><a href=https://iana.org/help/example-domains>Learn more</a><script src=/s.js></script></body>
"Example Domain" survives only in <title>, which is not part of the body text. The new page text itself says not to rely on it for testing.
The immediate fix is to point this test at a page on the local test server. Matching the new text or the title would only reset the clock until the next change. The rest of this issue covers the other tests with the same exposure.
Tests that reach the public network
| Binding | Location | Host | Notes |
|---|---|---|---|
| Python | py/test/selenium/webdriver/common/bidi/browser_tests.py (test_create_user_context_with_accept_insecure_certs, ..._with_direct_proxy, ..._with_proxy_and_accept_insecure_certs) |
self-signed.badssl.com, example.com |
Asserts on third-party page content ("example domain", badssl h1) |
| Python | py/test/selenium/webdriver/common/_bidi/browser_tests.py |
same | Near-duplicate of the file above; confirm whether both directories should exist |
| Python | py/test/selenium/webdriver/common/bidi/browser_tests.py (..._manual_proxy_all_params) |
example.com |
Goes through the local fake proxy, but already xfail on Chrome because of HTTPS upgrade |
| Python | py/test/selenium/webdriver/remote/remote_hub_connection_tests.py |
wrong.host.badssl.com |
Asserts on the TLS error text from a third-party host |
| Python | py/test/selenium/webdriver/common/example2.py |
www.google.com |
Searches Google and asserts on results; nothing references it, candidate for deletion |
| .NET | dotnet/test/webdriver/NetworkInterceptionTests.cs (TransformNetworkResponse) |
www.selenium.dev |
Response is replaced by the handler, so a local page works equally well |
| .NET | dotnet/test/webdriver/BiDi/Input/CombinedInputActionsTests.cs (Paint) |
kleki.com |
[Test] is commented out; delete or port to a local page |
| Ruby | rb/spec/integration/selenium/webdriver/manager_spec.rb |
saucelabs.com subdomains, watir.com, selenium.dev |
Guarded with skip_unless: {driver: :none} so they never run; they need subdomains or HTTPS |
| Rust | rust/tests/mirror_tests.rs |
registry.npmmirror.com |
Third-party mirror; Selenium Manager tests need network by design, but this adds a dependency on a host we do not control |
| Shared pages | common/src/web/mouse_interaction.html |
selenium.dev |
Loads jQuery and jQuery UI CSS/JS from selenium.dev instead of the local copies |
| Java | java/test/org/openqa/selenium/PageLoadingTest.java |
www.thisurldoesnotexist.comx |
Not a reserved TLD; currently @NotYetImplemented for all browsers |
| Java | java/test/org/openqa/selenium/grid/distributor/DistributorNodeAvailabilityTest.java |
does.not.exist |
Not a reserved TLD |
Real domains used only as data (no network access)
Not failures today, but worth normalizing:
google.com: JavaEndToEndTestandGridSupplier.main(spoofed session, never loads), JStest/lib/http_test.js,webdriver_test.js,until_test.js, Ruby IE options and Firefox profile specs, Pythonexample2.pytest.com,some.host.com,cheese.com,my.domain.com,selenium2.ru,idp.com,foo.com,boo.com,yes.com,froogle.com,another.server.com,get.com,navigate-to.com,http_proxy.com,https_proxy.com,tracking.com,other.com,proxy.org,pac.url,a.base.url: mocks, proxy config, and parsing tests across all bindingsexample.com/example.orgin unit tests: reserved by RFC 2606 and acceptable as data, as long as nothing resolves or loads itHttpClientTestBase.shouldAllowUrlsWithSchemesToBeUsedcreates a client forhttp://example.com(the code comment already calls it a terrible choice); the request itself goes to the local server
Already correct, no change needed: *.test, *.example, and *.invalid hosts (not_a_valid_url.test, invalid-domain-that-does-not-exist.test, doesnotexist.invalid, rp.example, idp.example, grid.example), and URLs in comments, skip reasons, and error message assertions.
Proposed changes
- Serve everything from the local test web server.
- Insecure cert tests: use the test server's existing self-signed HTTPS endpoint instead of badssl.com.
- Proxy tests: navigate to a reserved hostname (for example
http://proxied.test/) that only resolves via the fake proxy, and use a local page for the direct-proxy case. - Cookie domain and secure cookie tests: add subdomain and HTTPS support to the test environment (for example
*.localhostaliases) so the Ruby specs can run instead of being permanently skipped. TransformNetworkResponse: use a local page.mouse_interaction.html: reference the jQuery files already incommon/src/web.
- Replace unresolvable-host URLs with reserved names (
.invalidor.test). - Replace real domains used as data with reserved ones (
example.com,*.example,*.test). - Delete dead code:
example2.py(andgoogle_one_boxif unused), the disabledPainttest. - Decide on the Rust mirror test: keep with an explicit note that it depends on a third party, or point it at a mirror we control.
- Add a lint check that fails on new non-reserved hosts in test directories, with an allowlist for comments, bug tracker links, and Selenium Manager tests.
- Once clean, audit
requires-networktags so only targets that truly need it (browser and driver downloads, Selenium Manager) keep it.
Have you considered any alternatives or workarounds?
Marking the affected tests as flaky or skipping them in CI hides the problem and loses coverage. Keeping public URLs but retrying does not help with content changes or blocked networks.
- Lingua principale
- Java
- Stelle
- 34.6k
- Fork
- 8.7k
- Merge medio
- 1g 23h
- PR unite (30g)
- 86
Preparare l'ambiente
Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.
- Nessun Dockerfile né file Docker Compose
- Ha un modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di SeleniumHQ/selenium
-
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 82/100
SeleniumHQ/selenium#18323 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 88/100
SeleniumHQ/selenium#17843 · 2 commenti ·
I maintainer di solito rispondono entro 1 giorno
-
[java] DriverFinderTest property fixture doesn't clear/restore selenium.webdriver.verboseForse di nuovo libera @stefanpoia l’ha presa 25 giorni fa e non c’è nessuna pull request aperta. Aperta
Difficoltà 1/5 1-3 ore Idoneità per principianti 88/100
SeleniumHQ/selenium#17836 · 3 commenti ·
I maintainer di solito rispondono entro 1 giorno
-
A-needs-triaging C-java D-chrome D-firefox I-defect OS-linux
Difficoltà 4/5 1-2 giorni Idoneità per principianti 53/100
SeleniumHQ/selenium#18324 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
[Bug]: [py] Generated bidi.UserAgentClientHints is empty — inline CDDL params drop the commandForse già presa @titusfortner l’ha presa 3 giorni fa. ApertaC-py OS-mac
SeleniumHQ/selenium#18312 · 2 assegnatari ·
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di SeleniumHQ/selenium
Issue simili
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
btraceio/jafar#228 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 83/100
jenkinsci/git-client-plugin#1807 ·
-
area-templates triage:bot-seen triage:needs-human
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
microsoft/aspire#22356 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno
-
[Coral-Hive] locate() is typed as STRING instead of INT, so view schemas report a string columnAperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 72/100
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 84/100
apache/flink-agents#1395 ·
I maintainer di solito rispondono entro 1 giorno