mechanize._mechanize.FormNotFoundError: no form matching name 'form'

Start with the README.md example and reproduce the failure around handelsregister.py lines 72 and 76, where the Advanced search link is followed and the form is selected. Inspect the current response and determine whether the website change is compatible with the existing approach; done means the issue has a verified diagnosis and a clear maintenance path or documented limitation.

Hey,

I've tried the example from the README.md and installed the app as described.
Unfortunatley it fails because the HTML docs has no HTML-Form named form.

send: b'GET / HTTP/1.1\r\nAccept-Encoding: gzip\r\nUser-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.5 Safari/605.1.15\r\nAccept-Language: en-GB,en;q=0.9\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nConnection: close\r\nHost: www.handelsregister.de\r\n\r\n'
reply: 'HTTP/1.1 302 Found\r\n'
header: Date: Sun, 14 Jul 2024 09:27:36 GMT
header: Server: Apache
header: Strict-Transport-Security: max-age=31536000; preload
header: Referrer-Policy: origin-when-cross-origin
header: Location: https://www.handelsregister.de/rp_web/welcome.xhtml
header: Cache-Control: max-age=15
header: Expires: Sun, 14 Jul 2024 09:27:51 GMT
header: Content-Length: 235
header: Connection: close
header: Content-Type: text/html; charset=iso-8859-1
b'<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">\n<html><head>\n<title>302 Found</title>\n</head><body>\n<h1>Found</h1>\n<p>The document has moved <a href="https://www.handelsregister.de/rp_web/welcome.xhtml">here</a>.</p>\n</body></html>\n'
*****************************************************
send: b'GET /rp_web/welcome.xhtml HTTP/1.1\r\nAccept-Encoding: gzip\r\nUser-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.5 Safari/605.1.15\r\nAccept-Language: en-GB,en;q=0.9\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nConnection: close\r\nHost: www.handelsregister.de\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Sun, 14 Jul 2024 09:27:37 GMT
header: Server: Apache-Coyote/1.1
header: Strict-Transport-Security: max-age=31536000; preload
header: Referrer-Policy: origin-when-cross-origin
header: Referrer-Policy: origin-when-cross-origin
header: Expires: Thu, 01 Jan 1970 01:00:00 GMT
header: Pragma: no-cache
header: Expires: Tue, 08 Aug 2006 10:00:00 GMT
header: Content-Type: text/html;charset=UTF-8
header: Content-Length: 41571
header: Vary: Accept-Encoding
header: X-Content-Type-Options: nosniff
header: Cache-Control: must-revalidate, proxy-revalidate, no-store, no-cache, s-max-age=0, max-age=0
header: X-Frame-Options: SAMEORIGIN
header: X-XSS-Protection: 1; mode=block
header: X-Permitted-Cross-Domain-Policies: master-only
header: Set-Cookie: JSESSIONID=53FBAE13BD34A6F4DF932C52BC83C2F5.tc05n02; Path=/; HttpOnly
header: Connection: close
b'<!DOCTYPE html>...too much for github...</html>'
*****************************************************
send: b'GET /rp_web/welcome.xhtml HTTP/1.1\r\nAccept-Encoding: gzip\r\nReferer: https://www.handelsregister.de\r\nCookie: JSESSIONID=53FBAE13BD34A6F4DF932C52BC83C2F5.tc05n02\r\nUser-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.5 Safari/605.1.15\r\nAccept-Language: en-GB,en;q=0.9\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nConnection: close\r\nHost: www.handelsregister.de\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Date: Sun, 14 Jul 2024 09:27:37 GMT
header: Server: Apache-Coyote/1.1
header: Strict-Transport-Security: max-age=31536000; preload
header: Referrer-Policy: origin-when-cross-origin
header: Referrer-Policy: origin-when-cross-origin
header: Expires: Thu, 01 Jan 1970 01:00:00 GMT
header: Pragma: no-cache
header: Expires: Tue, 08 Aug 2006 10:00:00 GMT
header: Content-Type: text/html;charset=UTF-8
header: Content-Length: 40427
header: Vary: Accept-Encoding
header: X-Content-Type-Options: nosniff
header: Cache-Control: must-revalidate, proxy-revalidate, no-store, no-cache, s-max-age=0, max-age=0
header: X-Frame-Options: SAMEORIGIN
header: X-XSS-Protection: 1; mode=block
header: X-Permitted-Cross-Domain-Policies: master-only
header: Connection: close
b'<!DOCTYPE html>...too much for github...</html>'
*****************************************************
Registerportal | Homepage
Traceback (most recent call last):
  File "/Users/DASchenk/Projects/private/handelsregister/handelsregister.py", line 185, in <module>
    companies = h.search_company()
                ^^^^^^^^^^^^^^^^^^
  File "/Users/DASchenk/Projects/private/handelsregister/handelsregister.py", line 76, in search_company
    self.browser.select_form(name="form")
  File "/Users/DASchenk/Projects/private/handelsregister/.venv/lib/python3.11/site-packages/mechanize/_mechanize.py", line 681, in select_form
    raise FormNotFoundError("no form matching " + description)
mechanize._mechanize.FormNotFoundError: no form matching name 'form'

What I discovered so far is that follow_link (on line 72) with "Advanced search" is responding with the main page and not with the "rendered" response of "Erweiterte Suche".

response_search = self.browser.follow_link(text="Advanced search")

Link-element with title "Advanced search" on main page, that is included in the response:

<a tabindex="-1" title="Advanced search" class="ui-menuitem-link ui-corner-all  rpNavMainMenuItem" href="#" onclick="PF(\'sidebar1\').hide();PrimeFaces.ab({s:&quot;j_idt27&quot;,f:&quot;headerForm&quot;});return false;">
  <span class="ui-menuitem-icon ui-icon fa fa-search-plus" aria-hidden="true"></span>
  <span class="ui-menuitem-text">Advanced search</span>
</a>

Could someone with more knowleadge of this lib check if this was working with the current website of Handelsregister or is there a major change that breaks the lib?
I'm glad to help solve this issue, but before I try to overcome the JQuery rendering with Selenium I'd like to know if mechanize was before able to handle this behaviour?

Greetings David