Feature Request: Better support for archiving URLs beind HTTP basic auth

Type

• General question or discussion
• Propose a brand new feature
• Request modification of existing behavior or design

What is the problem that your feature request solves

Archive a website behind a login form. I think this is mainly a documentation issue but currently the documentation is very sparse on how to archive webpages behind sign-in prompts. And for HTTP Basic Auth websites it looks like the only possible way is currently to archive them using dept=0 and with the credentials embedded into the URL. dept=1 doesn't work as the credentials aren't preserved to all the discovered URLs and therefore they throw an authentication error. Further more it looks like URLs like http://user:pass@domain are considered to be distinct from http://domain URLs and not automatically detected as being the same but with the login credentials (aka links from other pages won't work, also when depth=1 was used for a site referencing them their version without login is also added to the index).

Describe the ideal specific solution you'd want, and whether it fits into any broader scope of changes

Document how to do capture a website that is behind a credential prompt.

What hacks or alternative solutions have you tried to solve the problem?

If there is no native solution, probably having to use a mitmproxy and disable TLS certificate validation or something similar?

How badly do you want this new feature?

• It's an urgent deal-breaker, I can't live without it
• It's important to add it in the near-mid term future
• It would be nice to have eventually

---

• I'm willing to contribute dev time / money to fix this issue
• I like ArchiveBox so far / would recommend it to a friend
• I've had a lot of difficulty getting ArchiveBox set up