Is the HTTPS proxy support known-working under real-world conditions?

Start by reading the referenced connection flow in lib/net/http.rb and Net::Protocol#ssl_socket_connect, then run the ProxySock examples against the HTTPS proxy. Done means confirming whether HTTPS proxying fails as described and documenting a reproducible result or a validated correction path.

I've been debugging HTTPS proxy support all day and I'm coming to the conclusion the released implementation may be be broken.

Stepping through an example invocation:

n = Net::HTTP.new('google.com', 443, '127.0.0.1', 4433, nil, nil, nil, true)
n.use_ssl = true
n.get('/')

• connect is invoked
• Which opens a TCP socket to the proxy]
• And then opens a SSL socket over the TCP proxy socket
• It then sends the CONNECT command with the needed headers
• Then, after more setup, it attempts to open the endpoint SSL socket against the raw proxy socket, not the proxy SSL socket

I don't believe the last step works.

To test I banged out a little helper class that does the initial proxy connection setup:

require 'net/http'

class ProxySock
  attr_accessor :proxy_sock, :s

  def initialize
    @s = TCPSocket.open('127.0.0.1', 4433, nil, nil)
    @proxy_sock = OpenSSL::SSL::SSLSocket.new(@s)
    Net::Protocol.new.send(:ssl_socket_connect, @proxy_sock, 1.0)
  end

  def close
    @proxy_sock.close
  ensure
    @s.close
  end
end

Then, as a baseline, I checked that basic HTTP proxying was working:

ps = ProxySock.new
begin
  ps.proxy_sock.write("GET http://google.com/ HTTP/1.1\r\n\r\n")
  puts ps.proxy_sock.gets("\r\n\r\n")
ensure
  ps.close
end

This returns the expected HTTP/1.1 301 Moved Permanently Location: http://www.google.com/ response.

Then, I tried the flow Net::HTTP currently does:

ps = ProxySock.new
begin
  ps.proxy_sock.write("CONNECT google.com:443 HTTP/1.1\r\n\r\n")
  puts ps.proxy_sock.gets("\r\n\r\n")
  
  endpoint_sock = OpenSSL::SSL::SSLSocket.new(ps.s)
  Net::Protocol.new.send(:ssl_socket_connect, endpoint_sock, 1.0)
  endpoint_sock.write("GET http://google.com/ HTTP/1.1\r\n\r\n")
  puts endpoint_sock.gets("\r\n\r\n")
ensure
  ps.close
end

This throws the following error, which is the same error I get from Net::HTTP:

home/tom/.rbenv/versions/3.4.1/lib/ruby/3.4.0/net/protocol.rb:46:in 'OpenSSL::SSL::SSLSocket#connect_nonblock': SSL_connect returned=1 errno=0 peeraddr=127.0.0.1:4433 state=error: invalid alert (OpenSSL::SSL::SSLError)
        from /home/tom/.rbenv/versions/3.4.1/lib/ruby/3.4.0/net/protocol.rb:46:in 'Net::Protocol#ssl_socket_connect'
        from tmp/logic_test.rb:59:in '<main>'

As a second test I tried performing another HTTP proxy test, this time using CONNECT:

ps = ProxySock.new
begin
  ps.proxy_sock.write("CONNECT google.com:80 HTTP/1.1\r\n\r\n")
  puts ps.proxy_sock.gets("\r\n\r\n")

  ps.proxy_sock.write("GET http://google.com/ HTTP/1.1\r\n\r\n")
  puts ps.proxy_sock.gets("\r\n\r\n")

  ps.s.write("GET http://google.com/ HTTP/1.1\r\n\r\n")
  puts ps.s.gets("\r\n\r\n")
ensure
  ps.close
end

This outputs two blocks. The first block uses the SSL socket and returns HTTP/1.1 301 Moved Permanently, as expected. The second block attempts to use the underlying TCP socket, same as we're trying to do for the endpoint SSL socket, and that returns �*o�Ń7�t��4��w4Q���k�9o� which appears to be encrypted data.

When using a HTTPS proxy the socket s IO will be encrypted, I don't believe this is the correct handle to use for the endpoint encryption.. I believe we need to initialize the endpoint ssl over the proxy_sock to nest the encryption.

To this end, I tried endpoint_sock = OpenSSL::SSL::SSLSocket.new(ps.proxy_sock) but that simply throws wrong argument type OpenSSL::SSL::SSLSocket (expected File) (TypeError). So I don't have a working HTTPS over HTTPS proxy example on hand.

I'm currently of the opinion the implementation here is broken. Am I mistaken? Is there a flaw in my analysis and test cases?