Is the HTTPS proxy support known-working under real-world conditions?

まず lib/net/http.rb と Net::Protocol#ssl_socket_connect の参照されている接続フローを読み、次に HTTPS プロキシに対して ProxySock の例を実行します。HTTPS プロキシが説明どおりに失敗するかを確認し、再現可能な結果または検証済みの修正方針を文書化できれば完了です。

I've been debugging HTTPS proxy support all day and I'm coming to the conclusion the released implementation may be be broken.

Stepping through an example invocation:

n = Net::HTTP.new('google.com', 443, '127.0.0.1', 4433, nil, nil, nil, true)
n.use_ssl = true
n.get('/')

• connect is invoked
• Which opens a TCP socket to the proxy]
• And then opens a SSL socket over the TCP proxy socket
• It then sends the CONNECT command with the needed headers
• Then, after more setup, it attempts to open the endpoint SSL socket against the raw proxy socket, not the proxy SSL socket

I don't believe the last step works.

To test I banged out a little helper class that does the initial proxy connection setup:

require 'net/http'

class ProxySock
  attr_accessor :proxy_sock, :s

  def initialize
    @s = TCPSocket.open('127.0.0.1', 4433, nil, nil)
    @proxy_sock = OpenSSL::SSL::SSLSocket.new(@s)
    Net::Protocol.new.send(:ssl_socket_connect, @proxy_sock, 1.0)
  end

  def close
    @proxy_sock.close
  ensure
    @s.close
  end
end

Then, as a baseline, I checked that basic HTTP proxying was working:

ps = ProxySock.new
begin
  ps.proxy_sock.write("GET http://google.com/ HTTP/1.1\r\n\r\n")
  puts ps.proxy_sock.gets("\r\n\r\n")
ensure
  ps.close
end

This returns the expected HTTP/1.1 301 Moved Permanently Location: http://www.google.com/ response.

Then, I tried the flow Net::HTTP currently does:

ps = ProxySock.new
begin
  ps.proxy_sock.write("CONNECT google.com:443 HTTP/1.1\r\n\r\n")
  puts ps.proxy_sock.gets("\r\n\r\n")
  
  endpoint_sock = OpenSSL::SSL::SSLSocket.new(ps.s)
  Net::Protocol.new.send(:ssl_socket_connect, endpoint_sock, 1.0)
  endpoint_sock.write("GET http://google.com/ HTTP/1.1\r\n\r\n")
  puts endpoint_sock.gets("\r\n\r\n")
ensure
  ps.close
end

This throws the following error, which is the same error I get from Net::HTTP:

home/tom/.rbenv/versions/3.4.1/lib/ruby/3.4.0/net/protocol.rb:46:in 'OpenSSL::SSL::SSLSocket#connect_nonblock': SSL_connect returned=1 errno=0 peeraddr=127.0.0.1:4433 state=error: invalid alert (OpenSSL::SSL::SSLError)
        from /home/tom/.rbenv/versions/3.4.1/lib/ruby/3.4.0/net/protocol.rb:46:in 'Net::Protocol#ssl_socket_connect'
        from tmp/logic_test.rb:59:in '<main>'

As a second test I tried performing another HTTP proxy test, this time using CONNECT:

ps = ProxySock.new
begin
  ps.proxy_sock.write("CONNECT google.com:80 HTTP/1.1\r\n\r\n")
  puts ps.proxy_sock.gets("\r\n\r\n")

  ps.proxy_sock.write("GET http://google.com/ HTTP/1.1\r\n\r\n")
  puts ps.proxy_sock.gets("\r\n\r\n")

  ps.s.write("GET http://google.com/ HTTP/1.1\r\n\r\n")
  puts ps.s.gets("\r\n\r\n")
ensure
  ps.close
end

This outputs two blocks. The first block uses the SSL socket and returns HTTP/1.1 301 Moved Permanently, as expected. The second block attempts to use the underlying TCP socket, same as we're trying to do for the endpoint SSL socket, and that returns �*o�Ń7�t��4��w4Q���k�9o� which appears to be encrypted data.

When using a HTTPS proxy the socket s IO will be encrypted, I don't believe this is the correct handle to use for the endpoint encryption.. I believe we need to initialize the endpoint ssl over the proxy_sock to nest the encryption.

To this end, I tried endpoint_sock = OpenSSL::SSL::SSLSocket.new(ps.proxy_sock) but that simply throws wrong argument type OpenSSL::SSL::SSLSocket (expected File) (TypeError). So I don't have a working HTTPS over HTTPS proxy example on hand.

I'm currently of the opinion the implementation here is broken. Am I mistaken? Is there a flaw in my analysis and test cases?