FreshRSS/FreshRSS

Add new option for CURLOPT_HTTPPROXYTUNNEL

Open

#7.209 aberto em 11 de jan. de 2025

Ver no GitHub
 (20 comments) (0 reactions) (0 assignees)PHP (1.171 forks)batch import
Bug (unconfirmed)Hacktoberfesthelp wanted

Métricas do repositório

Stars
 (15.022 stars)
Métricas de merge de PR
 (Mesclagem média 11d 12h) (36 fundiu PRs em 30d)

Description

Describe the bug

I'm running FreshRSS in a small k8s cluster. By default, everything works fine besides some feeds that fail due to cloudflare challenges. I've had good experiences with just using a vpn for those, so I'm trying to use my vpn over gluetun as a proxy like this https://github.com/FreshRSS/FreshRSS/issues/3965 :

  'curl_options' => 
  array (
CURLOPT_PROXYTYPE => CURLPROXY_HTTP,    
CURLOPT_PROXY => 'gluetun-9d6e4ac1.download.svc.cluster.local',    
CURLOPT_PROXYPORT => 8888,    
  ),

However, I then get the error below, which looks similar to https://github.com/FreshRSS/FreshRSS/issues/3965

A feed could not be found at `https://hnrss.org/frontpage`; the status code is `200` and content-type is `` [https://hnrss.org/frontpage] 

When trying to use curl directly in the pod, it works:

curl --proxy gluetun-9d6e4ac1.download.svc.cluster.local:8888 https://hnrss.org/frontpage -v
*   Trying 10.43.86.62:8888...
* Connected to gluetun-9d6e4ac1.download.svc.cluster.local (10.43.86.62) port 8888 (#0)
* allocate connect buffer
* Establish HTTP proxy tunnel to hnrss.org:443
> CONNECT hnrss.org:443 HTTP/1.1
> Host: hnrss.org:443
> User-Agent: curl/7.88.1
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 OK
< Date: Sat, 11 Jan 2025 01:07:56 GMT
< Transfer-Encoding: chunked
* Ignoring Transfer-Encoding in CONNECT 200 response
< 
* CONNECT phase completed
* CONNECT tunnel established, response 200
* ALPN: offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=hnrss.org
*  start date: Dec  9 08:18:53 2024 GMT
*  expire date: Mar  9 08:18:52 2025 GMT
*  subjectAltName: host "hnrss.org" matched cert's "hnrss.org"
*  issuer: C=US; O=Let's Encrypt; CN=R10
*  SSL certificate verify ok.
* using HTTP/2
* h2h3 [:method: GET]
* h2h3 [:path: /frontpage]
* h2h3 [:scheme: https]
* h2h3 [:authority: hnrss.org]
* h2h3 [user-agent: curl/7.88.1]
* h2h3 [accept: */*]
* Using Stream ID: 1 (easy handle 0x55de7a757ce0)
> GET /frontpage HTTP/2
> Host: hnrss.org
> user-agent: curl/7.88.1
> accept: */*
> 
< HTTP/2 200 
< server: nginx
< date: Sat, 11 Jan 2025 01:07:56 GMT
< content-type: application/xml; charset=utf-8
< last-modified: Fri, 10 Jan 2025 22:48:13 GMT
< x-algolia-url: https://hn.algolia.com/api/v1/search_by_date?numericFilters=created_at_i%3E%3D1735952657&restrictSearchableAttributes=title&tags=front_page
< expires: Sat, 11 Jan 2025 01:19:17 GMT
< cache-control: max-age=900
< x-cache: HIT
< 
<rss version="2.0" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Hacker News: Front Page</title><link>https://news.ycombinator.com/</link><description>Hacker News RSS</description><docs>https://hnrss.org/</docs><generator>hnrss v2.1.1</generator><lastBuildDate>Sat, 11 Jan 2025 01:04:17 +0000</lastBuildDate><atom:link href="https://hnrss.org/frontpage" rel="self" type="application/rss+xml"></atom:link><item><title><![CDATA[Portals and Quake]]></title><description><![CDATA[
<p>Article URL: <a href="https://30fps.net/pages/pvs-portals-and-quake/">https://30fps.net/pages/pvs-portals-and-quake/</a></p>
<p>Comments URL: <a href="https://news.ycombinator.com/item?id=42661185">https://news.ycombinator.com/item?id=42661185</a></p>
<p>Points: 39</p>
<p># Comments: 1</p>
]]></description><pubDate>Fri, 10 Jan 2025 22:48:13 +0000</pubDate><link>https://30fps.net/pages/pvs-portals-and-quake/</link><dc:creator>ibobev</dc:creator><comments>https://news.ycombinator.com/item?id=42661185</comments><guid isPermaLink="false">https://news.ycombinator.com/item?id=42661185</guid></item>(more items)</* Connection #0 to host gluetun-9d6e4ac1.download.svc.cluster.local left intact
channel></rss>

To Reproduce

Given the use of k8s, I think it might be a bit hard to reproduce the exact setup. I figured I'd ask here first for general advice to see if I'm missing something stupid or obvious. I'm happy to try to give a reproduction case using gluetun, free vpn servers, and docker compose if it helps (in hopes that the problem isn't related to something else :' ) )

Expected behavior

When using a proxy configured via curl_options, FreshRSS should successfully parse feeds that return 200 status codes with valid RSS content (as demonstrated by the working curl command).

FreshRSS version

1.25.0

System information

  • Database version: SQLite
  • PHP version: 8.2.26
  • Installation type: Docker
  • Web server type: Apache
  • Device: Laptop
  • OS: debian bookworm on 6.1.0-25-amd64 with k3s v1.31.4+k3s1
  • Browser: Firefox 133.

Additional context

I hope this is the right place to ask this since I've seen related issues, but I'm happy to move this to a discussion instead.

Guia do colaborador