Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

perf: Upstream transports keep only 2 idle connections per host

Aperta Adatta ai principianti
  • Qualcosa non va?
#1,546 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
80/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
go
Ambito
backend, networking

Direzione di ricerca

Leggi core/tlsbridge/upstream.go e core/listener/forwardproxy/server.go nelle impostazioni di trasporto indicate nell’issue. Aggiungi in entrambi i punti l’impostazione inclusa per il pool di connessioni, quindi esegui più volte l’esperimento fornito TestUpstreamPoolReuse da core/tlsbridge/. Il risultato è raggiunto quando viene aperta circa una connessione upstream per worker, anziché un numero molte volte superiore.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug

What happens

The proxy keeps only 2 idle upstream connections per host. When more than two requests to one host overlap, each extra request opens a new TCP+TLS connection rather than reusing one, and that connection is closed afterwards instead of being kept for the next request.

This hurts most on a gateway that speaks only HTTP/1.1, where each connection carries one request at a time. LiteLLM is one: it offers no h2 in ALPN. Each extra handshake costs about 40-90 ms on the laptop's VPN-reached gateways.

Reproduction

The bridge's own upstream client (tlsbridge.NewUpstreamClient), callers running at the same time, 10 sequential requests each, against one h1-only TLS origin. We counted the TCP connections the origin accepted, over three runs:

Concurrent callers Ideal Today With MaxIdleConnsPerHost: 100
4 4 5 / 16 / 19 5 / 5 / 5
10 10 36 / 39 / 45 18 / 19 / 23
20 20 66 / 76 / 79 46 / 49 / 49

Through the forward proxy end to end (10 bridged clients × 10 requests), the origin accepted 69 connections.

Test source (drop into core/tlsbridge/) 
package tlsbridge

import (
	"crypto/tls"
	"io"
	"net"
	"net/http"
	"net/http/httptest"
	"sync"
	"sync/atomic"
	"testing"
)

func TestUpstreamPoolReuse(t *testing.T) {
	var conns atomic.Int32
	origin := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { io.WriteString(w, "ok") }))
	origin.Config.ConnState = func(_ net.Conn, s http.ConnState) {
		if s == http.StateNew {
			conns.Add(1)
		}
	}
	origin.TLS = &tls.Config{NextProtos: []string{"http/1.1"}} // h1-only, like LiteLLM
	origin.StartTLS()
	defer origin.Close()
	const workers = 10
	c, _ := NewUpstreamClient(nil, true)
	var wg sync.WaitGroup
	for i := 0; i < workers; i++ {
		wg.Add(1)
		go func() {
			defer wg.Done()
			for j := 0; j < 10; j++ {
				if resp, err := c.Get(origin.URL); err == nil {
					io.Copy(io.Discard, resp.Body)
					resp.Body.Close()
				}
			}
		}()
	}
	wg.Wait()
	t.Logf("%d callers x 10 requests: %d upstream connections (ideal %d)", workers, conns.Load(), workers)
}

Why

Both transports set MaxIdleConns: 100 (the total across all hosts) but not MaxIdleConnsPerHost, so Go's DefaultMaxIdleConnsPerHost of 2 applies:

  • core/tlsbridge/upstream.go:31: every decrypted (bridged) request goes through this transport.
  • core/listener/forwardproxy/server.go:212: plain-HTTP forward-proxy requests.

Possible fix

Add MaxIdleConnsPerHost: 100 next to MaxIdleConns in both transports. Two lines.

Even with the fix, concurrent callers still open more connections than the ideal; Go dials a new connection whenever none is idle at that moment. A MaxConnsPerHost cap would bound that too, but it makes extra requests wait in a queue, so it is a separate decision.

Acceptance

The test above opens about workers connections, not several times that, across repeated runs.

Related, not verified

proxy.log on a laptop shows 65 fast EOF failures (128-221 ms) on LiteLLM POSTs. That is the usual sign of reusing a connection the server had just closed. Go retries such a request only when it can replay the body, and the forward proxy never sets r.GetBody after buffering the body (setBody, server.go:1306). We did not reproduce this, so it is noted here rather than filed.

Found by an experiment-driven audit of main at 43cf5539; every number above was re-measured on that commit.

Assisted-By: Claude (Anthropic AI) [email protected]

Lingua principale
Go
Stelle
21
Fork
43
Merge medio
14h 25m
PR unite (30g)
245

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di rossoctl/cortex

Tutte le issue di rossoctl/cortex

Issue simili

Altre issue su Go

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.