Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

perf: Upstream transports keep only 2 idle connections per host

Open Beginner friendly
  • Is something wrong?
#1,546 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
80/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
go
Domain
backend, networking

Research direction

Read core/tlsbridge/upstream.go and core/listener/forwardproxy/server.go at the transport settings named in the issue. Add the included connection pool setting in both places, then run the provided TestUpstreamPoolReuse experiment from core/tlsbridge/ across repeated runs. Done means it opens about one upstream connection per worker, rather than several times that.

Written by the indexing model from the issue text.

Description

bug

What happens

The proxy keeps only 2 idle upstream connections per host. When more than two requests to one host overlap, each extra request opens a new TCP+TLS connection rather than reusing one, and that connection is closed afterwards instead of being kept for the next request.

This hurts most on a gateway that speaks only HTTP/1.1, where each connection carries one request at a time. LiteLLM is one: it offers no h2 in ALPN. Each extra handshake costs about 40-90 ms on the laptop's VPN-reached gateways.

Reproduction

The bridge's own upstream client (tlsbridge.NewUpstreamClient), callers running at the same time, 10 sequential requests each, against one h1-only TLS origin. We counted the TCP connections the origin accepted, over three runs:

Concurrent callers Ideal Today With MaxIdleConnsPerHost: 100
4 4 5 / 16 / 19 5 / 5 / 5
10 10 36 / 39 / 45 18 / 19 / 23
20 20 66 / 76 / 79 46 / 49 / 49

Through the forward proxy end to end (10 bridged clients × 10 requests), the origin accepted 69 connections.

Test source (drop into core/tlsbridge/) 
package tlsbridge

import (
	"crypto/tls"
	"io"
	"net"
	"net/http"
	"net/http/httptest"
	"sync"
	"sync/atomic"
	"testing"
)

func TestUpstreamPoolReuse(t *testing.T) {
	var conns atomic.Int32
	origin := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { io.WriteString(w, "ok") }))
	origin.Config.ConnState = func(_ net.Conn, s http.ConnState) {
		if s == http.StateNew {
			conns.Add(1)
		}
	}
	origin.TLS = &tls.Config{NextProtos: []string{"http/1.1"}} // h1-only, like LiteLLM
	origin.StartTLS()
	defer origin.Close()
	const workers = 10
	c, _ := NewUpstreamClient(nil, true)
	var wg sync.WaitGroup
	for i := 0; i < workers; i++ {
		wg.Add(1)
		go func() {
			defer wg.Done()
			for j := 0; j < 10; j++ {
				if resp, err := c.Get(origin.URL); err == nil {
					io.Copy(io.Discard, resp.Body)
					resp.Body.Close()
				}
			}
		}()
	}
	wg.Wait()
	t.Logf("%d callers x 10 requests: %d upstream connections (ideal %d)", workers, conns.Load(), workers)
}

Why

Both transports set MaxIdleConns: 100 (the total across all hosts) but not MaxIdleConnsPerHost, so Go's DefaultMaxIdleConnsPerHost of 2 applies:

  • core/tlsbridge/upstream.go:31: every decrypted (bridged) request goes through this transport.
  • core/listener/forwardproxy/server.go:212: plain-HTTP forward-proxy requests.

Possible fix

Add MaxIdleConnsPerHost: 100 next to MaxIdleConns in both transports. Two lines.

Even with the fix, concurrent callers still open more connections than the ideal; Go dials a new connection whenever none is idle at that moment. A MaxConnsPerHost cap would bound that too, but it makes extra requests wait in a queue, so it is a separate decision.

Acceptance

The test above opens about workers connections, not several times that, across repeated runs.

Related, not verified

proxy.log on a laptop shows 65 fast EOF failures (128-221 ms) on LiteLLM POSTs. That is the usual sign of reusing a connection the server had just closed. Go retries such a request only when it can replay the body, and the forward proxy never sets r.GetBody after buffering the body (setBody, server.go:1306). We did not reproduce this, so it is noted here rather than filed.

Found by an experiment-driven audit of main at 43cf5539; every number above was re-measured on that commit.

Assisted-By: Claude (Anthropic AI) [email protected]

Dominant language
Go
Stars
21
Forks
43
Avg merge
14h 25m
Merged PRs (30d)
245

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from rossoctl/cortex

All issues in rossoctl/cortex

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.