Bound the Client monitor's memory and fix reader and writer registry cleanup

Aperta
  • Qualcosa non va?
#185 1 commento 0 reazioni 1 assegnatario Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

@ffumero2003 ci sta già lavorando.

Dal 28/10/2026.

Valutazione

Questa issue non è ancora stata valutata.

Descrizione

bug

Context

Every b2.Client records every HTTP call in four x/window counters, one of them "forever". The reducer appends each call to a slice, and a zero-size window keeps its value indefinitely, so memory grows by one record per request for the life of the client. This is always on: NewClient installs the recording transport whether or not the caller uses Status or ServeHTTP, and every request takes the client-wide slock.

The registries behind Status have related problems:

  • readers are removed only by Reader.Close; a reader read to EOF but not closed (io.ReadAll and io.Copy do not close) stays in Client.sReaders with its chunk buffers (up to ConcurrentDownloads x 10 MB);
  • a writer closed without writing is never removed: Close takes the never-started branch, which registers the writer and returns before the deferred removeWriter;
  • both maps are keyed by bucket/name, so two concurrent transfers of one name overwrite each other's entry, and the first to finish removes the other's.
Current behavior

At a5e0d13:

  • b2/b2.go:63-68: newMethodCounter(0, 0), // forever.
  • b2/monitor.go:75-94: the reducer appends every record.
  • x/window/window.go:53-67, :133-136 and the package doc at :43-44 (slice reducers grow linearly).
  • b2/b2.go:204-229: clientTransport.RoundTrip records every call under slock; b2/baseline.go:200-206: authOptions always installs it, both when authorizing and when restoring saved auth data.
  • b2/monitor.go:153-195: registries keyed by fmt.Sprintf("%s/%s", bucket, name).
  • b2/reader.go:280 and :104: added in initFunc, removed only in Close.
  • b2/writer.go:242: added in init; b2/writer.go:719-726: a writer that never started calls init and returns before the deferred removeWriter at :728. b2/sync/sync.go:393-399: with WithCompare(CompareSHA1) the sync engine passes an io.TeeReader, so an empty file is copied without any Write and takes this branch.

Reproduced locally: 50 empty writers closed leave 50 registry entries. The RPC growth follows from the code: 10 million requests retain 10 million records.

Expected behavior

The monitor uses bounded memory regardless of client lifetime, and Status lists each active transfer exactly once.

Acceptance criteria

  • R1. One million recorded calls keep the monitor's retained size constant (benchmark or heap check).
  • R2. A reader read to EOF without Close is no longer listed in Status().
  • R3. Two concurrent writers of the same name both appear, and finishing one leaves the other listed.
  • R4. A writer closed without writing is not listed in Status() after Close returns.

In scope

Proposed change
  1. Replace the all-time slice with per-method counters (count, total duration, status histogram), or drop the forever window; aggregate the bounded windows the same way.
  2. Make RPC recording opt-in (for example a WithMonitoring() client option) for those who use ServeHTTP.
  3. Key registries by a unique per-transfer ID; remove readers on EOF or error as well as on Close; remove a writer on every Close path, including the one for a writer that never wrote (or register it only once data is sent).

Out of scope

  • Unrelated work outside the components and behavior described in In scope.

Verification

Run the repository regression checks with live credentials unset:

unset B2_APPLICATION_KEY_ID B2_APPLICATION_KEY B2_ACCOUNT_ID B2_ACCOUNT_KEY
go build ./...
go vet ./...
go test -race -count=1 ./b2 ./x/window
  • Record the benchmark or heap check for one million calls and the retained-size result, as required above.
  • Test reader EOF without Close, closing an empty writer, and two concurrent same-name transfers; assert the stated registry contents after each transition.
  • Expected result: the checks above succeed and the observable outcomes in Acceptance criteria are demonstrated. If a check cannot run, report Could not run because: ...; if it starts but fails, report Could not complete because: .... A skipped check is not a pass.

Links

None

Lingua principale
Go
Stelle
66
Fork
18
Merge medio
1g 10h
PR unite (30g)
132

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di Backblaze/blazer

Tutte le issue di Backblaze/blazer

Issue simili

Altre issue su Go

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.