Writer.Close panics when a small upload fails with WithCancelOnError set

Aperta Adatta ai principianti
  • Qualcosa non va?
#54 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
2/5
Tempo stimato
1-3 ore
Idoneità per principianti
82/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
go
Ambito
backend

Direzione di ricerca

Leggi b2/writer.go, in particolare Writer.setErr, sendChunk e Close; l’issue include una causa concreta del nil-pointer e una guard suggerita. Inizia controllando i test Writer esistenti per WithCancelOnError, poi riproduci gli errori per gli upload piccoli e all’avvio dei file di grandi dimensioni. Il lavoro è completato quando questi percorsi restituiscono gli errori di upload senza causare panic e i test pertinenti passano.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

bug

I was adding panic recovery to my libraries, and while analyzing the b2 library and api, and my clanker spotted a possible issue with the cancel recovery. I asked it prepare a small report and reviewed it myself.

With WithCancelOnError set, a small upload that fails makes Writer.Close panic with a nil pointer dereference instead of returning the error. Here "small" means under ChunkSize, sent in a single b2_upload_file call. It happens on any failure: a refused upload (4xx/5xx), a timeout, or a write context cancelled before Close, which is how a caller abandons an upload.

Cause

setErr cancels the large file whenever ctxf is set:

	if w.ctxf == nil {
		return
	}
	if w.errf == nil {
		w.errf = func(error) {}
	}
	w.errf(w.file.cancel(w.ctxf()))

w.file is only assigned when the writer switches to the large-file API (w.file = lf, in sendChunk). A small upload goes through simpleWriteFile from Close instead (L541, L558), so w.file is still nil when that upload's error reaches setErr.

Reproduction

This is against v0.8.0 (also seen on v0.7.2) with Go 1.27.1. A fake B2 API refuses the upload with a 400:

repro_test.go 
package blazerrepro

import (
	"context"
	"encoding/json"
	"net/http"
	"net/http/httptest"
	"testing"

	"github.com/Backblaze/blazer/b2"
)

// fakeB2 answers just enough of the B2 API for a small upload, and refuses
// the upload itself with a 400.
func fakeB2(t *testing.T) string {
	mux := http.NewServeMux()
	srv := httptest.NewServer(mux)
	t.Cleanup(srv.Close)
	mux.HandleFunc("/b2api/v3/b2_authorize_account", func(w http.ResponseWriter, r *http.Request) {
		json.NewEncoder(w).Encode(map[string]any{
			"accountId":          "account",
			"authorizationToken": "token",
			"apiInfo": map[string]any{"storageApi": map[string]any{
				"apiUrl": srv.URL, "downloadUrl": srv.URL,
				"absoluteMinimumPartSize": 5000000, "recommendedPartSize": 100000000,
			}},
		})
	})
	mux.HandleFunc("/b2api/v3/b2_list_buckets", func(w http.ResponseWriter, r *http.Request) {
		json.NewEncoder(w).Encode(map[string]any{"buckets": []map[string]any{{"bucketId": "id", "bucketName": "bucket", "bucketType": "allPrivate"}}})
	})
	mux.HandleFunc("/b2api/v3/b2_get_upload_url", func(w http.ResponseWriter, r *http.Request) {
		json.NewEncoder(w).Encode(map[string]any{"bucketId": "id", "uploadUrl": srv.URL + "/upload", "authorizationToken": "token"})
	})
	mux.HandleFunc("/upload", func(w http.ResponseWriter, r *http.Request) {
		w.WriteHeader(http.StatusBadRequest)
		json.NewEncoder(w).Encode(map[string]any{"status": 400, "code": "bad_request", "message": "refused"})
	})
	return srv.URL
}

func TestFailedSmallUploadWithCancelOnError(t *testing.T) {
	ctx := context.Background()
	client, err := b2.NewClient(ctx, "id", "key", b2.APIBase(fakeB2(t)))
	if err != nil {
		t.Fatal(err)
	}
	bucket, err := client.Bucket(ctx, "bucket")
	if err != nil {
		t.Fatal(err)
	}

	w := bucket.Object("small.txt").NewWriter(ctx, b2.WithCancelOnError(context.Background, nil))
	_, err = w.Write([]byte("well under ChunkSize"))
	if err != nil {
		t.Fatal(err)
	}
	err = w.Close() // want the 400 back as an error
	t.Logf("Close: %v", err)
}

--- FAIL: TestFailedSmallUploadWithCancelOnError (0.00s)
panic: runtime error: invalid memory address or nil pointer dereference [recovered, repanicked]
[signal SIGSEGV: segmentation violation code=0x2 addr=0x18 pc=0x1008d1680]

goroutine 6 [running]:
...
github.com/Backblaze/blazer/b2.(*Writer).setErr(0x5e3d47250680, {0x100e58438, 0x5e3d47238dc0})
	.../github.com/!backblaze/[email protected]/b2/writer.go:125 +0x1b0
github.com/Backblaze/blazer/b2.(*Writer).Close.func1()
	.../github.com/!backblaze/[email protected]/b2/writer.go:558 +0x168
sync.(*Once).doSlow(0x5e3d47250680?, 0x1008d8550?)
	.../src/sync/once.go:78 +0xdc
sync.(*Once).Do(...)
	.../src/sync/once.go:69
github.com/Backblaze/blazer/b2.(*Writer).Close(0x5e3d47250680)
	.../github.com/!backblaze/[email protected]/b2/writer.go:536 +0x44

Cancelling the writer's context before Close, instead of refusing the upload, panics at the same line.

The same nil w.file is also reachable from Write. When a write fills ChunkSize and b2_start_large_file fails, sendChunk returns before assigning w.file, and Write passes that error to setErr (L280-L282). To see it, answer b2_start_large_file with the same 400 in the fake above, then run:

	w := bucket.Object("large.bin").NewWriter(ctx, b2.WithCancelOnError(context.Background, nil))
	w.ChunkSize = 10
	_, err = w.Write([]byte("more than ten bytes, so a large file starts"))

This panics in setErr at writer.go:125, called from writer.go:281.

Suggested fix

When no large file was started, there is nothing to cancel, so setErr can return before the cancel:

	if w.ctxf == nil || w.file == nil {
		return
	}

With that change, the reproduction's Close returns b2_upload_file: 400: refused, and the Write above returns b2_start_large_file: 400: refused.

Until a release carries the fix, we work around it in our wrapper's Close. Because the panic happens inside Close's sync.Once, a second Close returns the error setErr had already recorded.

Lingua principale
Go
Stelle
66
Fork
18
Merge medio
1g 10h
PR unite (30g)
132

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di Backblaze/blazer

Tutte le issue di Backblaze/blazer

Issue simili

Altre issue su Go

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.