EC Audit: C1: metabase finds an EC part by index prefix, so part 1 also matches parts 10–19
I maintainer di solito rispondono entro 1 giorno
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 2/5
- Tempo stimato
- 1-3 ore
- Idoneità per principianti
- 78/100
- Tipo di issue
- Bug
- Chiarezza
- Specificata chiaramente
- Stato di attività
- Attiva
- Stack tecnologico
- go
- Ambito
- databases, distributed-systems
Direzione di ricerca
Inizia da pkg/local_object_storage/metabase/ec.go, vicino a resolveECPartInMetaBucket, e confronta la ricerca delle parti con quella dell’indice delle regole. Poi aggiungi un test di regressione ai test del motore di archiviazione locale degli oggetti, usando le parti 1 e 10 e seguendo la riproduzione nell’issue. Esegui go test ./pkg/local_object_storage/engine/ -run TestECAuditC1 -v; il lavoro è completato quando la richiesta della parte 1 restituisce solo la parte 1 oppure “non trovato” se è assente.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Expected Behavior
A request for part i of an EC rule returns the object whose __NEOFS__EC_PART_IDX is exactly i, or "not found".
Current Behavior
resolveECPartInMetaBucket builds the key prefix …__NEOFS__EC_PART_IDX<delim><i> and takes the first key that starts with it (metabase/ec.go#L189-L203). The value is the tail of the key and nothing ends it, so index 1 also matches 10…19, 2 matches 20…29, and so on up to 6 matching 60…63. When a particular index is requested the function returns at once, without comparing the stored value. The rule index a few lines above is compared with bytes.Equal and is not affected.
Children of the parent are walked in OID order. So the wrong part comes back
- when the shard holds only the part with the longer index, and
- when it holds both and the OID of the longer one sorts first (12 of 20 random cases in the test below).
Any rule with 11 or more parts is affected, for example EC 12/4.
Nothing after the lookup notices:
Shard.GetECPart,HeadECPart,GetECPartRangeand theRead*variants read the resolved object and do not look at its part index (shard/ec.go#L91-L123).- A node that fetches a whole part from another node checks the EC attributes of the reply (get/ec.go#L685), so that case is caught. A range of a part comes without a header (get/ec.go#L1367-L1379) and is taken as is. Ranged GET, the streaming full GET (parts after the first are read as ranges) and the policer's part fetch (policer/ec.go#L440-L443) all read ranges.
- The policer counts a local
HeadECParthit as "part present" (policer/ec.go#L229-L266).
What this means for EC 12/4, by reading the code: the node holding part 1 is down. NodeSequenceForPart(1, 16, …) asks nodes 1, 2, 3, … in turn. Nodes 2–9 say "not found". Node 10 answers with part 10. A ranged read then returns bytes of part 10 in place of part 1, with status OK, instead of rebuilding part 1 from parity.
Possible Solution
Compare the whole key, as is done for the rule index: the value is the last component, so bytes.Equal(k, partPref) is enough when pi.Index >= 0. A test with parts 1 and 10 of one rule would cover it; the existing "another part" case uses 34 and 35.
Steps to Reproduce (for bugs)
- Save the test below as
pkg/local_object_storage/engine/ecaudit_c1_test.go. go test ./pkg/local_object_storage/engine/ -run TestECAuditC1 -v
It passes, that is, it asserts the wrong behaviour:
HeadECPart(index=1) returned part with index 10
GetECPart(index=1) returned payload of part 10 (64 bytes)
GetECPartRange(index=1, off=8, len=16) returned bytes of part 10
part 10 returned for index 1 in 12 of 20 cases (whenever OID(part 10) < OID(part 1))
ecaudit_c1_test.go
package engine
import (
"bytes"
"context"
"io"
"testing"
iec "github.com/nspcc-dev/neofs-node/internal/ec"
"github.com/nspcc-dev/neofs-node/internal/testutil"
"github.com/nspcc-dev/neofs-node/pkg/local_object_storage/blobstor"
cidtest "github.com/nspcc-dev/neofs-sdk-go/container/id/test"
neofscryptotest "github.com/nspcc-dev/neofs-sdk-go/crypto/test"
"github.com/nspcc-dev/neofs-sdk-go/object"
oidtest "github.com/nspcc-dev/neofs-sdk-go/object/id/test"
"github.com/stretchr/testify/require"
)
func TestECAuditC1_PartIndexPrefix(t *testing.T) {
ctx := context.Background()
cnr := cidtest.ID()
signer := neofscryptotest.Signer()
newPart := func(parent object.Object, idx int) object.Object {
obj, err := iec.FormObjectForECPart(signer, parent, testutil.RandByteSlice(64), iec.PartInfo{RuleIndex: 0, Index: idx})
require.NoError(t, err)
return obj
}
t.Run("only part 10 stored, part 1 requested", func(t *testing.T) {
parent := *generateObjectWithCID(cnr)
parent.SetID(oidtest.ID())
e := testNewEngineWithShards(testNewShard(t, 1))
p10 := newPart(parent, 10)
require.NoError(t, e.Put(ctx, &p10, nil))
hdr, err := e.HeadECPart(ctx, cnr, parent.GetID(), iec.PartInfo{RuleIndex: 0, Index: 1})
require.NoError(t, err) // want: ErrObjectNotFound
got, err := iec.GetPartInfo(hdr)
require.NoError(t, err)
t.Logf("HeadECPart(index=1) returned part with index %d, OID %s", got.Index, hdr.GetID())
require.Equal(t, 10, got.Index)
hdr, rc, err := e.GetECPart(ctx, cnr, parent.GetID(), iec.PartInfo{RuleIndex: 0, Index: 1}, false)
require.NoError(t, err)
b, err := io.ReadAll(rc)
require.NoError(t, err)
rc.Close()
require.True(t, bytes.Equal(b, p10.Payload()))
t.Logf("GetECPart(index=1) returned payload of part 10 (%d bytes)", len(b))
_, _, rc, err = e.GetECPartRange(ctx, cnr, parent.GetID(), iec.PartInfo{RuleIndex: 0, Index: 1}, blobstor.NewPayloadRange(8, 16), false)
require.NoError(t, err)
b, err = io.ReadAll(rc)
require.NoError(t, err)
rc.Close()
require.True(t, bytes.Equal(b, p10.Payload()[8:24]))
t.Logf("GetECPartRange(index=1, off=8, len=16) returned bytes of part 10")
})
t.Run("parts 1 and 10 stored, part 1 requested", func(t *testing.T) {
wrong := 0
const n = 20
for range n {
parent := *generateObjectWithCID(cnr)
parent.SetID(oidtest.ID())
e := testNewEngineWithShards(testNewShard(t, 1))
p1, p10 := newPart(parent, 1), newPart(parent, 10)
require.NoError(t, e.Put(ctx, &p1, nil))
require.NoError(t, e.Put(ctx, &p10, nil))
hdr, err := e.HeadECPart(ctx, cnr, parent.GetID(), iec.PartInfo{RuleIndex: 0, Index: 1})
require.NoError(t, err)
if hdr.GetID() == p10.GetID() {
wrong++
require.Negative(t, bytes.Compare(p10.GetID().Marshal(), p1.GetID().Marshal()))
}
}
t.Logf("part 10 returned for index 1 in %d of %d cases (whenever OID(part 10) < OID(part 1))", wrong, n)
require.NotZero(t, wrong)
})
}
Context
Found by an audit of the erasure coding code. The audit was static: code reading plus unit-level probe tests in the packages concerned. No cluster was run, so there is no end-to-end reproduction.
Regression
Not known.
Your Environment
- Version used: 0.58.2 (
1f0d51b5d) - Server setup and configuration: none, unit tests only
- Operating System and version (
uname -a): Darwin 25.6.0 arm64, Go 1.27.3
- Lingua principale
- Go
- Stelle
- 37
- Fork
- 56
- Merge medio
- 1g 18h
- PR unite (30g)
- 42
Preparare l'ambiente
- Nessun Dockerfile né file Docker Compose
- Nessun modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di nspcc-dev/neofs-node
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 75/100
nspcc-dev/neofs-node#4311 ·
I maintainer di solito rispondono entro 1 giorno
-
Check alpha scope in IRApertahal:proposed I4 neofs-storage S3 security U4
Difficoltà 2/5 1-3 ore Idoneità per principianti 69/100
nspcc-dev/neofs-node#4280 · 2 commenti ·
I maintainer di solito rispondono entro 1 giorno
-
bug neofs-storage
Difficoltà 3/5 1-2 giorni Idoneità per principianti 68/100
nspcc-dev/neofs-node#4333 ·
I maintainer di solito rispondono entro 1 giorno
-
bug neofs-storage
Difficoltà 3/5 1-2 giorni Idoneità per principianti 65/100
nspcc-dev/neofs-node#4332 ·
I maintainer di solito rispondono entro 1 giorno
-
bug hal:proposed neofs-storage
Difficoltà 3/5 1-2 giorni Idoneità per principianti 68/100
nspcc-dev/neofs-node#4331 · 2 commenti ·
I maintainer di solito rispondono entro 1 giorno
Tutte le issue di nspcc-dev/neofs-node
Issue simili
-
area/networking kind/bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 69/100
siderolabs/talos#14972 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 72/100
I maintainer di solito rispondono entro 1 giorno
-
kubectl wait outputs cryptic "error: unrecognized condition: []" when --for is omittedForse già presa @muskan-creates352 l’ha presa oggi. Apertakind/bug needs-triage
Difficoltà 2/5 1-3 ore Idoneità per principianti 84/100
kubernetes/kubectl#1910 · 1 commento ·
-
needs-triage
Difficoltà 2/5 1-3 ore Idoneità per principianti 66/100
stacklok/toolhive-catalog#1967 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 2/5 1-3 ore Idoneità per principianti 66/100
unbound-force/gaze#353 ·
I maintainer di solito rispondono entro 1 giorno