EC Audit: C1: metabase finds an EC part by index prefix, so part 1 also matches parts 10–19
Los mantenedores suelen responder en 1 día
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 2/5
- Tiempo estimado
- 1-3 horas
- Aptitud para principiantes
- 78/100
- Tipo de issue
- Error
- Claridad
- Bien especificado
- Estado de actividad
- Activo
- Stack tecnológico
- go
- Área
- databases, distributed-systems
Línea de trabajo
Empieza en pkg/local_object_storage/metabase/ec.go, cerca de resolveECPartInMetaBucket, y compara la búsqueda de partes con la búsqueda en el índice de reglas. Después, añade una prueba de regresión a las pruebas del motor de almacenamiento local de objetos usando las partes 1 y 10 y siguiendo la reproducción del issue. Ejecuta go test ./pkg/local_object_storage/engine/ -run TestECAuditC1 -v; se considera terminado cuando solicitar la parte 1 devuelve solo la parte 1 o indica que no se encontró si está ausente.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Expected Behavior
A request for part i of an EC rule returns the object whose __NEOFS__EC_PART_IDX is exactly i, or "not found".
Current Behavior
resolveECPartInMetaBucket builds the key prefix …__NEOFS__EC_PART_IDX<delim><i> and takes the first key that starts with it (metabase/ec.go#L189-L203). The value is the tail of the key and nothing ends it, so index 1 also matches 10…19, 2 matches 20…29, and so on up to 6 matching 60…63. When a particular index is requested the function returns at once, without comparing the stored value. The rule index a few lines above is compared with bytes.Equal and is not affected.
Children of the parent are walked in OID order. So the wrong part comes back
- when the shard holds only the part with the longer index, and
- when it holds both and the OID of the longer one sorts first (12 of 20 random cases in the test below).
Any rule with 11 or more parts is affected, for example EC 12/4.
Nothing after the lookup notices:
Shard.GetECPart,HeadECPart,GetECPartRangeand theRead*variants read the resolved object and do not look at its part index (shard/ec.go#L91-L123).- A node that fetches a whole part from another node checks the EC attributes of the reply (get/ec.go#L685), so that case is caught. A range of a part comes without a header (get/ec.go#L1367-L1379) and is taken as is. Ranged GET, the streaming full GET (parts after the first are read as ranges) and the policer's part fetch (policer/ec.go#L440-L443) all read ranges.
- The policer counts a local
HeadECParthit as "part present" (policer/ec.go#L229-L266).
What this means for EC 12/4, by reading the code: the node holding part 1 is down. NodeSequenceForPart(1, 16, …) asks nodes 1, 2, 3, … in turn. Nodes 2–9 say "not found". Node 10 answers with part 10. A ranged read then returns bytes of part 10 in place of part 1, with status OK, instead of rebuilding part 1 from parity.
Possible Solution
Compare the whole key, as is done for the rule index: the value is the last component, so bytes.Equal(k, partPref) is enough when pi.Index >= 0. A test with parts 1 and 10 of one rule would cover it; the existing "another part" case uses 34 and 35.
Steps to Reproduce (for bugs)
- Save the test below as
pkg/local_object_storage/engine/ecaudit_c1_test.go. go test ./pkg/local_object_storage/engine/ -run TestECAuditC1 -v
It passes, that is, it asserts the wrong behaviour:
HeadECPart(index=1) returned part with index 10
GetECPart(index=1) returned payload of part 10 (64 bytes)
GetECPartRange(index=1, off=8, len=16) returned bytes of part 10
part 10 returned for index 1 in 12 of 20 cases (whenever OID(part 10) < OID(part 1))
ecaudit_c1_test.go
package engine
import (
"bytes"
"context"
"io"
"testing"
iec "github.com/nspcc-dev/neofs-node/internal/ec"
"github.com/nspcc-dev/neofs-node/internal/testutil"
"github.com/nspcc-dev/neofs-node/pkg/local_object_storage/blobstor"
cidtest "github.com/nspcc-dev/neofs-sdk-go/container/id/test"
neofscryptotest "github.com/nspcc-dev/neofs-sdk-go/crypto/test"
"github.com/nspcc-dev/neofs-sdk-go/object"
oidtest "github.com/nspcc-dev/neofs-sdk-go/object/id/test"
"github.com/stretchr/testify/require"
)
func TestECAuditC1_PartIndexPrefix(t *testing.T) {
ctx := context.Background()
cnr := cidtest.ID()
signer := neofscryptotest.Signer()
newPart := func(parent object.Object, idx int) object.Object {
obj, err := iec.FormObjectForECPart(signer, parent, testutil.RandByteSlice(64), iec.PartInfo{RuleIndex: 0, Index: idx})
require.NoError(t, err)
return obj
}
t.Run("only part 10 stored, part 1 requested", func(t *testing.T) {
parent := *generateObjectWithCID(cnr)
parent.SetID(oidtest.ID())
e := testNewEngineWithShards(testNewShard(t, 1))
p10 := newPart(parent, 10)
require.NoError(t, e.Put(ctx, &p10, nil))
hdr, err := e.HeadECPart(ctx, cnr, parent.GetID(), iec.PartInfo{RuleIndex: 0, Index: 1})
require.NoError(t, err) // want: ErrObjectNotFound
got, err := iec.GetPartInfo(hdr)
require.NoError(t, err)
t.Logf("HeadECPart(index=1) returned part with index %d, OID %s", got.Index, hdr.GetID())
require.Equal(t, 10, got.Index)
hdr, rc, err := e.GetECPart(ctx, cnr, parent.GetID(), iec.PartInfo{RuleIndex: 0, Index: 1}, false)
require.NoError(t, err)
b, err := io.ReadAll(rc)
require.NoError(t, err)
rc.Close()
require.True(t, bytes.Equal(b, p10.Payload()))
t.Logf("GetECPart(index=1) returned payload of part 10 (%d bytes)", len(b))
_, _, rc, err = e.GetECPartRange(ctx, cnr, parent.GetID(), iec.PartInfo{RuleIndex: 0, Index: 1}, blobstor.NewPayloadRange(8, 16), false)
require.NoError(t, err)
b, err = io.ReadAll(rc)
require.NoError(t, err)
rc.Close()
require.True(t, bytes.Equal(b, p10.Payload()[8:24]))
t.Logf("GetECPartRange(index=1, off=8, len=16) returned bytes of part 10")
})
t.Run("parts 1 and 10 stored, part 1 requested", func(t *testing.T) {
wrong := 0
const n = 20
for range n {
parent := *generateObjectWithCID(cnr)
parent.SetID(oidtest.ID())
e := testNewEngineWithShards(testNewShard(t, 1))
p1, p10 := newPart(parent, 1), newPart(parent, 10)
require.NoError(t, e.Put(ctx, &p1, nil))
require.NoError(t, e.Put(ctx, &p10, nil))
hdr, err := e.HeadECPart(ctx, cnr, parent.GetID(), iec.PartInfo{RuleIndex: 0, Index: 1})
require.NoError(t, err)
if hdr.GetID() == p10.GetID() {
wrong++
require.Negative(t, bytes.Compare(p10.GetID().Marshal(), p1.GetID().Marshal()))
}
}
t.Logf("part 10 returned for index 1 in %d of %d cases (whenever OID(part 10) < OID(part 1))", wrong, n)
require.NotZero(t, wrong)
})
}
Context
Found by an audit of the erasure coding code. The audit was static: code reading plus unit-level probe tests in the packages concerned. No cluster was run, so there is no end-to-end reproduction.
Regression
Not known.
Your Environment
- Version used: 0.58.2 (
1f0d51b5d) - Server setup and configuration: none, unit tests only
- Operating System and version (
uname -a): Darwin 25.6.0 arm64, Go 1.27.3
- Lenguaje dominante
- Go
- Estrellas
- 37
- Forks
- 56
- Merge medio
- 1 d 18 h
- PR fusionados (30 d)
- 42
Preparar el entorno
- Sin Dockerfile ni archivo de Docker Compose
- Sin plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de nspcc-dev/neofs-node
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 75/100
nspcc-dev/neofs-node#4311 ·
Los mantenedores suelen responder en 1 día
-
Check alpha scope in IRAbiertohal:proposed I4 neofs-storage S3 security U4
Dificultad 2/5 1-3 horas Aptitud para principiantes 69/100
nspcc-dev/neofs-node#4280 · 2 comentarios ·
Los mantenedores suelen responder en 1 día
-
bug neofs-storage
Dificultad 3/5 1-2 días Aptitud para principiantes 68/100
nspcc-dev/neofs-node#4333 ·
Los mantenedores suelen responder en 1 día
-
bug neofs-storage
Dificultad 3/5 1-2 días Aptitud para principiantes 65/100
nspcc-dev/neofs-node#4332 ·
Los mantenedores suelen responder en 1 día
-
EC Audit: H6: partly stored EC PUT is answered with "internal server error" and no object IDAbiertobug hal:proposed neofs-storage
Dificultad 3/5 1-2 días Aptitud para principiantes 68/100
nspcc-dev/neofs-node#4331 · 2 comentarios ·
Los mantenedores suelen responder en 1 día
Todos los issues de nspcc-dev/neofs-node
Issues similares
-
Pin and verify Security Insights schema in lintinsights targetPosiblemente ocupada @imshubham22apr-gif la tomó hace 5 días. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 72/100
gemaraproj/gemara#618 ·
Los mantenedores suelen responder en 2 días
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 64/100
a2aproject/a2a-go#597 ·
Los mantenedores suelen responder en 1 día
-
scopes: GetAllowedTopics, GetProtectedTopics, and GetScopedTopics include empty topic names and do not trim whitespace charactersPosiblemente ocupada @patelshivani2283-lab la tomó hoy. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
Los mantenedores suelen responder en 1 día
-
area/networking kind/bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 69/100
siderolabs/talos#14972 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 1/5 Menos de una hora Aptitud para principiantes 72/100
Los mantenedores suelen responder en 1 día