Serialize only assigned audit examples, not the full parent dataset

Aberta
  • Algo errado?
#329 4 comentários 0 reações 0 responsáveis Ver no GitHub

Mantenedores costumam responder em até 1 dia

@0xzoz já está trabalhando nisso.

Desde 12/11/2026.

Avaliação

Dificuldade
4/5
Tempo estimado
3-5 dias
Facilidade para iniciantes
48/100
Tipo de issue
Bug
Clareza
Claramente especificada
Status de atividade
Ativa
Stack de tecnologia
python, pytorch
Domínio
machine-learning, security

Direção de pesquisa

Start with the subset and save code at cache_model_1/services/modelowner.py lines 237–241 and the equivalent in cache_model_0/services/modelowner.py; then inspect related partition writers for the same pattern. Add regression coverage for both services using the synthetic sentinel example and verify after serialization and encryption round trips that only assigned inputs and labels are recoverable while sampling remains deterministic. Done means the saved assignment contains no unselected parent data and its size scales with the assignment.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

Approved

Observed against ed175d0deceddfa2e63175ec7752c800b31c1fbf during the November 10 Fashion-MNIST local test investigation. The subsequent full round with three notebooks and a Raspberry Pi used recorded workarounds; its success does not establish that this upstream defect is fixed. Related: #276 and #40.

Problem and evidence

Both reference model-owner services create a torch.utils.data.Subset(test_data, random_indices) and save that object. A Subset holds its parent dataset; torch.save serializes that parent too. Encryption protects the resulting archive in transit but does not remove unassigned examples from it.

An independent synthetic round-trip reproduced a saved assignment with length 500 whose loaded .dataset still had all 10,000 examples. Example 9,999 was accessible despite being outside the chosen assignment. Our completed round used materialized TensorDataset objects to avoid this behavior; its success therefore does not validate the original serializer.

This is more than a large-file problem. A recipient can inspect examples beyond its assigned shard, undermining the intended restricted audit distribution/resampling boundary for non-public data. The ordinary evaluator still iterates the assigned 500 rows. This finding does not establish that evaluation scored all 10,000 examples or that the observed scores were invalid.

Proposed scope

Materialize the selected examples into a standalone dataset before hashing, encryption and upload. Apply the fix to both reference models and inspect related partition writers for the same serialization pattern. Preserve sampling seeds, selected indices, labels and existing evaluation behavior. Do not combine this with a new sampling-policy design.

Acceptance criteria
  • A synthetic parent has recognizable sentinel examples outside the assigned indices.
  • After save/load (and the encryption/decryption round trip), only assigned examples are recoverable; no parent dataset or unselected tensor storage remains embedded.
  • Exact assigned inputs/labels and deterministic indices match the previous sampling logic.
  • File size scales with the assignment, not the parent dataset.
  • Both reference services have regression coverage.
Minimal synthetic reproduction
import io
import torch
from torch.utils.data import TensorDataset, Subset

parent = TensorDataset(torch.arange(10_000))
assigned = Subset(parent, list(range(500)))
buf = io.BytesIO()
torch.save(assigned, buf)
buf.seek(0)
# Only load this locally generated synthetic object.
loaded = torch.load(buf, weights_only=False)
print(len(loaded), len(loaded.dataset), loaded.dataset[9999][0].item())
# 500 10000 9999
Full-round test report

See Discussion #331 — Fashion-MNIST with three notebooks and a Raspberry Pi for the setup diagram, completed protocol roles, results and experiment adaptations.

Linguagem predominante
Python
Estrelas
4
Forks
7
Merge médio
5d 15h
PRs com merge (30d)
43

Preparar o ambiente

Este projeto não oferece contêiner de desenvolvimento, Dockerfile nem guia de contribuição, então a configuração fica por sua conta: comece pelo README e veja nosso guia da primeira contribuição para os passos gerais.

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de InfiniteZeroFoundation/DevNet

Todas as issues de InfiniteZeroFoundation/DevNet

Issues semelhantes

Mais issues de Python

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.