Serialize only assigned audit examples, not the full parent dataset

Ouverte
  • Quelque chose ne va pas ?
#329 4 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Les mainteneurs répondent en général sous 1 jour

@0xzoz y travaille déjà.

Depuis le 12/11/2026.

Évaluation

Difficulté
4/5
Temps estimé
3-5 jours
Accessibilité débutants
48/100
Type d'issue
Bug
Clarté
Clairement spécifiée
Activité
Active
Stack technique
python, pytorch
Domaine
machine-learning, security

Piste de recherche

Start with the subset and save code at cache_model_1/services/modelowner.py lines 237–241 and the equivalent in cache_model_0/services/modelowner.py; then inspect related partition writers for the same pattern. Add regression coverage for both services using the synthetic sentinel example and verify after serialization and encryption round trips that only assigned inputs and labels are recoverable while sampling remains deterministic. Done means the saved assignment contains no unselected parent data and its size scales with the assignment.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

Approved

Observed against ed175d0deceddfa2e63175ec7752c800b31c1fbf during the November 10 Fashion-MNIST local test investigation. The subsequent full round with three notebooks and a Raspberry Pi used recorded workarounds; its success does not establish that this upstream defect is fixed. Related: #276 and #40.

Problem and evidence

Both reference model-owner services create a torch.utils.data.Subset(test_data, random_indices) and save that object. A Subset holds its parent dataset; torch.save serializes that parent too. Encryption protects the resulting archive in transit but does not remove unassigned examples from it.

An independent synthetic round-trip reproduced a saved assignment with length 500 whose loaded .dataset still had all 10,000 examples. Example 9,999 was accessible despite being outside the chosen assignment. Our completed round used materialized TensorDataset objects to avoid this behavior; its success therefore does not validate the original serializer.

This is more than a large-file problem. A recipient can inspect examples beyond its assigned shard, undermining the intended restricted audit distribution/resampling boundary for non-public data. The ordinary evaluator still iterates the assigned 500 rows. This finding does not establish that evaluation scored all 10,000 examples or that the observed scores were invalid.

Proposed scope

Materialize the selected examples into a standalone dataset before hashing, encryption and upload. Apply the fix to both reference models and inspect related partition writers for the same serialization pattern. Preserve sampling seeds, selected indices, labels and existing evaluation behavior. Do not combine this with a new sampling-policy design.

Acceptance criteria
  • A synthetic parent has recognizable sentinel examples outside the assigned indices.
  • After save/load (and the encryption/decryption round trip), only assigned examples are recoverable; no parent dataset or unselected tensor storage remains embedded.
  • Exact assigned inputs/labels and deterministic indices match the previous sampling logic.
  • File size scales with the assignment, not the parent dataset.
  • Both reference services have regression coverage.
Minimal synthetic reproduction
import io
import torch
from torch.utils.data import TensorDataset, Subset

parent = TensorDataset(torch.arange(10_000))
assigned = Subset(parent, list(range(500)))
buf = io.BytesIO()
torch.save(assigned, buf)
buf.seek(0)
# Only load this locally generated synthetic object.
loaded = torch.load(buf, weights_only=False)
print(len(loaded), len(loaded.dataset), loaded.dataset[9999][0].item())
# 500 10000 9999
Full-round test report

See Discussion #331 — Fashion-MNIST with three notebooks and a Raspberry Pi for the setup diagram, completed protocol roles, results and experiment adaptations.

Langage dominant
Python
Étoiles
4
Forks
7
Merge moyen
5 j 15 h
PR mergées (30 j)
43

Préparer son environnement

Ce projet ne fournit ni conteneur de développement, ni Dockerfile, ni guide de contribution : l'installation est à votre charge. Commencez par son README, et consultez notre guide de la première contribution pour les étapes générales.

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Autres issues de InfiniteZeroFoundation/DevNet

Toutes les issues de InfiniteZeroFoundation/DevNet

Issues similaires

Plus d'issues Python

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.