Serialize only assigned audit examples, not the full parent dataset

Offen
  • Stimmt etwas nicht?
#329 4 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen

Maintainer antworten meist innerhalb von 1 Tag

@0xzoz arbeitet bereits daran.

Seit 12.11.2026.

Bewertung

Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Anfängerfreundlichkeit
48/100
Issue-Typ
Bug
Klarheit
Klar beschrieben
Aktivitätsstatus
Aktiv
Tech-Stack
python, pytorch
Bereich
machine-learning, security

Rechercherichtung

Start with the subset and save code at cache_model_1/services/modelowner.py lines 237–241 and the equivalent in cache_model_0/services/modelowner.py; then inspect related partition writers for the same pattern. Add regression coverage for both services using the synthetic sentinel example and verify after serialization and encryption round trips that only assigned inputs and labels are recoverable while sampling remains deterministic. Done means the saved assignment contains no unselected parent data and its size scales with the assignment.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Beschreibung

Approved

Observed against ed175d0deceddfa2e63175ec7752c800b31c1fbf during the November 10 Fashion-MNIST local test investigation. The subsequent full round with three notebooks and a Raspberry Pi used recorded workarounds; its success does not establish that this upstream defect is fixed. Related: #276 and #40.

Problem and evidence

Both reference model-owner services create a torch.utils.data.Subset(test_data, random_indices) and save that object. A Subset holds its parent dataset; torch.save serializes that parent too. Encryption protects the resulting archive in transit but does not remove unassigned examples from it.

An independent synthetic round-trip reproduced a saved assignment with length 500 whose loaded .dataset still had all 10,000 examples. Example 9,999 was accessible despite being outside the chosen assignment. Our completed round used materialized TensorDataset objects to avoid this behavior; its success therefore does not validate the original serializer.

This is more than a large-file problem. A recipient can inspect examples beyond its assigned shard, undermining the intended restricted audit distribution/resampling boundary for non-public data. The ordinary evaluator still iterates the assigned 500 rows. This finding does not establish that evaluation scored all 10,000 examples or that the observed scores were invalid.

Proposed scope

Materialize the selected examples into a standalone dataset before hashing, encryption and upload. Apply the fix to both reference models and inspect related partition writers for the same serialization pattern. Preserve sampling seeds, selected indices, labels and existing evaluation behavior. Do not combine this with a new sampling-policy design.

Acceptance criteria
  • A synthetic parent has recognizable sentinel examples outside the assigned indices.
  • After save/load (and the encryption/decryption round trip), only assigned examples are recoverable; no parent dataset or unselected tensor storage remains embedded.
  • Exact assigned inputs/labels and deterministic indices match the previous sampling logic.
  • File size scales with the assignment, not the parent dataset.
  • Both reference services have regression coverage.
Minimal synthetic reproduction
import io
import torch
from torch.utils.data import TensorDataset, Subset

parent = TensorDataset(torch.arange(10_000))
assigned = Subset(parent, list(range(500)))
buf = io.BytesIO()
torch.save(assigned, buf)
buf.seek(0)
# Only load this locally generated synthetic object.
loaded = torch.load(buf, weights_only=False)
print(len(loaded), len(loaded.dataset), loaded.dataset[9999][0].item())
# 500 10000 9999
Full-round test report

See Discussion #331 — Fashion-MNIST with three notebooks and a Raspberry Pi for the setup diagram, completed protocol roles, results and experiment adaptations.

Vorherrschende Sprache
Python
Sterne
4
Forks
7
Ø Merge
5 T. 15 Std.
Gemergte PRs (30 T.)
43

Entwicklungsumgebung

Dieses Projekt bietet weder Dev-Container noch Dockerfile noch Beitragsleitfaden – die Einrichtung liegt bei Ihnen. Beginnen Sie mit der README; die allgemeinen Schritte stehen in unserem Leitfaden für den ersten Beitrag.

Erste Schritte

  1. Lesen Sie das ganze Issue und danach den Beitragsleitfaden des Projekts.
  2. Schreiben Sie ins Issue, dass Sie es übernehmen — das erspart doppelte Arbeit.
  3. Forken Sie das Repository und arbeiten Sie in einem Branch.
  4. Öffnen Sie einen Pull Request, der die Issue-Nummer nennt.

Mehr aus InfiniteZeroFoundation/DevNet

Alle Issues in InfiniteZeroFoundation/DevNet

Ähnliche Issues

Weitere Issues zu Python

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.