Optional ingest-time gating for untrusted document sources

Ouverte
#63 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub

Personne n'a encore pris cette issue.

Évaluation

Difficulté
5/5
Temps estimé
Plus d'une semaine
Accessibilité débutants
30/100
Type d'issue
Fonctionnalité
Clarté
À clarifier
Activité
Active
Stack technique
python
Domaine
database, security

Piste de recherche

No implementation is selected yet. Read _ingest_location, _resolve_local, _locations_from_files_table, and add_docs to understand the current trust behavior, then review the DID-matlab companion issue and bridge sync requirements. Done would require an agreed optional gating design, matching Python and MATLAB changes, and tests for trusted and attacker-authored sources.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Description

Context

Follow-up to #58 and #60. After #60 (PR #62), _ingest_location refuses only unsafe uid values via _is_safe_uid — the ingest source location is trusted verbatim (_resolve_local, no containment check), on the reasoning that the caller who says "add this file to my DB" chose the source. This unblocks legitimate ingest workflows (NDI-python stages sources under /tmp/ndi-vhsb-*/ and adds them to a DB whose .ndi lives elsewhere) and keeps the destination fully constrained under <FileDir>/<uid>.

The read-side _is_safe_local_location filter (_locations_from_files_table) still defends orig_location, so a crafted stored location cannot steer open_doc to read outside db_dir through the orig_location branch.

The gap

There is one code path that ingestion isn't gated against, and it only matters when the document JSON is attacker-controlled (a cloud pull, not a locally authored document):

  • A document with location='../../etc/passwd' (or any traversal / absolute-outside path pointing at a readable file) now ingests successfully on POSIX. shutil.copyfile resolves the traversal, reads the target file, and writes its contents into <FileDir>/<uid>.
  • On a later open_doc, the read-side filter refuses orig_location, but by then the cache candidate at <FileDir>/<uid> — built from the safe uid — already exists and wins the earlier loop; the smuggled contents are served back.

There is no such issue when the source of the document is trusted (the local NDI-python call, say). It is specifically the cloud-pull path where the JSON is not the caller's.

What we're not doing

For now, no change. The trade-off from #60 stands: reintroducing a source-side containment check breaks the legitimate workflow, and this residual surface only matters for the "attacker-authored document" case — one downstream packages can also mitigate by not blindly ingesting cloud-pulled documents.

What an optional gate might look like

If we do decide to close this later:

  • Narrow the guard to relative locations with a traversal segment (a relative path whose _resolve_local result escapes db_dir) — this refuses the ../../etc/passwd shape without refusing an absolute path that legitimately lives outside db_dir. It's stricter than "no traversal", weaker than "must be inside db_dir".
  • Or make the guard opt-in via add_docs(..., trust_sources=True|False), defaulting to True (today's behavior) and set to False by callers who pull documents from a cloud store. Downstream (NDI cloud pull) then sets trust_sources=False and gets the strict containment check.

Either would need matching changes on the MATLAB side and a bridge sync note. See the DID-matlab companion issue for the parity version.

Langage dominant
Python
Étoiles
0
Forks
1
Merge moyen
2 h 15 min
PR mergées (30 j)
40

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Par où commencer

  1. Lisez l'issue en entier, puis le guide de contribution du projet.
  2. Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
  3. Forkez le dépôt et travaillez sur une branche.
  4. Ouvrez une pull request qui référence le numéro de l'issue.

Issues similaires

Plus d'issues Python

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.