Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

`darnit-reproducibility`: environment-generation sieve (deterministic-first, LLM fallback)

Aperta
  • Qualcosa non va?
#422 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
35/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Tranquilla
Stack tecnologico
docker, python
Ambito
build-system, tooling

Direzione di ricerca

Start by reading the control-sieve pattern and related issue #421, then inspect the repo2docker Python API (repo2docker.Repo2Docker). The issue asks for a design and prototype, not a named file or test: define the deterministic-to-LLM-to-manual pass behavior, evidence and artifact choices, and operator sign-off. Done means the deterministic reference integration and auditable design are documented without silently applying generated artifacts.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

module-reproducibility

Summary

darnit is positioned to help with the reproducibility edge cases --
the places where existing environment-generation tooling (repo2docker,
Cloud Native Buildpacks, devcontainer templates) doesn't reach. This
issue scopes a "sieve" for environment generation that mirrors
darnit's own control-sieve pattern: try deterministic tools first,
escalate to an LLM only where they can't conclude, always with
operator sign-off before applying.

This is not a plan to compete with repo2docker or buildpacks. It's a
plan to route to them first, then fill the gaps.

Prior art (deterministic tier)

  • repo2docker (Jupyter/Binder): detects requirements.txt,
    environment.yml, Pipfile, install.R, apt.txt, postBuild,
    runtime.txt, Dockerfile, default.nix. Produces a Dockerfile.
    Stops at: no env files -> bare python:3.11; pyproject.toml
    without a lockfile; compiled deps beyond apt.txt (CUDA, MKL,
    custom C libs).
  • Cloud Native Buildpacks (pack): ordered buildpack list; first
    match wins. Stops at: needs Procfile / entry point; doesn't fit
    scientific workflow shapes (Nextflow, Snakemake, papermill); emits
    OCI images, not inspectable Dockerfiles.
  • docker init: interactive prompt; not a fit for automation.
  • Devcontainer templates: static templates keyed on
    .devcontainer/devcontainer.json presence + language; requires
    operator to pick a template.
  • Nix flake generators: minimal skeleton, not repo-analysis-driven.

Where LLM tier earns its place

Concrete edge cases the deterministic tools do not handle today:

  • Compiled deps beyond apt.txt (repo mentions PyTorch + CUDA
    version in README prose; no lockfile encodes the CUDA choice).
  • Non-buildpack workflow shapes (snakemake --cores 8,
    nextflow run main.nf, papermill notebook.ipynb -p param 3).
  • System prerequisites in README prose that no deterministic tool
    parses (Install OpenBabel first, then run pip install .).
  • pyproject.toml with editable install (pip install -e .) but no
    lockfile -- repo2docker punts.
  • Multi-stage builds where the researcher's own tool must be compiled
    before the experiment runs (Layer 1 vs Layer 2 per #415).

Proposed shape

Same pass shape as sieve controls:

sieve = [
  ("repo2docker",  detect: any env file in the list above),
  ("buildpacks",   detect: `pack builder inspect` returns a match),
  ("heuristic",    detect: single-language repo with one obvious lockfile),
  ("llm_eval",     always-runs-if-preceding-inconclusive; prompt: file
                    listing, top-level READMEs, detected language
                    signals, evidence of compiled-deps),
  ("manual",       operator prompt with best-effort skeleton to edit),
]

Each pass either produces a Dockerfile + justification, or falls
through. LLM output is a candidate, not a conclusion (Constitution
IV -- Never Guess User Values). Operator confirmation gates
application; the artifact is not written into the repo silently.

Scope

  • Design the sieve above.
  • Prototype the deterministic tier via repo2docker's Python API
    (repo2docker.Repo2Docker) as the reference integration.
  • Design the LLM tier's prompt shape and evidence bundle (leans on
    the LLM-reasoning-capture work from #421 -- prompt and reasoning
    MUST land in a durable artifact next to the Dockerfile).
  • Decide artifact surface: Dockerfile vs devcontainer.json vs both;
    where it lands in the repo; what darnit records in the attestation.

Non-goals for v0

  • Rewriting repo2docker's or buildpacks' detection logic.
  • Nix flake generation (narrow audience; revisit once the sieve
    shape stabilizes).
  • Runtime execution verification (that's the ambitious Option C from
    the design conversation -- separate issue if pursued).
  • Auto-applying LLM-generated Dockerfiles without operator sign-off.

Constitution alignment

  • I. Plugin Separation: the sieve lives inside
    darnit-reproducibility; darnit core doesn't grow a Docker
    dependency.
  • II. Conservative-by-Default: LLM tier produces candidates, not
    conclusions.
  • IV. Never Guess User Values: operator sign-off is a hard
    precondition to writing anything into the repo.
  • V. Sieve Pipeline Integrity: the deterministic->LLM->manual
    ordering is the same pattern the control sieve uses; each pass
    either conclusively produces an artifact or hands off.

Related

  • #414 (signal extraction module) -- the sieve's detect signals are
    the same signals this feeds.
  • #415 (layer + tightness taxonomy) -- the produced environment
    should declare which (layer, tightness) it targets.
  • #416 (heuristic checks) -- an audit that finds "no environment
    declaration" is the trigger; this issue is the remediation half.
  • #421 (LLM reasoning capture) -- prerequisite for LLM tier's output
    being auditable.
  • #418 / #419 / #420 (determinism set) -- companion; the whole point
    of a hermetic env is to make audit + remediation more
    deterministic downstream.
Lingua principale
Python
Stelle
22
Fork
27
Merge medio
1g 20h
PR unite (30g)
39

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di darnitdevorg/darnit

Tutte le issue di darnitdevorg/darnit

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.