[FEATURE] Replace the stdin/stdout JSON hand-off between Python and Julia OPF with file-based exchange

Aperta
  • Qualcosa non va?
#698 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
48/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Attiva
Stack tecnologico
json, julia, python
Ambito
data

Direzione di ricerca

Start with pm_optimize and Main.jl, where the issue says Python writes input to Julia’s stdin and Julia emits results on stdout; trace how arguments and the existing results directory are handled. Check the subprocess and result conversion paths, then run the relevant optimization tests. Done means input and result are exchanged through files and stdout carries logs only, with results still loaded into the eDisGo object.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

area: opf effort: large enhancement priority: medium

Current state

pm_optimize writes the network as one JSON line to the Julia subprocess's stdin; Main.jl
prints the result JSON to stdout, interleaved with debug logs, printlns and Gurobi
banners. Python identifies the result line by out.rstrip().startswith('{"name"').

Why this is fragile

  1. Result detection depends on unguaranteed JSON key order. JSON.json(::Dict) serializes
    in hash/iteration order — "name" being the first key is coincidence, not contract. If the
    result line ever starts with another key (different Julia/JSON.jl version, changed key set),
    the startswith branch misses it: from_powermodels is never called, no results are
    written to the eDisGo object    , the raw JSON is echoed to stdout, and the run still logs
    "Julia process was successful." — silent loss of results on a successful solve.
  2. Logs and payload share one channel, separated only by prefix heuristics ("Set parameter",
    "Academic").
  3. The input contract ("exactly one JSON line on stdin", read via readline) is implicit and
    unenforced; large inputs are written in full before stdout is drained (OS pipe-buffer
    deadlock risk for very large networks).

Proposal

Exchange input and result as files (paths passed as arguments, e.g. into the existing
results directory or a temp dir): Python writes <run>_input.json, Julia writes
<run>_result.json; stdout remains purely a log channel. This removes the key-order trap, the
log/payload mixing and the pipe-buffer risk in one step.

Interim hardening (if stdout must stay temporarily): wrap the result in an unambiguous sentinel
(e.g. ###RESULT###{…}) instead of keying on {"name".

Lingua principale
Python
Stelle
48
Fork
22
Merge medio
1g 8h
PR unite (30g)
37

Preparare l'ambiente

  • Nessun Dockerfile né file Docker Compose
  • Ha un modello di pull request
  • Nessuna guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di openego/eDisGo

Tutte le issue di openego/eDisGo

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.