The bundled uorf-evidence catalogue duplicates the uORF records already in orf-evidence

Aperta
#201 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
45/100
Tipo di issue
Refactoring
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
python
Ambito
data

Direzione di ricerca

Start with orfdata/catalog.py to trace _parse_catalog, _load_catalog, and load_uorf_catalog, then inspect orfdata/build.py and pyproject.toml for the default build output and packaged data. Check how tests and downstream users rely on the legacy catalogue before changing its handling. Done means the bundled uORF data is no longer duplicated, the default build produces the full catalogue, and legacy file support or its removal is addressed.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Problem

orfdata ships two human GRCh38 catalogues, built by the same build.py from the same pinned sources:

  • data/orf-evidence: all 7,264 ORFs;
  • data/uorf-evidence: 3,771 uORF and uoORF models in the legacy isovar.uorf_evidence.v1 schema.

Every record in the second catalogue is also in the first. Drop the four fields that the legacy loader fills in with fixed values, and the 3,771 uORF/uoORF records in orf-evidence/catalog.json.gz are identical to the 3,771 records in uorf-evidence/catalog.json.gz. As a result:

  • the wheel carries 672 KB of duplicate data;
  • the catalogue parser keeps a legacy-schema branch to read it;
  • python -m orfdata.build without --all-orfs still writes the legacy catalogue by default.

Evidence

Read at d74c9c2.

The records match. I decompressed both files and compared the records with jq -S, after applying del(.source, .species, .taxon_id, .translation_table) to the orf-evidence side. Across the 3,083 uORF and 688 uoORF records there are no differences.

  • Every uORF/uoORF record in orf-evidence has source="gencode_phase1", species="homo_sapiens", taxon_id=9606 and translation_table=1. These are the values that _record(row, legacy=True) fills in (orfdata/catalog.py:971-972).
  • The metadata differs only in dataset_version (2026-10-08.2 against 2026-10-11.1), model_scope, and the legacy file's sources list and manifest schema.

Code kept for the second copy:

  • orfdata/catalog.py:951-962: _parse_catalog accepts both schemas and handles the legacy one separately.
  • orfdata/catalog.py:1009-1035: UORFRecord, UORFCatalog, UORFPeptide = PeptideEvidence and load_uorf_catalog. load_uorf_catalog loads resource="uorf-evidence" and then filters to uORF/uoORF records anyway (:1033-1034).
  • orfdata/build.py:563,574,595,599: the schema, loader and manifest all depend on all_orfs. :617 sets the default --output to orfdata/data/uorf-evidence.

Packaging and docs:

  • pyproject.toml:61 packages both directories. orf-evidence/catalog.json.gz is 1,304,648 bytes and uorf-evidence/catalog.json.gz is 671,566 bytes.
  • docs/reference.md:327-328: "load_uorf_catalog() preserves Isovar's upstream-only snapshot ... New code should use the broader load_catalog() API."

Users of the legacy API:

  • isovar, read at 064cd02: isovar/uorf_catalog.py, isovar/cli/isovar_uorf_evidence.py and isovar/uorf_annotation.py. isovar's tests/test_uorf_catalog.py:181-190 reads data/uorf-evidence/manifest.json and catalog.json.gz directly.
  • varcode, read at 96dc77b: varcode/uorf.py:13-20.

Suggested fix

Keep load_uorf_catalog() and UORFCatalog as the uORF view, but build them from the one bundled catalogue. Load orf-evidence (as _load_catalog(..., resource="orf-evidence") does now) and keep the uORF/uoORF records:

def load_uorf_catalog(path=None, *, expected_sha256=None):
    """Upstream ORFs (uORF, uoORF) from the bundled catalogue, or from a local file."""
    catalog = _load_catalog(path, expected_sha256=expected_sha256)
    return UORFCatalog(
        [r for r in catalog.records if r.biotype in ("uORF", "uoORF")], catalog.metadata)

Then:

  1. Delete orfdata/data/uorf-evidence/ and remove it from package-data.
  2. Make build.py build the full catalogue into orfdata/data/orf-evidence by default.
  3. Keep the isovar.uorf_evidence.v1 reader only for legacy files that users supply, or drop it in a major release.
  4. Ask isovar to point tests/test_uorf_catalog.py at orf-evidence. Its record counts come from the records, which are identical.

https://claude.ai/code/session_01JHJhwj29oDqoeu7TYd34Pi

Lingua principale
Python
Stelle
1
Fork
1
Merge medio
10h 11m
PR unite (30g)
51

Preparare l'ambiente

Questo progetto non fornisce container di sviluppo, Dockerfile né guida per i contributori, quindi l'ambiente è a tuo carico: parti dal suo README e consulta la nostra guida al primo contributo per i passaggi generali.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di openvax/orfdata

Tutte le issue di openvax/orfdata

Issue simili

Altre issue su Python

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.