Analysis Export with Provenance

Open
#48 1 comment 0 reactions 1 assignee View on GitHub

@NetZissou is already working on this.

Since Sep 16, 2026.

Assessment

This issue has not been assessed yet.

Description

design enhancement

Motivation

Neither app can export the analysis table today: embed_explore only saves/repartitions images, precalculated has no download at all. Compute results (2D projections, KMeans runs) live only in the user's Streamlit session.

Exporting projection coordinates as a snapshot is also the only reliable way to reproduce a plot: cuML t-SNE (default solver) is not deterministic even with a seed (see #46 / README reproducibility section), so a "re-run with seed" recipe cannot reproduce a GPU t-SNE view but a coordinate table always can.

Scope

One export action, shared by both apps (shared/components), producing two files:

1. Table (CSV) flat, one row per record
group columns notes
key uuid (precalculated) / file_path (embed_explore) join key back to source data
projections <tag>_x, <tag>_y per projection run this session tag = method + seed, e.g. tsne_s614, umap_s614, pca
clustering every KMeans (k=…) column as in df_plot today
metadata original metadata columns toggle "include original metadata", default on

Embedding vectors are never exported.

2. Provenance sidecar (JSON)
  • app, emb-explorer version, library versions (sklearn / umap-learn / cuml), export timestamp
  • source dataset (parquet name or image folder) + record count + data hash
  • per projection tag: method, backend, seed, key params (perplexity / n_neighbors), run timestamp
  • per KMeans column: k, backend, seed

Session data model

  • Retain every projection run as an (x, y) pair under its tag (tsne_s614_x/_y, umap_s614_x/_y, pca_x/_y); the chart shows the current one. Unseeded runs get a run counter (tsne_r2) since each is a legitimately different view.
  • Retained projection pairs are carried across re-projection the same way KMeans (k=…) columns are today (embed_explore currently rebuilds df_plot per projection and copies only KMeans columns).
Dominant language
Python
Stars
8
Forks
0
PR merge metrics
No merged PRs in 30d

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Imageomics/emb-explorer

All issues in Imageomics/emb-explorer

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.