WebUI forced rescan clears the wrong VAE cache when the active config uses --keys

Open
#3,342 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
72/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python
Domain
backend, databases

Research direction

Start with _get_global_config in routes/dataset_viewer.py and trace how its result reaches DatasetScanService._resolve_vae_cache_dir and clear_vae_cache in services/dataset_scan_service.py. Check how _build_scan_args and CacheJobService._resolve_global_key handle ---prefixed keys, then add a regression test for a ---keyed active config. Done when the configured VAE cache directory is resolved and cleared during a forced rescan.

Written by the indexing model from the issue text.

Description

bug webui

Summary

A WebUI forced rescan clears the wrong VAE cache directory when the active training config uses CLI-style keys (--output_dir, --model_family). Saved configs commonly use those keys; see simpletuner/examples/_shared/default.json. The real cache is left in place, so latents from the old geometry outlive the rescan.

Checked on refactor/provider-transform-cache (931460c57).

Details

  • routes/dataset_viewer.py:77 _get_global_config() returns the active config as stored, with -- keys.
  • DatasetScanService._build_scan_args (services/dataset_scan_service.py:177) strips the prefix, so the scan's own settings are right.
  • DatasetScanService._resolve_vae_cache_dir (dataset_scan_service.py:228-249) reads global_config.get("output_dir") and global_config.get("model_family") by plain name. It is reached through _local_vae_clear_directory → clear_vae_cache after a forced rescan (dataset_scan_service.py:545).

Probe, with cache_dir_vae: "{output_dir}/cache/vae/{model_family}/{id}":

{"output_dir": "/runs/a", "model_family": "flux"}      -> /runs/a/cache/vae/flux/train
{"--output_dir": "/runs/a", "--model_family": "flux"}  -> <cwd>/.simpletuner_output/cache/vae/train

Proposed fix

Strip -- once, where the viewer reads the active config (_get_global_config), and remove the now-redundant per-call stripping:

  • _build_scan_args;
  • the dual lookup in CacheJobService._resolve_global_key, if every caller goes through the stripped config.

Test: a ---keyed active config resolves the VAE cache directory to the configured path, and a forced rescan clears that directory.

Related commit on refactor/fable-5.2-datasets

  • c421b0ca2: read the active config's CLI-style keys when the WebUI scans a dataset. It strips -- in _get_global_config, the same fix, but on a base where _build_scan_args didn't strip either.
Dominant language
Python
Stars
2.9k
Forks
295
Avg merge
3h 28m
Merged PRs (30d)
42

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from bghira/SimpleTuner

All issues in bghira/SimpleTuner

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.