Fixes and improvements from running 2.4.1 on 4 PCs with large saves — how would you like them?

Aperta
  • Qualcosa non va?
#21 18 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
8/100
Tipo di issue
Funzionalità
Chiarezza
Da chiarire
Stato di attività
Attiva
Stack tecnologico
github-actions, go, sqlite
Ambito
backend, data

Direzione di ricerca

This is a coordination thread for a large body of work in progress, not a task to start from. It names many open PRs (#49, #51, #52, #56, #68, #69, #70, #72), and the maintainer is waiting on design choices, including #71's push loop. Newcomers should not pick it up; instead look for a separate issue with one named failing test, such as TestSoak_ManyAlternatingRoundsNeverConflict.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Status (updated 2026-10-26)

Everything goes into dev-public. Each PR's description has the details; this is the map.

How these were found. Every fix here started from a real failure, mostly a red CI run, and was reproduced before it was fixed: each comes with a test that fails without it. #61–#65 came out of a long run of the e2e tests on Windows runners with the race detector, about 155,000 test runs over 11 rounds. For example, "daemon never became ready" appeared 10 times in about 28,000 runs before #61 and not once in about 52,000 after. With all fixes together, one round ran about 33,000 tests without a failure. Where a cause is inferred rather than reproduced, the PR says so.

Merged — thank you: #22, #23, #24, #25, #28, #49, #50, #52, #53, #54, #55, #56, #57, #58, #61, #62, #63, #64, #65, #68, #69, #70, #72, #73, #74, #78, #80, #84, #90, #98, #99, #100, #106, #107, #113, #115 and #116.

Open, in the order I'd suggest:

PR Why
#114 A save folder that can't be written is said once, with what to do, and retried after 30 minutes. Updated as you asked: a file the game holds open isn't mistaken for it.
#118 Device settings found by what the files hold, as suggestions that keep syncing until confirmed. Updated as you asked (IF NOT EXISTS), and the snapshot history is now compared by the database. Migration 0042.
#117 A game being played is held back from syncing until its session ends. Updated as you asked: one idle rule (no processor time for five minutes), and a session gone all but idle for an hour without a save ends.
#125 A snapshot taken in the millisecond a session began is named for it (TestSessionNamesTheSnapshotAlreadyTaken failed 11 in 40 on Windows).
#126 "Track game" lists the games running now first.
#51 A false conflict when a device edits again before the other has recorded what it took.
#26 Sync speed: batched small files, an "unchanged" manifest shortcut, presence probed on its own.
#27 Closest devices first, by measured link speed. Migration 0039 (now IF NOT EXISTS). Needs #26.

Drafts that build on the above:

PR Why Needs
#127 A game tracked without an App ID is offered its game database link, set only when confirmed. #118
#128 Sessions for protected games and games installed anywhere; a safety net for games whose program isn't seen. #117, #118

CI: red runs (as of 2026-10-17), and why

Run Where Failing test Cause
#375, #379, #384 dev-public TestReportedWAN_AlternatingEditsMustNotConflict #71's push loop; fixed by #78 (merged).
#385 #77 TestJourney_DeviceSpecificConfigNeverTravels (step 2) Found at last, from the logs #57 prints: B's "sync-complete" report to A timed out (it has a 2 s limit and isn't retried), so A's base stayed a round behind and B's next save read as "both sides modified". Same failure as #314, #328, #334, #361. Fixed by #80 (merged).
#381 dev-public TestCrossingSyncs_AFirstPullIsNotTakenForADifferentSave The same reports timing out, in both directions, over a minute: the receiving daemon seems to hold them while it applies a sync. Addressed by #80 (merged).
#394 #80 TestBranchFollow_FetchesTheOtherDevicesSaveNotDeletesIt A's once-a-minute sync switched it back to the old branch before B could follow: #85.
#386 #52 TestDeletion_SurvivesALostReportWhenServed Not reproduced: 0 failures in 648 runs with the race detector, under load, on current dev-public. Probably the same timing, fixed by #80.

Waiting on you

  • Reviews, in the order above.
  • Your view on the question in the latest comment: whether to remove automatic snapshots that are only steps on the way to a newer one.

Waiting on me

  • Nothing open.

Postponed, as you asked: save versions: design issue #29 and drafts #30, #31 and #36. Draft #34 is closed (replaced by #117 and #128). Draft #32 closes once #114 is in, since its other parts are in dev-public (#116 and the launcher files). The gap tests in an earlier comment suggest versions are no longer needed for safety; I'll go by what you decide.

Test build: https://github.com/ChakraFusion/OpenSave/releases/tag/v2.4.1-fixes.9 (unofficial pre-release on my fork).

Background: the original post (2026-10-04)

Hi! I've been running OpenSave across four Windows PCs (LAN plus VPN), with some large saves. The largest is Project Zomboid at ~240k files. Along the way I found several bugs, a few of which delete or lose save data, and fixed them in my fork:
https://github.com/ChakraFusion/OpenSave/tree/fixes-2.4.1

I've split it into focused PRs, as the README asks. Every branch passes CI on its own. The ones marked standalone apply to v2.4.1 directly. The rest are stacked: each builds on the one before, and I'll rebase them as earlier ones land. The larger features are linked to issues so we can discuss the approach first, but the code is ready, so you can read it alongside.

Bug fixes

  1. "Keep mine" can delete files on the device whose version was kept: a 300-file save becomes 100 files on v2.4.1. Adds a guard against syncs that delete most of a save. (standalone)
  2. Changes OpenSave writes itself are taken for the game saving: snapshot floods during a sync, and syncs echoing back. A save the game writes right after a pull now still counts as the game's. (standalone)
  3. A save folder on a different drive letter on another PC (D: vs E:) is found, and a drive this PC doesn't have no longer raises a warning. (standalone)

Performance
4. Manifests and snapshots on large saves: rescanning 240k files 13–19 s → 1.3 s; a follow-up snapshot 80 s → 9 s. (standalone)
5. Batched small-file pulls, an "unchanged" manifest shortcut and gzip: 5,000 files 42 s → 12 s. Presence is probed every 10 s. (stacked on 1)
6. Network-aware sync: link speeds are measured, the closest devices are synced first, and VPN block fetches no longer fail on a fixed 30 s limit. (stacked on 5)
7. Smaller memory footprint (GOGC 50, 1 GiB soft limit). (standalone, optional)

Larger features (stacked, in this order)
8. Save versions: syncs decide by version vectors instead of guessing from files. This is the foundation for most of what follows. → design issue #29
9. Snapshot integrity: never incomplete; identified by content, so every PC agrees and keeps each snapshot once; snapshots fully contained in newer ones are removed.
10. Launcher files (Steam's remotecache.vdf) aren't saves; an unwritable folder is reported once and retried after 30 min; the snapshot after a pull is faster.
11. Device settings: per-PC graphics/config files are kept out of sync, from Ludusavi's config entries and content detection, with per-file control.
12. Play sessions: sync is held while a game is played anywhere; games are detected wherever installed, protected games included; an activity safety net covers unrecognised games.
13. "Track game" from the game database; one folder, one game, with an offer to link entries; folders holding only Steam's files are ignored.
14. Putting back an emptied save takes the other device's save and version; emptying isn't a version until confirmed.
15. Database: WAL, batched deletion-record clears (put-back went from ~1 hour in SQLite to seconds), and free space handed back.
16. Restore writes as many files at once as the drive does best (HDD, SATA or NVMe), with progress and time left.

Coordination

  • Migrations: mine continue after 0037_game_root_mapped, numbered 0038–0043 in stack order. I'll renumber them if anything lands in between.
  • i18n: I'll follow #18 / #20. New UI text can go through the i18n setup once the components I touch are extracted, or I can do that extraction myself.
  • Flaky tests on your CI: two of your own tests failed intermittently for me on plain v2.4.1, TestJourney_DeviceSpecificConfigNeverTravels (CI) and TestSettings_AutoDeleteOldSnapshots_SweepsWhenSwitchedOn (locally under load). Happy to look into them if useful.
  • Extra save locations: TestMultiRootScenario_ALateReportDoesNotHideADivergence ("already in sync") failed twice on Windows runners for me. Without a merge base, an extra location falls back to comparing modification times against the peer-wide last-sync time, which any late report moves. I think save versions for extra locations would be the real fix, and I'm happy to do that as a follow-up once versions land.

Does this order and size work for you? Is there anything you'd rather not have, or would like done differently?

Where everything is

  • Bug fixes: #22 (Keep mine), #23 (own writes), #24 (other drive letter)
  • Performance: #25 (large saves), #26 (sync speed, on #22), #27 (network-aware, on #26), #28 (memory, optional)
  • Save versions: design issue #29, draft PR #30
  • Stacked on #30, in order: #31 snapshot integrity → #32 launcher files / unwritable folders → #33 device settings → #34 play sessions → #35 Track game → #36 put-back → #37 database → #38 parallel restore
Lingua principale
Go
Stelle
225
Fork
9
Merge medio
1g 21h
PR unite (30g)
72

Preparare l'ambiente

  • Include un Dockerfile o un file Docker Compose
  • Nessun modello di pull request
  • Nessuna guida per i contributori

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di Liquid-co/OpenSave

Tutte le issue di Liquid-co/OpenSave

Issue simili

Altre issue su Go

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.