Auto upload: ideas for predictable behaviour, duplicate-free re-scans and recovery
維護者通常 1 天內回覆
還沒有人認領這個 Issue。
評估
研究方向
This issue is a six increment proposal and does not name app files or tests. Start by reading the related issues and pull requests listed under the increments, then identify one increment with maintainers. Done means implementing that increment's stated behavior and verifying it with the relevant tests; the proposal itself needs discussion before the full design is settled.
由索引模型根據 Issue 內容生成。
描述
How to use GitHub
- Please use the 👍 reaction to show that you are interested into the same feature.
- Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
- Subscribe to receive notifications on status change and new comments.
Is your feature request related to a problem? Please describe.
Auto upload with "keep original" works most of the time. When it fails, it's hard to notice and hard to recover from:
- A phone silently stops uploading, and nobody notices for weeks (#14945, #16334, #17212).
- The only way to catch up is "Also upload existing files". That re-uploads every photo on the phone that the app has no record of, including photos already uploaded and later moved or deleted on the server (#16348, #6911).
- Identical files become conflicts, or
IMG (2).jpgcopies with "Rename" (#4255, #8948, #12861). - Re-adding a folder, changing its target, or reinstalling loses the app's record of what it uploaded (#12967, #16802, #11090).
Reliability has improved a lot over the last year: the new auto upload worker, the rescan worker in 35.0.0, the stuck-worker fix coming in 35.0.2, and the work coordinated in #16550. Thank you for that. Hopefully silent stops will become rare. Uploads can still fail for reasons outside the app's control (OEM battery management, permissions, a full server, expired credentials), so it remains important that failures are visible and that recovery is predictable and doesn't re-upload everything. That's the focus of this proposal; it doesn't replace the ongoing bug fixing.
Many issues and PRs touch parts of this. This issue tries to connect them under one model of how I believe auto upload should behave, so each part can be discussed and built separately. It follows the increment format of #285.
My ideas depend on a few new concepts, such as a "window" deciding which files to consider for upload and an improved upload-log. These are explained in more detail below.
Expected behaviour per case
I have tried to compile a list of different "use cases" and how I believe the app should behave in each case.| Case | Behaviour |
|---|---|
| A folder is added | Everything in the window (see below) is uploaded except files in the log or identical on the server |
| A new photo is taken | Uploaded |
| A removed folder is added back | Nothing already uploaded is sent again; the log is per file, not per rule |
| An old photo is moved or restored into a folder | Judged by capture date and window, then the normal decision |
| Two rules cover the same file | Uploaded once |
| A photo is edited on the phone after upload | New content, so the collision policy applies |
| New phone or reinstall | The log is restored from its server copy. Without one, moved or deleted files come back, and this is documented |
| Uploads stall completely | A notification "Auto upload stalled since ‹time›: ‹reason›", plus "last successful upload" shown per rule |
| A gap of missed photos | Picked up by the next periodic rescan, since they aren't in the log. "Scan now" makes it immediate |
| One file keeps failing | Marked failed with a reason, doesn't block the queue, Retry / Ignore |
Describe the solution you'd like
Scope: upload only. No two-way sync and no syncing of deletions (that's #17570). Local files are only changed by the existing "original file" action.
Principles:
- A file is uploaded once per account, not once per folder rule. Re-adding or editing a rule never re-uploads a file that was already uploaded.
- Files that are already on the server are always skipped. Skip / Rename / Overwrite / Ask only apply when the existing file is different (same name, different content).
- Every outcome is visible. Each file ends in a named state. Stalls and permanent failures produce a notification, never silence.
- Recovery is a scan, not a reset. Users never need to delete a rule or clear app data to get uploads going again.
Per-file decision:
candidate (in rule folder, captured inside the "upload from" window - see below)
├─ in this account's upload log? ──────────── yes → skip
└─ target path free on server? ────────────── yes → upload, then log it
└─ exists: identical? ─────────────────── yes → skip, log it
different → collision policy (default: Ask)
I suggest changing a few settings slightly, not adding a lot of new ones:
- "Also upload existing files" becomes "Upload files from: All / ‹date›", compared against the capture date. It's never reset implicitly.
- The collision setting is labelled "If a different file with the same name exists".
Recovery: one new menu entry, "Scan now…", with a From date and a mode:
- This idea is based on #17462 but instead of a simple button, I think we should add a date-picker and mode-selector (implemented in 2 separate increments below to not block the current work):
- Re-scan (default): respects the upload log. Safe to run any time.
- Missing: ignores the log and skips files identical on the server. Warns that moved or deleted files will come back.
- All: ignores the log and uploads everything within the window (conflicts resolved according to settings).
The time window:
Today, the only options for selecting files to consider for upload is "All" ("Upload existing files" enabled) or "From the config was saved" (The timestamp the app uses when "Upload existing files" are disabled.
My idea is to introduce a simple way to set a time-window to select which files to consider for uploads.
If uploads have failed for a period of time, one might want to resume uploads from a known date when the you know the upload last worked without worrying that hundreds of old files are re-uploaded. Or you can manually stop uploads for a period, and then re-enable them later, and decide if you want to upload all or only parts of the backlog.
The improved upload log:
My idea is an upload log that is computed on the device and keyed by content hash, so it works with any server. Server-side checksums (nextcloud/server#11138, nextcloud/server#56057) would improve the identity check but are not a prerequisite. The server already accepts OC-Checksum (nextcloud/server#51729).
Increments
I have split this up in 6 increments. Increments 1, 2 and 3 are independent of each other and work with what the app has today. Increments 4–6 build on each other.
- 1. Visibility: "last successful upload" per rule, a stall notification, and failed files that don't block the queue. No server changes needed. Relates to #8154 ("Last auto upload ran at xx:yy"), #14233, #16639, #16207, #260. The battery-saver banner (#15717) is the pattern to reuse.
Depends on: nothing. - 2. Always skip identical files, under every collision policy, and send
OC-Checksumon upload. Relates to #8948, #4255, #10194, #12861, and the earlier attempt #16231.
Depends on: nothing. Uses checksums where the server has them, and today's metadata comparison otherwise. - 3. "Scan now…" with a From date, Re-scan mode only. Relates to #15782, #15051, #15887 and draft #17462.
Depends on: nothing. It respects the app's existing record of uploaded files, and the date limits any re-upload to the chosen window if that record has been lost. - 4. A per-file upload log keyed by content hash, kept across rule changes, and covering files that were skipped. Relates to #12967 (and #17522).
Depends on: nothing, but it's the foundation for 5 and 6. - 5. "Scan now…" modes Missing and All. They're added to the dialog from increment 3.
Depends on: 3 and 4. Both modes are defined as "ignore the log", so they need a reliable log to ignore. - 6. A server copy of the log, restored on a new device or reinstall. Android backup is disabled on purpose (see the maintainer comments in #11090), so a per-device log file in the user's own Nextcloud is used instead. Relates to #11090 and #6911.
Depends on: 4.
Describe alternatives you've considered
- A plain "force upload" button (#15887, #17462): without a date window and a reliable log, it either re-uploads everything or misses the gap.
- Relying on server presence only: it can't know about files that were moved or deleted after upload.
- Android backup or migration of the app database: it's disabled, many users avoid Google backup, and migration tools change
DATE_ADDED, mtimes and paths.
Additional context
I'm a self-hosting user. Our family's phones auto-upload to private folders, and we then move the keepers into a shared folder. That's how we ran into these problems. I'm not an Android developer, so I can't vouch for code changes myself. I can help with testing builds, collecting logs, and refining the expected behaviour above. The code reading behind this was done with AI assistance (Claude); I checked the conclusions against the tracker.
- 主要語言
- Kotlin
- 星號
- 5.7k
- 分支
- 2.1k
- 平均合併
- 1 天 22 小時
- 30 天內合併 PR
- 135
環境準備
在瀏覽器裡用你自己的 GitHub 帳號啟動這個專案的開發容器。
- 沒有 Dockerfile 或 Docker Compose 檔案
- 有 Pull Request 範本
- 閱讀貢獻指南
從這裡開始
- 先讀完整個 Issue,再讀專案的貢獻指南。
- 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
- Fork 儲存庫,在一個分支上完成修改。
- 送出 Pull Request,並在描述裡引用這個 Issue 編號。
nextcloud/android 的其他 Issue
-
0. Needs triage bug
難度 2/5 1-3 小時 新手友好度 64/100
維護者通常 1 天內回覆
-
難度 1/5 1 小時以內 新手友好度 86/100
nextcloud/android#17845 · 2 則留言 ·
維護者通常 1 天內回覆
-
[Bug] Lock owned by another client (e.g. Text app) cannot be unlocked from the app可能已有人在做 @falkloewert 於 28 天前認領。 未關閉
難度 2/5 1-3 小時 新手友好度 72/100
nextcloud/android#17837 · 2 個 reaction ·
維護者通常 1 天內回覆
-
0. Needs triage
難度 2/5 1-3 小時 新手友好度 84/100
nextcloud/android#17533 · 1 則留言 ·
維護者通常 1 天內回覆
-
RC 34.1.0-01: Duplicated wording in creation menu — "New" prefix + creator name produces "New New document" ("Neu Neues Dokument")可能已有人在做 @ChristianTannheimer 於 118 天前認領。 未關閉0. Needs triage bug
難度 2/5 1-3 小時 新手友好度 85/100
維護者通常 1 天內回覆
查看 nextcloud/android 的全部 Issue
相似的 Issue
-
bug needs-triage
難度 2/5 1-3 小時 新手友好度 84/100
neutraltone/settee#453 ·
維護者通常 1 天內回覆
-
Failures in nested test classes (Outer$Inner) are not annotated on their source file可能已有人在做 @NopeCharon 今天認領。 未關閉
難度 2/5 1-3 小時 新手友好度 72/100
mikepenz/action-junit-report#1661 ·
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 76/100
Kotlin/kotlin-lsp#309 ·
-
area:data-quality importer size:S
難度 2/5 1-3 小時 新手友好度 72/100
enorm-labs/event-junkie#5136 ·
維護者通常 1 天內回覆
-
難度 2/5 1-3 小時 新手友好度 70/100
維護者通常 1 天內回覆