Auto upload: ideas for predictable behaviour, duplicate-free re-scans and recovery
Maintainer thường phản hồi trong vòng 1 ngày
Chưa có ai nhận issue này.
Đánh giá
- Độ khó
- 5/5
- Thời gian dự kiến
- Hơn một tuần
- Mức phù hợp với người mới
- 25/100
Hướng nghiên cứu
This issue is a six increment proposal and does not name app files or tests. Start by reading the related issues and pull requests listed under the increments, then identify one increment with maintainers. Done means implementing that increment's stated behavior and verifying it with the relevant tests; the proposal itself needs discussion before the full design is settled.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Mô tả
How to use GitHub
- Please use the 👍 reaction to show that you are interested into the same feature.
- Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
- Subscribe to receive notifications on status change and new comments.
Is your feature request related to a problem? Please describe.
Auto upload with "keep original" works most of the time. When it fails, it's hard to notice and hard to recover from:
- A phone silently stops uploading, and nobody notices for weeks (#14945, #16334, #17212).
- The only way to catch up is "Also upload existing files". That re-uploads every photo on the phone that the app has no record of, including photos already uploaded and later moved or deleted on the server (#16348, #6911).
- Identical files become conflicts, or
IMG (2).jpgcopies with "Rename" (#4255, #8948, #12861). - Re-adding a folder, changing its target, or reinstalling loses the app's record of what it uploaded (#12967, #16802, #11090).
Reliability has improved a lot over the last year: the new auto upload worker, the rescan worker in 35.0.0, the stuck-worker fix coming in 35.0.2, and the work coordinated in #16550. Thank you for that. Hopefully silent stops will become rare. Uploads can still fail for reasons outside the app's control (OEM battery management, permissions, a full server, expired credentials), so it remains important that failures are visible and that recovery is predictable and doesn't re-upload everything. That's the focus of this proposal; it doesn't replace the ongoing bug fixing.
Many issues and PRs touch parts of this. This issue tries to connect them under one model of how I believe auto upload should behave, so each part can be discussed and built separately. It follows the increment format of #285.
My ideas depend on a few new concepts, such as a "window" deciding which files to consider for upload and an improved upload-log. These are explained in more detail below.
Expected behaviour per case
I have tried to compile a list of different "use cases" and how I believe the app should behave in each case.| Case | Behaviour |
|---|---|
| A folder is added | Everything in the window (see below) is uploaded except files in the log or identical on the server |
| A new photo is taken | Uploaded |
| A removed folder is added back | Nothing already uploaded is sent again; the log is per file, not per rule |
| An old photo is moved or restored into a folder | Judged by capture date and window, then the normal decision |
| Two rules cover the same file | Uploaded once |
| A photo is edited on the phone after upload | New content, so the collision policy applies |
| New phone or reinstall | The log is restored from its server copy. Without one, moved or deleted files come back, and this is documented |
| Uploads stall completely | A notification "Auto upload stalled since ‹time›: ‹reason›", plus "last successful upload" shown per rule |
| A gap of missed photos | Picked up by the next periodic rescan, since they aren't in the log. "Scan now" makes it immediate |
| One file keeps failing | Marked failed with a reason, doesn't block the queue, Retry / Ignore |
Describe the solution you'd like
Scope: upload only. No two-way sync and no syncing of deletions (that's #17570). Local files are only changed by the existing "original file" action.
Principles:
- A file is uploaded once per account, not once per folder rule. Re-adding or editing a rule never re-uploads a file that was already uploaded.
- Files that are already on the server are always skipped. Skip / Rename / Overwrite / Ask only apply when the existing file is different (same name, different content).
- Every outcome is visible. Each file ends in a named state. Stalls and permanent failures produce a notification, never silence.
- Recovery is a scan, not a reset. Users never need to delete a rule or clear app data to get uploads going again.
Per-file decision:
candidate (in rule folder, captured inside the "upload from" window - see below)
├─ in this account's upload log? ──────────── yes → skip
└─ target path free on server? ────────────── yes → upload, then log it
└─ exists: identical? ─────────────────── yes → skip, log it
different → collision policy (default: Ask)
I suggest changing a few settings slightly, not adding a lot of new ones:
- "Also upload existing files" becomes "Upload files from: All / ‹date›", compared against the capture date. It's never reset implicitly.
- The collision setting is labelled "If a different file with the same name exists".
Recovery: one new menu entry, "Scan now…", with a From date and a mode:
- This idea is based on #17462 but instead of a simple button, I think we should add a date-picker and mode-selector (implemented in 2 separate increments below to not block the current work):
- Re-scan (default): respects the upload log. Safe to run any time.
- Missing: ignores the log and skips files identical on the server. Warns that moved or deleted files will come back.
- All: ignores the log and uploads everything within the window (conflicts resolved according to settings).
The time window:
Today, the only options for selecting files to consider for upload is "All" ("Upload existing files" enabled) or "From the config was saved" (The timestamp the app uses when "Upload existing files" are disabled.
My idea is to introduce a simple way to set a time-window to select which files to consider for uploads.
If uploads have failed for a period of time, one might want to resume uploads from a known date when the you know the upload last worked without worrying that hundreds of old files are re-uploaded. Or you can manually stop uploads for a period, and then re-enable them later, and decide if you want to upload all or only parts of the backlog.
The improved upload log:
My idea is an upload log that is computed on the device and keyed by content hash, so it works with any server. Server-side checksums (nextcloud/server#11138, nextcloud/server#56057) would improve the identity check but are not a prerequisite. The server already accepts OC-Checksum (nextcloud/server#51729).
Increments
I have split this up in 6 increments. Increments 1, 2 and 3 are independent of each other and work with what the app has today. Increments 4–6 build on each other.
- 1. Visibility: "last successful upload" per rule, a stall notification, and failed files that don't block the queue. No server changes needed. Relates to #8154 ("Last auto upload ran at xx:yy"), #14233, #16639, #16207, #260. The battery-saver banner (#15717) is the pattern to reuse.
Depends on: nothing. - 2. Always skip identical files, under every collision policy, and send
OC-Checksumon upload. Relates to #8948, #4255, #10194, #12861, and the earlier attempt #16231.
Depends on: nothing. Uses checksums where the server has them, and today's metadata comparison otherwise. - 3. "Scan now…" with a From date, Re-scan mode only. Relates to #15782, #15051, #15887 and draft #17462.
Depends on: nothing. It respects the app's existing record of uploaded files, and the date limits any re-upload to the chosen window if that record has been lost. - 4. A per-file upload log keyed by content hash, kept across rule changes, and covering files that were skipped. Relates to #12967 (and #17522).
Depends on: nothing, but it's the foundation for 5 and 6. - 5. "Scan now…" modes Missing and All. They're added to the dialog from increment 3.
Depends on: 3 and 4. Both modes are defined as "ignore the log", so they need a reliable log to ignore. - 6. A server copy of the log, restored on a new device or reinstall. Android backup is disabled on purpose (see the maintainer comments in #11090), so a per-device log file in the user's own Nextcloud is used instead. Relates to #11090 and #6911.
Depends on: 4.
Describe alternatives you've considered
- A plain "force upload" button (#15887, #17462): without a date window and a reliable log, it either re-uploads everything or misses the gap.
- Relying on server presence only: it can't know about files that were moved or deleted after upload.
- Android backup or migration of the app database: it's disabled, many users avoid Google backup, and migration tools change
DATE_ADDED, mtimes and paths.
Additional context
I'm a self-hosting user. Our family's phones auto-upload to private folders, and we then move the keepers into a shared folder. That's how we ran into these problems. I'm not an Android developer, so I can't vouch for code changes myself. I can help with testing builds, collecting logs, and refining the expected behaviour above. The code reading behind this was done with AI assistance (Claude); I checked the conclusions against the tracker.
- Ngôn ngữ chính
- Kotlin
- Star
- 5.7k
- Fork
- 2.1k
- Merge trung bình
- 1 ngày 22 giờ
- Pull request đã merge (30 ngày)
- 135
Chuẩn bị môi trường
Khởi chạy dev container của dự án ngay trên trình duyệt, bằng tài khoản GitHub của bạn.
- Không có Dockerfile hay tệp Docker Compose
- Có mẫu pull request
- Đọc hướng dẫn đóng góp
Bắt đầu từ đâu
- Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
- Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
- Fork repository và làm thay đổi trên một nhánh.
- Mở pull request có tham chiếu số hiệu của issue.
Issue khác của nextcloud/android
-
0. Needs triage bug
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 64/100
Maintainer thường phản hồi trong vòng 1 ngày
-
Độ khó 1/5 Dưới một giờ Mức phù hợp với người mới 86/100
nextcloud/android#17845 · 2 bình luận ·
Maintainer thường phản hồi trong vòng 1 ngày
-
[Bug] Lock owned by another client (e.g. Text app) cannot be unlocked from the appCó thể đã có người làm @falkloewert đã nhận 28 ngày trước. Đang mở
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 72/100
nextcloud/android#17837 · 2 reaction ·
Maintainer thường phản hồi trong vòng 1 ngày
-
0. Needs triage
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 84/100
nextcloud/android#17533 · 1 bình luận ·
Maintainer thường phản hồi trong vòng 1 ngày
-
RC 34.1.0-01: Duplicated wording in creation menu — "New" prefix + creator name produces "New New document" ("Neu Neues Dokument")Có thể đã có người làm @ChristianTannheimer đã nhận 118 ngày trước. Đang mở0. Needs triage bug
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 85/100
Maintainer thường phản hồi trong vòng 1 ngày
Tất cả issue của nextcloud/android
Issue tương tự
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 76/100
Kotlin/kotlin-lsp#309 ·
-
Two stored artist rows are not act names: "DJS" from a Renate floor heading and "Stevie Cox -"Đang mởarea:data-quality importer size:S
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 72/100
enorm-labs/event-junkie#5136 ·
Maintainer thường phản hồi trong vòng 1 ngày
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 70/100
Maintainer thường phản hồi trong vòng 1 ngày
-
Type: Possible bug
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 68/100
spacecowboy/Feeder#1318 ·
Maintainer thường phản hồi trong vòng 1 ngày
-
个人助手回复内容既无法手动选中复制,也没有一键复制Đang mở
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 70/100
wangzan101/Mika#644 ·
Maintainer thường phản hồi trong vòng 1 ngày