TaskProcessing mode never schedules a task on a group folder with Advanced Permissions: the batch is checked as an arbitrary mount user
Chưa có ai nhận issue này.
Đánh giá
- Độ khó
- 4/5
- Thời gian dự kiến
- 3-5 ngày
- Mức phù hợp với người mới
- 45/100
Hướng nghiên cứu
Start with lib/BackgroundJobs/ClassifierJob.php and lib/Classifiers/AbstractTaskProcessingClassifier.php, then trace how scheduleTask() validates file IDs and user access in Nextcloud’s TaskProcessing manager. The report describes both the mount lookup failure and the batch retry behavior; reproduce with group folders using Advanced Permissions. Done means readable files are scheduled and one inaccessible file does not keep the rest of a batch queued.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Mô tả
Which version of recognize are you using?
13.1.0, with recognize_backend 1.0.6 (CUDA)
Enabled Modes
Face recognition, Object recognition
TensorFlow mode
WASM mode
Downstream App
Memories App
Which Nextcloud version do you have installed?
35.0.2 (AIO)
Which Operating system do you have installed?
Debian 12 (LXC container on Proxmox VE); Nextcloud itself runs in the AIO containers
Which database are you running Nextcloud on?
PostgreSQL 18.6 (AIO)
Which Docker container are you using to run Nextcloud? (if applicable)
Nextcloud AIO v14.4.1
How much RAM does your server have?
4 GiB (container limit)
What processor Architecture does your CPU have?
x86_64
Describe the Bug
After switching to TaskProcessing mode with recognize_backend, nothing reached the backend. The queues filled up (about 21,700 files each for faces and imagenet), but no task was ever created, and every cron run logged the same pair of errors for both models. All the photos are in one group folder with Advanced Permissions. I asked Claude Code to trace it, and this is what it found:
ClassifierJob sets up the filesystem for whichever user getMountsForStorageId() returns first (ClassifierJob.php#L70-L75), and AbstractTaskProcessingClassifier::findUserForStorage() makes that same first user the task's user (AbstractTaskProcessingClassifier.php#L91-L100). There's no ORDER BY on that query, so on a group folder the user is whoever the database returns first. Here that is a user from a guest group, who isn't even among the mount users of the file below.
IManager::scheduleTask() then checks every input file. prepareTask() calls validateFileId(), which looks the file up through the root folder (Manager.php#L1896-L1898). Root::getByIdInPath() only searches group folder mounts that are already set up in the process, and each of those is filtered by its own user's ACL (Root.php#L402). So whether a file "exists" depends on which users the cron process happens to have set up. If none of them can read it, the file isn't found and the whole batch of up to 500 files is rejected. The files stay at the head of the queue, so the next run fails on the same batch, forever.
I checked this on one of the failing files, which 13 users have a mount for, setting up one user per fresh process and then calling getFirstNodeById(): it finds the file for the 3 admins and returns NULL for the other 10. Set up one of those 10 and an admin in the same process and it's found again.
The task user hits a second check after that: validateUserAccessToFile() requires the task's user to be among getMountsForFileId() for every file. The first user for the storage wasn't in that list for the file above, so I expect the same batch to fail there once the lookup is fixed.
As a local workaround I changed both call sites to prefer a mount that belongs to an admin and fall back to the first mount otherwise. With that, tasks are scheduled under the admin user, the backend picks them up, and faces and tags start landing. It works here because every admin can read the whole folder.
Possible fixes:
- Pick the task user per batch from users who can actually read every file in it, or split the batch by readable user. Admins happen to work on my instance, but an instance can have a group folder no admin is a member of.
- Don't let one unreadable file block the batch: on a
ValidationExceptionorUnauthorizedException, drop or skip that file and schedule the rest, instead of leaving all 500 queued. - On the server side,
validateFileId()could resolve the file in the task user's folder rather than the root folder. That's a nextcloud/server change, but it would make the result stop depending on process state.
Expected Behavior
Files in a group folder with Advanced Permissions get classified, and a file the chosen user can't read doesn't stop the rest of the batch.
To Reproduce
- Put photos in a group folder with Advanced Permissions, shared with several groups. Deny Read on a subfolder to most of them.
- Install recognize_backend so Recognize runs in TaskProcessing mode, and enable face or object recognition.
- Run a recrawl and let cron run.
- Every run logs the errors below and
oc_taskprocessing_tasksstays empty forapp_id = 'recognize'. Whether it fails depends on which users the cron process has set up, so it fails reliably once most users with a mount are denied the subfolder.
Debug log
Failed to schedule recognize:image:classification task
Temporary problem with imagenet classifier, trying again soon
Failed to schedule recognize:image:facerecognition task
Temporary problem with faces classifier, trying again soon
The previous exception on each:
OCP\TaskProcessing\Exception\ValidationException: Could not find file 10427
lib/private/TaskProcessing/Manager.php:1753
lib/private/TaskProcessing/Manager.php:1086
custom_apps/recognize/lib/Classifiers/AbstractTaskProcessingClassifier.php:74
custom_apps/recognize/lib/BackgroundJobs/ClassifyImagenetJob.php:46
custom_apps/recognize/lib/BackgroundJobs/ClassifierJob.php:79
- Ngôn ngữ chính
- PHP
- Star
- 700
- Fork
- 70
- Merge trung bình
- 4 ngày 15 giờ
- Pull request đã merge (30 ngày)
- 5
Chuẩn bị môi trường
- Không có Dockerfile hay tệp Docker Compose
- Không có mẫu pull request
- Đọc hướng dẫn đóng góp
Bắt đầu từ đâu
- Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
- Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
- Fork repository và làm thay đổi trên một nhánh.
- Mở pull request có tham chiếu số hiệu của issue.
Issue khác của nextcloud/recognize
-
Store each face detection once per file: in a group folder, every detection (incl. vector) is currently copied to every user who can see it causing the table to grow with the number of usersCó thể đã có người làm @marcelklehr đã nhận hôm nay. Đang mởenhancement
Độ khó 5/5 Hơn một tuần Mức phù hợp với người mới 35/100
-
bug
Độ khó 5/5 Hơn một tuần Mức phù hợp với người mới 15/100
-
enhancement
Độ khó 3/5 1-2 ngày Mức phù hợp với người mới 65/100
-
Model download button in UIĐang mởenhancement
Độ khó 4/5 3-5 ngày Mức phù hợp với người mới 45/100
-
Recognize maintenance routine unnecessarily redownloads Tensorflow CPU and GPU models every timeCó thể làm lại được @marcelklehr đã nhận 211 ngày trước và không có pull request nào đang mở. Đang mởbug
nextcloud/recognize#1478 · 3 bình luận · 1 người được giao ·
Tất cả issue của nextcloud/recognize
Issue tương tự
-
[Sync EN] Add more details about the array returned from openssl_x509_parse (php#2666) (#6013)Đang mởsync-en
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 72/100
Maintainer thường phản hồi trong vòng 1 ngày
-
fix(proxy): avoid exposing Reverb HTTP API (/apps) publicly in generated Traefik dynamic configĐang mở
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 78/100
coollabsio/coolify#12390 ·
Maintainer thường phản hồi trong vòng 1 ngày
-
win_build: parallel PostBuildEvent race in example_app_atiopencl and example_app_nvopencl copying openclapp_kernels.clCó thể đã có người làm @qixuancao đã nhận hôm nay. Đang mở
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 73/100
Maintainer thường phản hồi trong vòng 1 ngày
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 65/100
verbb/formie#3072 · 1 bình luận ·
Maintainer thường phản hồi trong vòng 1 ngày
-
Độ khó 2/5 1-3 giờ Mức phù hợp với người mới 78/100
Maintainer thường phản hồi trong vòng 1 ngày