TaskProcessing mode never schedules a task on a group folder with Advanced Permissions: the batch is checked as an arbitrary mount user
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Aptitud para principiantes
- 45/100
Línea de trabajo
Start with lib/BackgroundJobs/ClassifierJob.php and lib/Classifiers/AbstractTaskProcessingClassifier.php, then trace how scheduleTask() validates file IDs and user access in Nextcloud’s TaskProcessing manager. The report describes both the mount lookup failure and the batch retry behavior; reproduce with group folders using Advanced Permissions. Done means readable files are scheduled and one inaccessible file does not keep the rest of a batch queued.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Which version of recognize are you using?
13.1.0, with recognize_backend 1.0.6 (CUDA)
Enabled Modes
Face recognition, Object recognition
TensorFlow mode
WASM mode
Downstream App
Memories App
Which Nextcloud version do you have installed?
35.0.2 (AIO)
Which Operating system do you have installed?
Debian 12 (LXC container on Proxmox VE); Nextcloud itself runs in the AIO containers
Which database are you running Nextcloud on?
PostgreSQL 18.6 (AIO)
Which Docker container are you using to run Nextcloud? (if applicable)
Nextcloud AIO v14.4.1
How much RAM does your server have?
4 GiB (container limit)
What processor Architecture does your CPU have?
x86_64
Describe the Bug
After switching to TaskProcessing mode with recognize_backend, nothing reached the backend. The queues filled up (about 21,700 files each for faces and imagenet), but no task was ever created, and every cron run logged the same pair of errors for both models. All the photos are in one group folder with Advanced Permissions. I asked Claude Code to trace it, and this is what it found:
ClassifierJob sets up the filesystem for whichever user getMountsForStorageId() returns first (ClassifierJob.php#L70-L75), and AbstractTaskProcessingClassifier::findUserForStorage() makes that same first user the task's user (AbstractTaskProcessingClassifier.php#L91-L100). There's no ORDER BY on that query, so on a group folder the user is whoever the database returns first. Here that is a user from a guest group, who isn't even among the mount users of the file below.
IManager::scheduleTask() then checks every input file. prepareTask() calls validateFileId(), which looks the file up through the root folder (Manager.php#L1896-L1898). Root::getByIdInPath() only searches group folder mounts that are already set up in the process, and each of those is filtered by its own user's ACL (Root.php#L402). So whether a file "exists" depends on which users the cron process happens to have set up. If none of them can read it, the file isn't found and the whole batch of up to 500 files is rejected. The files stay at the head of the queue, so the next run fails on the same batch, forever.
I checked this on one of the failing files, which 13 users have a mount for, setting up one user per fresh process and then calling getFirstNodeById(): it finds the file for the 3 admins and returns NULL for the other 10. Set up one of those 10 and an admin in the same process and it's found again.
The task user hits a second check after that: validateUserAccessToFile() requires the task's user to be among getMountsForFileId() for every file. The first user for the storage wasn't in that list for the file above, so I expect the same batch to fail there once the lookup is fixed.
As a local workaround I changed both call sites to prefer a mount that belongs to an admin and fall back to the first mount otherwise. With that, tasks are scheduled under the admin user, the backend picks them up, and faces and tags start landing. It works here because every admin can read the whole folder.
Possible fixes:
- Pick the task user per batch from users who can actually read every file in it, or split the batch by readable user. Admins happen to work on my instance, but an instance can have a group folder no admin is a member of.
- Don't let one unreadable file block the batch: on a
ValidationExceptionorUnauthorizedException, drop or skip that file and schedule the rest, instead of leaving all 500 queued. - On the server side,
validateFileId()could resolve the file in the task user's folder rather than the root folder. That's a nextcloud/server change, but it would make the result stop depending on process state.
Expected Behavior
Files in a group folder with Advanced Permissions get classified, and a file the chosen user can't read doesn't stop the rest of the batch.
To Reproduce
- Put photos in a group folder with Advanced Permissions, shared with several groups. Deny Read on a subfolder to most of them.
- Install recognize_backend so Recognize runs in TaskProcessing mode, and enable face or object recognition.
- Run a recrawl and let cron run.
- Every run logs the errors below and
oc_taskprocessing_tasksstays empty forapp_id = 'recognize'. Whether it fails depends on which users the cron process has set up, so it fails reliably once most users with a mount are denied the subfolder.
Debug log
Failed to schedule recognize:image:classification task
Temporary problem with imagenet classifier, trying again soon
Failed to schedule recognize:image:facerecognition task
Temporary problem with faces classifier, trying again soon
The previous exception on each:
OCP\TaskProcessing\Exception\ValidationException: Could not find file 10427
lib/private/TaskProcessing/Manager.php:1753
lib/private/TaskProcessing/Manager.php:1086
custom_apps/recognize/lib/Classifiers/AbstractTaskProcessingClassifier.php:74
custom_apps/recognize/lib/BackgroundJobs/ClassifyImagenetJob.php:46
custom_apps/recognize/lib/BackgroundJobs/ClassifierJob.php:79
- Lenguaje dominante
- PHP
- Estrellas
- 700
- Forks
- 70
- Merge medio
- 4 d 15 h
- PR fusionados (30 d)
- 5
Preparar el entorno
- Sin Dockerfile ni archivo de Docker Compose
- Sin plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de nextcloud/recognize
-
Store each face detection once per file: in a group folder, every detection (incl. vector) is currently copied to every user who can see it causing the table to grow with the number of usersPosiblemente ocupada @marcelklehr la tomó hace 2 días. Abiertoenhancement
Dificultad 5/5 Más de una semana Aptitud para principiantes 35/100
-
bug
Dificultad 5/5 Más de una semana Aptitud para principiantes 15/100
-
enhancement
Dificultad 3/5 1-2 días Aptitud para principiantes 65/100
-
Model download button in UIAbiertoenhancement
Dificultad 4/5 3-5 días Aptitud para principiantes 45/100
-
Recognize maintenance routine unnecessarily redownloads Tensorflow CPU and GPU models every timeQuizá libre de nuevo @marcelklehr la tomó hace 213 días y no hay ningún pull request abierto. Abiertobug
Todos los issues de nextcloud/recognize
Issues similares
-
bug webbackend
Dificultad 2/5 1-3 horas Aptitud para principiantes 74/100
DOMjudge/domjudge#3886 · 1 comentario ·
Los mantenedores suelen responder en 1 día
-
bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 65/100
opensourcepos/opensourcepos#4794 ·
Los mantenedores suelen responder en 1 día
-
repo:cms
Dificultad 2/5 1-3 horas Aptitud para principiantes 68/100
craftcms/cms#20235 · 1 comentario ·
Los mantenedores suelen responder en 1 día
-
Dificultad 2/5 1-3 horas Aptitud para principiantes 65/100
Los mantenedores suelen responder en 1 día
-
REST API: decimal ids like 1.5 give 500 on folder and report endpointsPosiblemente ocupada @console-log-life la tomó hoy. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 65/100
Los mantenedores suelen responder en 3 días