Store each face detection once per file: in a group folder, every detection (incl. vector) is currently copied to every user who can see it causing the table to grow with the number of users
評価
- 難易度
- 5/5
- 見積もり時間
- 1週間以上
- 初心者へのやさしさ
- 35/100
調査の方向性
Start with FaceDetection, FaceDetectionMapper, TaskResultListener, and FsActionService, which the issue identifies as storing and copying detections per user. Trace how detections, vectors, and per-user cluster assignments are read and written, then inspect the existing database migrations and tests for those paths. Done means detection data is stored once per file while each user's clusters and names remain private and existing access and clustering behavior still works.
索引モデルが issue の本文から書いたものです。
説明
Describe the feature you'd like to request
Store the box and embedding of each face detection once per file, and keep only the per-user part (the cluster assignment) per user. Clusters and their names would stay private to each user, as they are now.
I'm running Recognize 13.1.0 with recognize_backend 1.0.6 (CUDA), Nextcloud 35.0.2 AIO (in an LXC Container on Proxmox 9.2.21). All my photos are in one group folder that 39 users can see. Detection runs on an Nvidia T1000 8GB.
I deployed Recognize on Sunday evening & let it churn through my library. Once the first face crawl had finished the ~21k faces in ~7k photos resulted in ~816k rows in oc_recognize_face_detections (one per face per user). The table is now 6.9 GB of a 9.5 GB database. Before Recognize, the AIO database dump was 0.87 GB; it's now 9.9 GB 🥲.
AIO writes that dump next to the database, so the first I saw of it was the nightly backup failing on a full disk. I asked Claude Code to investigate. Here's its output
TaskResultListenerinserts a completeFaceDetection, vector included, for every user returned bygetUsersWithFileAccess()(TaskResultListener.php#L228-L238). That list comes from the mount cache, so it's everyone who has the folder mounted.- When a file's access changes or it moves into a shared folder,
FsActionServicecopies each detection, vector included, to every new user withcopyDetectionsForFileFromUserToUser()(FsActionService.php#L188, #L353, FaceDetectionMapper.php#L151-L164). - The vector is a
jsonentity field (FaceDetection.php#L60), so each 512-value embedding is stored as text, about 7.5 KB per row.
Here that works out to roughly 290 KB for every face detected (39 × 7.5 KB), and the cost rises in step with the number of users who can see the folder. Inference already runs once per file since #939 fixed #916; only the storage repeats.
#948 and discussion #1063 ask for people to be shared between users. This request doesn't: each user would still get their own clusters and names, and only the detection data would be stored once. A comment on #948 already points out that detections, not only clusters, are per user (#948 comment). #1148, about users who mount a folder later getting no detections, was fixed in NC 32.
Describe the solution you'd like
Split the table in two:
recognize_face_detections: one row per face per file, withfile_id,x,y,width,heightandface_vector.- A per-user table (name up to you) with
detection_id,user_id,cluster_idandthreshold.
Clustering would still run per user, over the detections in files that user can see, and write cluster_id to that user's row. A user gaining access to a folder would get small rows instead of copies of every vector. With 39 users, the vector data here would drop from 6.9 GB to under 200 MB.
Storing the vector as binary float32 (2 KB for 512 values) would shrink it further, but that's a separate change, and #1315 already touched on it from another angle, so I'm only mentioning it.
Describe alternatives you've considered
- A
.noimagemarker on the group folder. That switches face recognition off for exactly the photos I want it for. - Fewer users on the folder. They need access to the photos, so that's not an option.
- Recognize's own settings. For faces, 13.1.0 has
faces.enabledandfaces.batchSize, and nothing per user or per folder. - Compression on the PostgreSQL side (
ALTER COLUMN face_vector SET COMPRESSION lz4). It might shrink the table on disk, but the dump is plain SQL and stays the same size, and the backup is where it broke. - More disk. That's what I did, but every new face still costs another ~290 KB.
🤖 Generated with Claude Code
- 主要言語
- PHP
- スター
- 700
- フォーク
- 70
- 平均マージ
- 4日 15時間
- マージ済み PR(30日)
- 5
環境構築
- Dockerfile・Docker Compose ファイルなし
- プルリクエストのテンプレートなし
- コントリビューションガイドを読む
はじめの一歩
- issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
- 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
- リポジトリをフォークし、ブランチを切って変更します。
- issue 番号を参照したプルリクエストを送ります。
nextcloud/recognize のほかの issue
-
bug
難易度 4/5 3〜5日 初心者へのやさしさ 45/100
-
bug
難易度 5/5 1週間以上 初心者へのやさしさ 15/100
-
enhancement
難易度 3/5 1〜2日 初心者へのやさしさ 65/100
-
enhancement
難易度 4/5 3〜5日 初心者へのやさしさ 45/100
-
Recognize maintenance routine unnecessarily redownloads Tensorflow CPU and GPU models every time再び着手できるかも @marcelklehr が 213 日前に担当しましたが、オープン中のプルリクエストはありません。 オープンbug
nextcloud/recognize の issue をすべて見る
似ている issue
-
bug webbackend
難易度 2/5 1〜3時間 初心者へのやさしさ 74/100
DOMjudge/domjudge#3886 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
bug
難易度 2/5 1〜3時間 初心者へのやさしさ 65/100
opensourcepos/opensourcepos#4794 ·
メンテナーはふだん 1 日以内に返信
-
repo:cms
難易度 2/5 1〜3時間 初心者へのやさしさ 68/100
craftcms/cms#20235 · コメント 1 件 ·
メンテナーはふだん 1 日以内に返信
-
難易度 2/5 1〜3時間 初心者へのやさしさ 65/100
メンテナーはふだん 1 日以内に返信
-
REST API: decimal ids like 1.5 give 500 on folder and report endpoints対応中かも @console-log-life が今日担当しました。 オープン
難易度 2/5 1〜3時間 初心者へのやさしさ 65/100
メンテナーはふだん 3 日以内に返信