Deleting a document orphans its segment-attachment vectors and destroys attachments other documents still reference

Start with api/tasks/clean_document_task.py and the batch-clean path dispatched from api/services/dataset_service.py:2074, then compare their cleanup flow with delete_segment_from_index_task.py and document_indexing_update_task.py. Run the shared-segment-image case in test_segment_index_cleanup_tasks.py and verify document deletion removes unreferenced attachment vectors, rows, and blobs while preserving attachments still referenced elsewhere.

Self Checks

• I have read the Contributing Guide and Language Policy
• I have searched for existing issues, including closed ones
• I confirm that I am using English to submit this report

Dify version

main at 22aff66cc0

Cloud or Self Hosted

Self Hosted (Source), Cloud

Steps to reproduce

In a multimodal knowledge base, delete a document that has segments with attachments. Both deletion paths mishandle those attachments: clean_document_task (single delete, dispatched from api/events/event_handlers/clean_when_document_deleted.py:13) and batch_clean_document_task (batch delete, dispatched from api/services/dataset_service.py:2074).

1. The attachment vectors are never removed.

Attachment vectors are written under doc_id == UploadFile.id (api/core/rag/datasource/vdb/vector_factory.py:201,209), but the node id list handed to index_processor.clean is built only from DocumentSegment.index_node_id:

# api/tasks/clean_document_task.py:64
index_node_ids = [segment.index_node_id for segment in segments if segment.index_node_id]

so the attachment vectors outlive the document permanently. RetrievalService.get_segment_attachment_infos then finds no UploadFile/binding for them and drops them silently, so they keep consuming top-k slots and quietly degrade recall for every subsequent query against that collection.

2. An attachment another document still references is destroyed.

SegmentAttachmentBinding.attachment_id is a user-supplied UploadFile id copied verbatim in SegmentService.create_segment (api/services/dataset_service.py:3512-3521) and carries no uniqueness constraint, so the same image may legitimately be bound to segments of several documents. Both tasks delete the UploadFile row and its storage blob for every binding the deleted document held, with no check for bindings that survive — and in the single-delete path the UploadFile delete even runs before the binding delete:

# api/tasks/clean_document_task.py:137-145
if attachment_ids:
    session.execute(delete(UploadFile).where(UploadFile.id.in_(attachment_ids)))
if binding_ids:
    session.execute(delete(SegmentAttachmentBinding).where(SegmentAttachmentBinding.id.in_(binding_ids)))

The rest of the codebase already treats this as the contract. delete_segment_from_index_task.py:86-121 and document_indexing_update_task.py:103-150 both delete the bindings, flush(), recompute orphan_attachment_ids from the bindings that remain, clean only those from the vector store, and only then drop the rows and blobs. test_segment_index_cleanup_tasks.py even has a shared-segment-image case pinning it. The two document-deletion tasks are the only paths that skip both steps.

✔️ Expected Behavior

Deleting a document removes the vectors, rows and blobs of the attachments nothing else references, and leaves attachments that another document still binds untouched — the same contract delete_segment_from_index_task already implements.

❌ Actual Behavior

Attachment vectors are orphaned in the collection forever, and an attachment shared with another document has its row and blob deleted out from under that document, so the surviving document's segment renders a broken attachment.