staged_insert1 stores object paths with the class name and an empty store name, so garbage collection can delete its objects

未關閉 適合新手
#1,701 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視

維護者通常 6 天內回覆

還沒有人認領這個 Issue。

評估

難度
2/5
預估耗時
1-3 小時
新手友好度
78/100
Issue 類型
缺陷
描述清晰度
描述清楚
活躍度
活躍
技術堆疊
mysql, postgresql, python

研究方向

閱讀 src/datajoint/staged_insert.py,特別查看第 103–124 行的路徑建構,以及第 214–222 行的 store 中繼資料;將它們與一般的 insert1 路徑比較。Issue 建議使用 self._table.table_name 和 config.resolve_store_name(store_name)。為一般資料表和 part 資料表上的物件屬性新增或執行測試,包含預設 store;當 staged 插入的資料列使用與 insert1 相同的路徑配置和解析後的 store 名稱時,即表示完成。

由索引模型根據 Issue 內容生成。

描述

bug

For an <object@> attribute, insert1 builds the object's path from the table name and records the resolved store name. staged_insert1 does neither (staged_insert.py#L103-L124, #L214-L222):

  • it builds the path from the class name;
  • it records the store as declared, which is "" for the default store (<object@>).

Garbage collection can then delete objects that rows still refer to.

Repro (master, eefe74b4; MySQL and PostgreSQL; stores local, the default, and other)

A Manual table Recording with data : <object@>, and a part table Session.Item with the same attribute. One row in each is inserted with insert1, and one with staged_insert1:

Row Inserted with Stored path Stored store
rec_id=1 insert1 _schema/<db>/recording/rec_id=1/data_… local
rec_id=2 staged_insert1 _schema/<db>/Recording/rec_id=2/data_….dat ""
item_id=1 insert1 _schema/<db>/session__item/sess_id=1/item_id=1/… local
item_id=2 staged_insert1 _schema/<db>/Item/sess_id=1/item_id=2/… ""

Consequences

  • Case-insensitive file stores (macOS, Windows). Recording/ and recording/ are the same folder there. The store listing reports it as recording/, so the collector does not match the staged row's path and reports its object as orphaned. collect(dry_run=False) deleted it, and fetch1 for rec_id=2 then raised MissingExternalFile.
  • Any store, after stores.default changes.
    • The empty store name of the staged rows resolves to the new default, so their objects can no longer be read. Rows inserted with insert1 name local and still read.
    • The collector for local also treats the staged objects as belonging to another store (gc.py#L183), and reports them as orphaned.
  • Part tables. The path gets the part's class name alone (Item), not the table name session__item.

These are gaps left by #1465 and #1575, which aligned the metadata of the regular insert path.

Expected

staged_insert1 stores the same path layout and the same resolved store name as insert1.

Suggested fix

Build the path with self._table.table_name, and record config.resolve_store_name(store_name).

主要語言
Python
星號
198
分支
103
平均合併
6 天 1 小時
30 天內合併 PR
10

環境準備

  • 提供 Dockerfile 或 Docker Compose 檔案
  • 沒有 Pull Request 範本
  • 閱讀貢獻指南

從這裡開始

  1. 先讀完整個 Issue,再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫,在一個分支上完成修改。
  4. 送出 Pull Request,並在描述裡引用這個 Issue 編號。

datajoint/datajoint-python 的其他 Issue

查看 datajoint/datajoint-python 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。