Migrate to DataJoint 2.x: 14 longblob attributes silently corrupt data, 1 attribute fails to declare
还没有人认领这个 Issue。
评估
调研方向
从 setup.py 开始，然后在包中搜索 longblob、attach、dj.schema 以及受影响的 14 个声明；issue 标明了所需的 DataJoint 2.x 映射。在修改定义之前，阅读链接的迁移指南并检查 tests/。完成的标准是所有声明都能在 DataJoint 2.x 下导入，依赖项的最低版本已提高，并且测试覆盖转换后的类型。
由索引模型根据 Issue 内容生成。
描述
This element has not been migrated to DataJoint 2.x. DataJoint no longer supports
pre-2.x, all schemas should be migrated, and a schema is either 2.x or pre-2.x — the
two are not interoperable.
Findings below come from running 2.x's own declaration parser over every attribute
declared in this package, so the counts are exact rather than estimated.
Silent data loss — 14 longblob attributes
Under 2.x a bare longblob is a raw binary column, not a DataJoint blob. Inserting a
numpy array into one stores str(array) — numpy's text repr, with the middle elided
for large arrays — and nothing raises on insert or on fetch. dtype and shape are
gone, and the values are not recoverable from the database afterwards.
This element declares 14 such attributes. The fix is to declare <blob>; the
underlying MySQL column type is longblob either way, so it is a definition edit and
not a table migration — as long as it happens before rows are written under 2.x.
Hard failures at declaration
These raise DataJointError under 2.x, so the affected tables cannot be declared at all:
| spelling | count | |
|---|---|---|
attach |
1 | → <attach> |
Also needed
- Native numeric types (12
int/smallint→int32/int16, 35float→float32). These work correctly under 2.x —
no corruption — but they are native types that warn at declaration and are not
portable across backends. Worth converting while the files are open. dj.schema→dj.Schema(4 call sites).dj.schemawas removed in 2.x;
importing the module raisesAttributeErrorbefore any table is declared.- Raise the DataJoint floor —
setup.pydatajoint>=0.14.6→datajoint>=2.3. The unpinned
floor is why a fresh install resolves 2.x today and hits all of the above.
Testing
This repo has a tests/ directory, which is a real advantage — please extend it to
cover the converted types as part of the migration.
How to do it
Work from Migrate to DataJoint 2.0.
It gives the full legacy → 2.0 type mapping, explains the :<blob>: column-comment
marker that is the mechanism behind the blob problem, and lays out the four phases.
This issue is Phase I — code only, against empty _v2 schemas, with no impact on
existing data. Migrating existing data is Phase III and is tracked separately.
Related: the upstream guard that would have made this loud instead of silent is
https://github.com/datajoint/datajoint-python/issues/1527. The same migration is
underway across the other elements.
- 主要语言
- Jupyter Notebook
- 星标
- 10
- 派生
- 44
- PR 合并指标
- 30 天内没有已合并 PR
环境准备
在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。
- 没有 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
datajoint/element-array-ephys 的其他 Issue
-
enhancement
难度 3/5 1-2 天 新手友好度 45/100
datajoint/element-array-ephys#207 ·
-
bug
难度 3/5 1-2 天 新手友好度 35/100
datajoint/element-array-ephys#206 ·
-
bug
难度 2/5 1-3 小时 新手友好度 35/100
datajoint/element-array-ephys#154 · 1 条评论 ·
-
bug
难度 3/5 1-2 天 新手友好度 35/100
datajoint/element-array-ephys#163 ·
-
bug
难度 3/5 1-2 天 新手友好度 30/100
datajoint/element-array-ephys#162 ·
查看 datajoint/element-array-ephys 的全部 Issue
相似的 Issue
-
难度 1/5 1 小时以内 新手友好度 77/100
-
难度 2/5 1-3 小时 新手友好度 88/100
Wetzel402/Skylite-UX#125 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 78/100
oracle/rust-oracledb#53 ·
-
难度 2/5 1-3 小时 新手友好度 78/100
-
难度 2/5 1-3 小时 新手友好度 85/100
objectionary/eo-graphs#82 ·