Hacktoberfest 2026: as issues que os mantenedores marcaram para outubro, abertas e boas para iniciantes. Ver issues do Hacktoberfest

Migrate to DataJoint 2.x: 14 longblob attributes silently corrupt data, 1 attribute fails to declare

Aberta
  • Algo errado?
#230 0 comentários 0 reações 0 responsáveis Ver no GitHub

Ninguém assumiu esta issue ainda.

Avaliação

Dificuldade
4/5
Tempo estimado
3-5 dias
Facilidade para iniciantes
58/100
Tipo de issue
Refatoração
Clareza
Razoavelmente clara
Status de atividade
Pouca atividade
Stack de tecnologia
python
Domínio
databases

Direção de pesquisa

Comece por setup.py e, em seguida, pesquise no pacote por longblob, attach, dj.schema e pelas 14 declarações afetadas; a issue identifica os mapeamentos necessários do DataJoint 2.x. Leia o guia de migração vinculado e inspecione tests/ antes de fazer as alterações nas definições. O trabalho estará concluído quando todas as declarações forem importadas no DataJoint 2.x, o limite mínimo da dependência tiver sido elevado e os testes cobrirem os tipos convertidos.

Escrita pelo modelo de indexação a partir do texto da issue.

Descrição

This element has not been migrated to DataJoint 2.x. DataJoint no longer supports
pre-2.x, all schemas should be migrated, and a schema is either 2.x or pre-2.x — the
two are not interoperable.

Findings below come from running 2.x's own declaration parser over every attribute
declared in this package, so the counts are exact rather than estimated.

Silent data loss — 14 longblob attributes

Under 2.x a bare longblob is a raw binary column, not a DataJoint blob. Inserting a
numpy array into one stores str(array) — numpy's text repr, with the middle elided
for large arrays — and nothing raises on insert or on fetch. dtype and shape are
gone, and the values are not recoverable from the database afterwards.

This element declares 14 such attributes. The fix is to declare <blob>; the
underlying MySQL column type is longblob either way, so it is a definition edit and
not a table migration — as long as it happens before rows are written under 2.x.

Hard failures at declaration

These raise DataJointError under 2.x, so the affected tables cannot be declared at all:

spelling count
attach 1 → <attach>
Also needed
  • Native numeric types (12 int/smallint → int32/int16, 35 float → float32). These work correctly under 2.x —
    no corruption — but they are native types that warn at declaration and are not
    portable across backends. Worth converting while the files are open.
  • dj.schema → dj.Schema (4 call sites). dj.schema was removed in 2.x;
    importing the module raises AttributeError before any table is declared.
  • Raise the DataJoint floor — setup.py datajoint>=0.14.6 → datajoint>=2.3. The unpinned
    floor is why a fresh install resolves 2.x today and hits all of the above.
Testing

This repo has a tests/ directory, which is a real advantage — please extend it to
cover the converted types as part of the migration.

How to do it

Work from Migrate to DataJoint 2.0.
It gives the full legacy → 2.0 type mapping, explains the :<blob>: column-comment
marker that is the mechanism behind the blob problem, and lays out the four phases.
This issue is Phase I — code only, against empty _v2 schemas, with no impact on
existing data. Migrating existing data is Phase III and is tracked separately.

Related: the upstream guard that would have made this loud instead of silent is
https://github.com/datajoint/datajoint-python/issues/1527. The same migration is
underway across the other elements.

Linguagem predominante
Jupyter Notebook
Estrelas
10
Forks
44
Nenhum PR com merge em 30d

Preparar o ambiente

Abrir no Codespaces

Inicia o contêiner de desenvolvimento do projeto no navegador, com a sua própria conta do GitHub.

Primeiros passos

  1. Leia a issue inteira e depois o guia de contribuição do projeto.
  2. Comente na issue dizendo que vai assumir — evita que duas pessoas façam o mesmo trabalho.
  3. Faça um fork do repositório e trabalhe em uma branch.
  4. Abra um pull request que referencie o número da issue.

Mais de datajoint/element-array-ephys

Todas as issues de datajoint/element-array-ephys

Issues semelhantes

Mais issues de Databases

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.