Validate that referenced DataItems are registered when writing
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 55/100
Research direction
Start by reading write_models and validate_for_write to understand where rows are written and how project_id is available; inspect the referenced DataItemDataSetAssociation.dataitem_id, ClusterMembership.item, and EmbeddingLocation.dataitem_id fields. Done means writes reject references missing from the DataItem table for the same project and report missing IDs or examples.
Written by the indexing model from the issue text.
Description
Problem
When write_models writes a table that references DataItems, it doesn't check whether those DataItems exist in the DataItem table for the same project. validate_for_write checks one row at a time and never reads other tables, so rows that point to unregistered DataItems are written without error.
Affected tables include DataItemDataSetAssociation.dataitem_id, ClusterMembership.item and EmbeddingLocation.dataitem_id.
ETL notebooks currently guard against this with their own asserts (e.g. etl_v1dd1196_01 checks against a local registered_ids set), but callers of write_models get no such check.
Proposal
Before writing, check that every referenced dataitem_id exists in the DataItem table under the row's project_id. Raise an error listing the missing IDs, or a few examples if there are many.
Related #81
- Dominant language
- Python
- Stars
- 4
- Forks
- 1
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 7
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- No Dockerfile or Docker Compose file
- Has a pull request template
- No contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from AllenInstitute/ConnectsCommonConnectivity
-
enhancement io
Difficulty 2/5 1-3 hours Newbie friendliness 65/100
-
bug documentation schema
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
-
bug schema
Difficulty 1/5 Under an hour Newbie friendliness 68/100
-
enhancement schema
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
-
question
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
All issues in AllenInstitute/ConnectsCommonConnectivity
Similar issues
-
Difficulty 1/5 Under an hour Newbie friendliness 78/100
521xueweihan/HelloGitHub#4237 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
jejjohnson/gaussx#721 ·
Maintainers usually reply within 1 day
-
mock-core-configs/etc/mock/rocky-10-riscv64.cfg contains the wrong architecture in config_opts['root']Possibly taken @jasonmontleon claimed this 4 days ago. Open
Difficulty 1/5 Under an hour Newbie friendliness 94/100
rpm-software-management/mock#1834 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 75/100
elastic/elastic-otel-python#683 ·
Maintainers usually reply within 1 day
-
ci-rerun: collect.py fails a whole pr-babysit cycle on one transient GitHub 5xx and never retriesOpenfollow-up
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
Maintainers usually reply within 1 day