Empty JSON array ([]) as a lookup-table reference file causes a confusing IntegrityError in load_table()
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 84/100
Research direction
Start in astrodbkit.astrodb.Database.load_table(), where data from json.load() is passed to insert().values(data). Reproduce the behavior with a top-level [] reference-table file and follow build_db_from_json(); done means the empty file is treated as a no-op and no default-valued row or IntegrityError is produced.
Written by the indexing model from the issue text.
Description
Where: astrodbkit.astrodb.Database.load_table() does conn.execute(self.metadata.tables[table].insert().values(data)) where data comes straight from json.load(). If the JSON file's top-level array is empty ([]), SQLAlchemy's insert().values([]) is interpreted as "insert one row using column defaults" (a known SQLAlchemy quirk: an empty list to a multi-row .values() isn't a no-op), not "insert nothing."
What happened: After emptying Publications.json, SourceTypeList.json, and ParameterList.json to [] (clearing stale template-example placeholder rows before real data existed), build_db_from_json() failed with sqlite3.IntegrityError: NOT NULL constraint failed: Publications.reference — a single default-valued row was attempted, which violated the non-nullable reference column. The error message gives no hint that the root cause was an empty-array JSON file rather than a real data problem.
Workaround: Deleted the three now-empty JSON files entirely instead of leaving them as [] — load_table()'s existing if os.path.exists(filename): guard cleanly skips a missing file (with an optional verbose "not found" message), which is the actual correct way to represent "no data yet" for a reference table.
Suggested change: Database.load_table() should special-case if not data: return before calling insert().values(data), so an empty-array reference file is a documented no-op instead of an inserted row of column defaults that then trips downstream NOT NULL constraints.
Cross-filed: Also filed against the astrodb-bot skills repo as astrodbtoolkit/astrodb-bot#99, which documents the same gotcha from the skill-user workaround side; this issue is for the actual astrodbkit fix.
Reported from a gotchas.md log filed by a skill user (2026-08-28).
- Dominant language
- Python
- Stars
- 12
- Forks
- 5
- PR merge metrics
- No merged PRs in 30d
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from astrodbtoolkit/AstrodbKit
-
Nulls and load_table Open
Difficulty 4/5 3-5 days Newbie friendliness 35/100
astrodbtoolkit/AstrodbKit#109 ·
-
astrodbtoolkit/AstrodbKit#105 · 1 assignee ·
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
astrodbtoolkit/AstrodbKit#104 ·
-
bug
astrodbtoolkit/AstrodbKit#103 · 1 assignee ·
-
Difficulty 3/5 1-2 days Newbie friendliness 42/100
astrodbtoolkit/AstrodbKit#101 · 1 comment ·
All issues in astrodbtoolkit/AstrodbKit
Similar issues
-
documentation help wanted
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
simonw/sqlite-utils#872 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100