Empty JSON array ([]) as a lookup-table reference file causes a confusing IntegrityError in load_table()

Open Beginner friendly
#110 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
84/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
python, sqlalchemy
Domain
databases

Research direction

Start in astrodbkit.astrodb.Database.load_table(), where data from json.load() is passed to insert().values(data). Reproduce the behavior with a top-level [] reference-table file and follow build_db_from_json(); done means the empty file is treated as a no-op and no default-valued row or IntegrityError is produced.

Written by the indexing model from the issue text.

Description

Where: astrodbkit.astrodb.Database.load_table() does conn.execute(self.metadata.tables[table].insert().values(data)) where data comes straight from json.load(). If the JSON file's top-level array is empty ([]), SQLAlchemy's insert().values([]) is interpreted as "insert one row using column defaults" (a known SQLAlchemy quirk: an empty list to a multi-row .values() isn't a no-op), not "insert nothing."

What happened: After emptying Publications.json, SourceTypeList.json, and ParameterList.json to [] (clearing stale template-example placeholder rows before real data existed), build_db_from_json() failed with sqlite3.IntegrityError: NOT NULL constraint failed: Publications.reference — a single default-valued row was attempted, which violated the non-nullable reference column. The error message gives no hint that the root cause was an empty-array JSON file rather than a real data problem.

Workaround: Deleted the three now-empty JSON files entirely instead of leaving them as []load_table()'s existing if os.path.exists(filename): guard cleanly skips a missing file (with an optional verbose "not found" message), which is the actual correct way to represent "no data yet" for a reference table.

Suggested change: Database.load_table() should special-case if not data: return before calling insert().values(data), so an empty-array reference file is a documented no-op instead of an inserted row of column defaults that then trips downstream NOT NULL constraints.

Cross-filed: Also filed against the astrodb-bot skills repo as astrodbtoolkit/astrodb-bot#99, which documents the same gotcha from the skill-user workaround side; this issue is for the actual astrodbkit fix.


Reported from a gotchas.md log filed by a skill user (2026-08-28).

Dominant language
Python
Stars
12
Forks
5
PR merge metrics
No merged PRs in 30d

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from astrodbtoolkit/AstrodbKit

All issues in astrodbtoolkit/AstrodbKit

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.