SqlCatalog table operations should filter on `iceberg_type`

Open
#3,337 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
68/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Quiet
Tech stack
python, sql
Domain
databases

Research direction

Start in the SqlCatalog table-operation methods, compare their predicates with the existing iceberg_type handling in list_tables from #3263, and reproduce the issue with the provided SQLite setup and view row. Done means load_table, drop_table, rename_table, and commit_table no longer treat rows with iceberg_type VIEW as tables, while TABLE and NULL rows continue to work.

Written by the indexing model from the issue text.

Description

Feature Request / Improvement

In #3263, the iceberg_type column was added to the SQLCatalog and filters it in list_tables, but the other table operations (i.e. load_table, drop_table, rename_table, commit_table) don't check it yet. That means a view row written by iceberg java or iceberg-rust can bleed in pythons SQLCatalog table operations.

Java's V1 catalog applies WHERE (iceberg_type = 'TABLE' OR iceberg_type IS NULL) everywhere (JdbcUtil.java#L168). We should do the same.

Probably worth extracting a small helper since the predicate would now appear in ~5+ places.

Repro steps:

catalog = SqlCatalog("test", uri="sqlite:///...", warehouse="...")
catalog.create_namespace("ns")

# bypass and sim java/rust writing a view row
with catalog.engine.connect() as conn:
    conn.execute(text(
        "INSERT INTO iceberg_tables VALUES "
        "('test', 'ns', 'a_view', 's3://fake/m.json', NULL, 'VIEW')"
    ))
    conn.commit()

catalog.drop_table(("ns", "a_view"))  # Silently deletes the view row :/
Dominant language
Python
Stars
1.1k
Forks
589
Avg merge
2d 4h
Merged PRs (30d)
72

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/iceberg-python

All issues in apache/iceberg-python

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.