Table properties override catalog configuration when constructing FileIO
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 55/100
Research direction
Start at Catalog._load_file_io and trace the REST _response_to_table and _response_to_staged_table paths, then inspect the corresponding Glue, Hive, SQL, DynamoDB, BigQuery, and StaticTable.from_metadata call sites. Verify how metadata properties are merged into FileIO construction and add coverage showing catalog configuration remains authoritative for deployment settings across the affected paths.
Written by the indexing model from the issue text.
Description
When a table is loaded, its metadata properties are merged into the property map used to construct the table's FileIO, and they take precedence over the operator's catalog configuration.
Catalog._load_file_io computes:
load_file_io({**self.properties, **properties}, location)
where properties is metadata.properties. Because table properties come last, a value stored in a table overrides the same key configured on the catalog. The REST path does the same at _response_to_table / _response_to_staged_table, and Glue, Hive, SQL, DynamoDB, BigQuery and StaticTable.from_metadata all funnel table metadata into FileIO construction the same way — 12 call sites in total.
The keys this reaches include implementation selection (py-io-impl, s3.retry-strategy-impl) and transport configuration (s3.endpoint, s3.proxy-uri, s3.signer / s3.signer.uri, gcs.service.host, hf.endpoint, the ADLS storage authorities). These are deployment concerns — an operator sets them on the catalog — but any principal who can commit to a table can currently override them for everyone who reads it.
Issue investigation generated via claude, reviewed by Sung, Kevin, Fokko.
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 589
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 72
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/iceberg-python
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
apache/iceberg-python#3996 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
apache/iceberg-python#3979 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
apache/iceberg-python#3885 ·
-
[Bug] PyArrowFileIO fails to propagate s3.ssl.ca-cert to pyarrow.fs.S3FileSystem tls_ca_file_path Open
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
apache/iceberg-python#3866 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
apache/iceberg-python#3836 · 1 comment ·
All issues in apache/iceberg-python
Similar issues
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 90/100
learningequality/ricecooker#747 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
BSData/horus-heresy-3rd-edition#3171 ·
-
enhancement
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
run-llama/llama_index#23199 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
KhronosGroup/glTF-Blender-IO#2769 ·