Allow AccessDeniedExceptions To Fail Silently On Auto Creating Tables

Open Beginner friendly
#13,758 2 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
72/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Quiet
Tech stack
aws, java
Domain
backend, databases

Research direction

Start in kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/data/IcebergWriterFactory.java, at the block that catches AlreadyExistsException and ForbiddenException during automatic table creation. Trace the auto-create path and verify that an AWS Glue AccessDeniedException is handled there without failing; done means existing databases do not prevent automatic table creation when CreateDatabase permission is missing.

Written by the indexing model from the issue text.

Description

bug
Apache Iceberg version

None

Query engine

None

Please describe the bug 🐞

Problem

I noticed that when we enabled the iceberg.tables.auto-create-enabled configuration that we were failing to create tables automatically, even when the Database exists already.

I noticed in this part of the code there is an explicit catching and ignoring of AlreadyExistsException and ForbiddenException to check if the database already exists.
https://github.com/apache/iceberg/blob/main/kafka-connect/kafka-connect/src/main/java/org/apache/iceberg/connect/data/IcebergWriterFactory.java

Now, we are on aws and when we are missing the CreateDatabase permission using Glue, we get this error

Caused by: software.amazon.awssdk.services.glue.model.AccessDeniedException: User: XXXXX is not authorized to perform: glue:CreateDatabase on resource: XXXXXX because no identity-based policy allows the glue:CreateDatabase action (Service: Glue, Status Code: 400, Request ID: XXXX)

And this does not extend the ForbiddenException. https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/glue/model/AccessDeniedException.html

Expectation

I would expect this access denied permission to also be a type of forbidden exception, even though it's wrapped around a 400 error instead of 403.

Proposal/Solution

I would propose catching also AccessDeniedExceptions in this block. As our current workaround, for permission reasons, is to disable the flag altogether and create the tables manually.

Willingness to contribute
  • I can contribute a fix for this bug independently
  • I would be willing to contribute a fix for this bug with guidance from the Iceberg community
  • I cannot contribute a fix for this bug at this time
Dominant language
Java
Stars
9.3k
Forks
3.5k
Avg merge
2d 11h
Merged PRs (30d)
143

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/iceberg

All issues in apache/iceberg

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.