Native Iceberg errors print data and delete file paths with their credentials

Open Beginner friendly
  • Is something wrong?
#7,559 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 2 days

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
73/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
rust, scala
Domain
backend, security

Research direction

Read iceberg_location_scoped.rs, iceberg_scan.rs, iceberg_write.rs, and planner.rs, along with the URL redaction helpers added in #7127. Find the listed error messages and check how the #7128 changes apply redaction to similar paths. Done when each message prints paths without credentials or query strings and the relevant tests pass.

Written by the indexing model from the issue text.

Description

requires-triage
Describe the bug

Found while working on #7128. Several native Iceberg errors print a data or delete file path exactly as the catalog recorded it:

  • iceberg_location_scoped.rs: "Invalid S3 path {path}", "S3 path {path} has no bucket"
  • iceberg_scan.rs: "Failed to stat delete file '{path}'" and the size check after it
  • iceberg_write.rs: "{path} is not the data file this output was made for", "... on the output of data file {path}"
  • planner.rs: "Delete file '{file_path}' has a negative {what}"

A location with a password in its userinfo or a SAS token or signature in its query string would end up in task failures and logs. Catalog locations rarely carry credentials, so this is low risk, but it is the same leak #7128 fixes for Parquet, Delta and object store paths.

Steps to reproduce

Each message formats the path with {path} as is, so a location like s3://alice:SECRET@bucket/t/delete.parquet?sig=TOKEN reaching any of them is printed with SECRET and TOKEN.

Expected behavior

These messages show the path without its password, username or query, using the URL redaction helpers #7127 adds, as the #7128 pull requests do for the other paths.

Additional context

Not part of the four #7128 pull requests, which cover object store setup, the Parquet writer, native Delta and the JVM rejected path reason.

Dominant language
Scala
Stars
1.3k
Forks
414
Avg merge
3d 5h
Merged PRs (30d)
228

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/datafusion-comet

All issues in apache/datafusion-comet

Similar issues

More Scala issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.