delta-io/delta

[Kernel] [TODO] Assert that log files belong to the table

Open

#1,968 opened on Aug 9, 2023

View on GitHub
 (3 comments) (0 reactions) (0 assignees)Scala (8,807 stars) (2,100 forks)batch import
enhancementgood first issuekernel

Description

Complete the below TODO.

/**
 * Verifies that a set of delta or checkpoint files to be read actually belongs to this table.
 */
private void assertLogFilesBelongToTable(Path logPath, Stream<FileStatus> allFiles) {
    // TODO:
}

https://github.com/delta-io/delta/blob/efa566fc24af0b2c65b08c2a7ad81e9c865e0e83/kernel/kernel-api/src/main/java/io/delta/kernel/internal/replay/LogReplay.java#L158

Contributor guide

Tech stack
java
Domain
database
Issue type
feature
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
fresh
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
JavaDelta Lake basicsFile path validation
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
70
Research direction
Examine the LogReplay.java file around line 158. Understand how logPath and allFiles are used. The assertion should check that each file's path starts with the table's log path or contains the table ID. Review existing validation methods in the same class for patterns. Consider using Java's FileStatus and Path APIs. The goal is to throw an appropriate exception if a file does not belong.