apache/hudi

Investigate and fix flaky tests in ITTestHoodieDataSource

Open

#16445 opened on Nov 30, 2025

View on GitHub
 (0 comments) (0 reactions) (1 assignee)Java (4,823 stars) (2,431 forks)batch import
area:testsfrom-jiragood first issuepriority:high

Description

{code:java} 2024-04-17T15:25:52.8085340Z [ERROR] Tests run: 132, Failures: 2, Errors: 0, Skipped: 1, Time elapsed: 1,042.604 s <<< FAILURE! - in org.apache.hudi.table.ITTestHoodieDataSource 2024-04-17T15:25:52.8088904Z [ERROR] testWriteAndReadWithDataSkipping{HoodieTableType}[1] Time elapsed: 2.654 s <<< FAILURE! 2024-04-17T15:25:52.8090232Z java.lang.AssertionError: 2024-04-17T15:25:52.8090677Z 2024-04-17T15:25:52.8093540Z Expected: is "[+I[id1, Danny, 23, 1970-01-01T00:00:01, par1], +I[id2, Stephen, 33, 1970-01-01T00:00:02, par1], +I[id3, Julian, 53, 1970-01-01T00:00:03, par2], +I[id4, Fabian, 31, 1970-01-01T00:00:04, par2], +I[id5, Sophia, 18, 1970-01-01T00:00:05, par3], +I[id6, Emma, 20, 1970-01-01T00:00:06, par3], +I[id7, Bob, 44, 1970-01-01T00:00:07, par4], +I[id8, Han, 56, 1970-01-01T00:00:08, par4]]" 2024-04-17T15:25:52.8096227Z but: was "[]" 2024-04-17T15:25:52.8097376Z at org.apache.hudi.table.ITTestHoodieDataSource.testWriteAndReadWithDataSkipping(ITTestHoodieDataSource.java:1613) 2024-04-17T15:25:52.8098523Z 2024-04-17T15:25:52.8099153Z [ERROR] testWriteAndReadWithDataSkipping{HoodieTableType}[2] Time elapsed: 2.652 s <<< FAILURE! 2024-04-17T15:25:52.8100231Z java.lang.AssertionError: 2024-04-17T15:25:52.8100586Z 2024-04-17T15:25:52.8103281Z Expected: is "[+I[id1, Danny, 23, 1970-01-01T00:00:01, par1], +I[id2, Stephen, 33, 1970-01-01T00:00:02, par1], +I[id3, Julian, 53, 1970-01-01T00:00:03, par2], +I[id4, Fabian, 31, 1970-01-01T00:00:04, par2], +I[id5, Sophia, 18, 1970-01-01T00:00:05, par3], +I[id6, Emma, 20, 1970-01-01T00:00:06, par3], +I[id7, Bob, 44, 1970-01-01T00:00:07, par4], +I[id8, Han, 56, 1970-01-01T00:00:08, par4]]" 2024-04-17T15:25:52.8105999Z but: was "[]" 2024-04-17T15:25:52.8107231Z at org.apache.hudi.table.ITTestHoodieDataSource.testWriteAndReadWithDataSkipping(ITTestHoodieDataSource.java:1613) {code}  

JIRA info

Comments

13/Sep/24 04:33;shubhamgondane;I can take a look into this, if this is not assigned to anyone.

 

Thanks,

Shubham;;;

24/Sep/24 05:39;xushiyan;[~shubhamgondane]  not seeing any PR linked to this jira, pls feel free to file a PR.;;;

29/Sep/24 23:06;shubhamgondane;[~xushiyan]  sure, will file a PR;;;

Contributor guide

Tech stack
java
Domain
testing
Issue type
test
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
4
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-2 days
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
active
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
mostly clear
Prerequisites
JavaApache HudiIntegration testingSpark
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
35
Research direction
Start by examining the test method testWriteAndReadWithDataSkipping in ITTestHoodieDataSource.java around line 1613. The failure shows that the data returned is empty, suggesting a problem in the write path or the read path with data skipping. Check the test setup and any recent changes to the data skipping module, and try to reproduce the flaky test locally with the identified HoodieTableType configurations.