[Bug] AppendTableITCase.testCompactionInStreamingMode is flaky and can time out under CI load
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 78/100
- Issue type
- Bug
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- java
- Domain
- stream-processing, testing-qa
Research direction
Start with AppendTableITCase.testCompactionInStreamingMode and its sibling testCompactionInStreamingModeWithMaxWatermark. Run the tests under CI-like load and inspect their 60-second polling limit; done means both tests reliably observe a COMPACT snapshot without timing out.
Written by the indexing model from the issue text.
Description
Search before asking
- I searched in the issues and found nothing similar.
Paimon version
master (645ce6f8d)
Compute Engine
Flink
Minimal reproduce step
AppendTableITCase.testCompactionInStreamingMode, and its sibling testCompactionInStreamingModeWithMaxWatermark, run a streaming INSERT from a datagen source at rows-per-second = 1 into an append table configured with compaction.min.file-num = 4, continuous.discovery-interval = 1 s, and a 500 ms checkpoint interval. They then poll once a second, for up to 60 s, for a snapshot whose commitKind is COMPACT.
In CI the poll sometimes reaches the 60 s limit and the test fails:
java.lang.RuntimeException: Time up for streaming execute, don't get expected result.
One example: https://github.com/apache/paimon/actions/runs/36149147465/job/108117843839
What doesn't meet your expectations?
The test should not time out. Nothing is broken in the append-compaction pipeline; the failure is a timing race inside the test.
On a healthy run the first COMPACT snapshot arrives in under 20 s: the streaming job starts, four small files accumulate at about one per second, the coordinator picks them up on its next discovery cycle, compaction runs, and the compacted snapshot commits. The 60 s budget is usually comfortable. Under CI load, streaming job deployment and checkpoint/commit progress can stall, and the first COMPACT snapshot occasionally shows up after 60 s.
The same test was reported in #1634 and closed without a fix.
Anything else?
#1634 suggested lowering compaction.min.file-num as a workaround. That only shortens the file-accumulation phase (roughly 4 s at one file per second), which is not where the time goes when the test times out. The stall is in job deployment and checkpoint progress, so giving the wait a larger timeout is the direct fix.
Are you willing to submit a PR?
- I'm willing to submit a PR!
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 490
Getting set up
We have not checked this project's setup files yet. Start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/paimon
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Maintainers usually reply within 1 day
-
REST Catalog alterTable silently reports success when the server returns a non-table/column 404Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
refinedmods/refinedstorage2#1414 · 1 comment ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
In Java's `LongBoundedSum`, setting `lower` to `Long.MIN_VALUE` under-estimates the sensitivityOpen
Difficulty 2/5 1-3 hours Newbie friendliness 73/100
google/differential-privacy#489 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
link-check link-check:manual
Difficulty 2/5 1-3 hours Newbie friendliness 85/100