[Bug] RowFormatReaderFactory leaks the input stream when reading the footer fails

Open Beginner friendly
#9,006 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
java

Research direction

Start at RowFormatReaderFactory.createReader and trace ownership of the stream after opening it, especially the footer and block-index parsing paths. Use the truncated-tail or corrupt-footer scenario described in the issue; done means failures close the input stream before being rethrown, including errors from seek, read, footer parsing, and block-index parsing.

Written by the indexing model from the issue text.

Description

Search before asking
  • I searched in the issues and found nothing similar.
Paimon version

master, 142f8239b

Compute Engine

Engine independent.

Minimal reproduce step

Found by code inspection rather than from a failing job. Read a row format file whose tail is truncated, or whose footer is corrupt, so that footer or block index parsing throws.

What doesn't meet your expectations?

RowFormatReaderFactory.createReader opens the stream first and only transfers ownership to RowFormatReader on the last line:

SeekableInputStream in = fileIO.newInputStream(path);

int tailSize = (int) Math.min(TAIL_PREFETCH_SIZE, fileSize);
long tailOffset = fileSize - tailSize;
in.seek(tailOffset);
byte[] tailBuf = new byte[tailSize];
IOUtils.readFully(in, tailBuf);

RowFileFooter footer = RowFileFooter.readFrom(tailBuf, tailSize - RowFileFooter.FOOTER_SIZE);

RowBlockIndex blockIndex;
...
    blockIndex = RowBlockIndex.readFrom(in, footer.indexOffset, footer.indexLength);

return new RowFormatReader(in, path, footer, blockIndex, rowType, projection, context.selection());

There is no try / catch anywhere in the method, so a throw from in.seek, IOUtils.readFully, RowFileFooter.readFrom or RowBlockIndex.readFrom leaves in open with no owner. Scanning a set of files where several are corrupt leaks one stream per file.

Anything else?

Same interaction as the sibling report on CachingSeekableInputStream: with the lifetime tracking added in #8962, a stream that is never closed holds its lease forever, so that entry's FileIO is never released. That matches today's behaviour rather than regressing it, but it does keep the fix from reaching the affected entries.

Fix shape: close in before rethrowing.

Are you willing to submit a PR?
  • I'm willing to submit a PR!
Dominant language
Java
Stars
3.4k
Forks
1.4k
Avg merge
1d 14h
Merged PRs (30d)
468

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/paimon

All issues in apache/paimon

Similar issues

More Java issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.