Skip to content

[Bug] RowFormatReaderFactory leaks the input stream when reading the footer fails #9006

Description

@wombatu-kun

Search before asking

  • I searched in the issues and found nothing similar.

Paimon version

master, 142f823

Compute Engine

Engine independent.

Minimal reproduce step

Found by code inspection rather than from a failing job. Read a row format file whose tail is truncated, or whose footer is corrupt, so that footer or block index parsing throws.

What doesn't meet your expectations?

RowFormatReaderFactory.createReader opens the stream first and only transfers ownership to RowFormatReader on the last line:

SeekableInputStream in = fileIO.newInputStream(path);

int tailSize = (int) Math.min(TAIL_PREFETCH_SIZE, fileSize);
long tailOffset = fileSize - tailSize;
in.seek(tailOffset);
byte[] tailBuf = new byte[tailSize];
IOUtils.readFully(in, tailBuf);

RowFileFooter footer = RowFileFooter.readFrom(tailBuf, tailSize - RowFileFooter.FOOTER_SIZE);

RowBlockIndex blockIndex;
...
    blockIndex = RowBlockIndex.readFrom(in, footer.indexOffset, footer.indexLength);

return new RowFormatReader(in, path, footer, blockIndex, rowType, projection, context.selection());

There is no try / catch anywhere in the method, so a throw from in.seek, IOUtils.readFully, RowFileFooter.readFrom or RowBlockIndex.readFrom leaves in open with no owner. Scanning a set of files where several are corrupt leaks one stream per file.

Anything else?

Same interaction as the sibling report on CachingSeekableInputStream: with the lifetime tracking added in #8962, a stream that is never closed holds its lease forever, so that entry's FileIO is never released. That matches today's behaviour rather than regressing it, but it does keep the fix from reaching the affected entries.

Fix shape: close in before rethrowing.

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions