Add defensive limits for BMFF metadata counts during parsing

[uwezkhan]

This patch adds internal defensive limits for attacker-controlled BMFF metadata counts parsed from AVIF/HEIF files.

Several parser paths allocate memory or perform repeated processing based on counts read directly from the bitstream (items, properties, extents, groups, etc.). Malformed files can use excessive values here to trigger disproportionate memory usage and parser overhead.

To mitigate this, this change introduces conservative internal upper bounds for key metadata structures and rejects files that exceed those limits during parsing.

The limits are intentionally generous for legitimate AVIF content while preventing pathological resource-exhaustion cases from malformed inputs.

[wantehchang]

It is a little excessive to add four decoder options that most users of libavif are unlikely to understand.

Also, the libavif code related to these four limits grows an array one element at a time. So the input file must have as many objects to cause the array size to grow. It is more important to defend against a small malicious file that causes libavif to allocate a lot of memory, i.e., what your description refers to as "disproportionate memory usage". Could you generate a PoC to demonstrate that? Alternatively, it would be good if you could justify why the default limits are guaranteed to be large enough in practice.

[y-guyon]

This was resolved without addressing the comments. Unresolving.

[uwezkhan]

Thanks for pointing that out. I've updated the patch to remove the four public decoder options entirely and replaced them with internal implementation limits, so there is no longer any API change.

Regarding the resource-exhaustion concern, I agree that the key question is whether the affected metadata structures can cause disproportionate parser cost in practice. I'm investigating that further and will follow up with either concrete measurements/PoCs or a narrower justification for the remaining limits.

[y-guyon]

In my opinion this was resolved without addressing the comments. Unresolving.

It is a little excessive to add four decoder options that most users of libavif are unlikely to understand.

Would introducing a single boolean suit you? I am worried about adding hardcoded limits that cannot be disabled by any compile- or run-time flag.

[wantehchang]

Yannis: You are much more familiar with ISOBMFF than I am, so I defer the risk assessment and API design to you. I think you can make a better judgment.

[uwezkhan]

On the boolean: it's moot if the limits go away, since there'd be nothing to gate. I left the longer answer in a PR comment, but the short version is these counts are already bounded by file size (arrays grow one push at a time), so the caps aren't buying anything over the existing limits.

[uwezkhan]

@wantehchang review on this PR?

[uwezkhan]

hey can i get a review on this? its been a month

[y-guyon]

This was resolved without addressing the comments. Unresolving.

[uwezkhan]

any update ?

[y-guyon]

I think a file/PoC exhibiting "disproportionate memory usage and parser overhead" would prove the need for this PR. Could you first add a test triggering a memory allocation failure, and/or an image that takes too long to decode, that cannot be handled by a reasonable imageSizeLimit value in the existing API please?

[uwezkhan]

Went back through the array code to answer this properly. The item, property, extent and group counts don't pre-size anything: the arrays start small (8/16/1/2 elements) and grow by doubling in avifArrayPush, one push per object actually read from the stream. So an array only reaches N if the file physically contains N objects, and the allocation stays bounded by the input size. Wantehchang's read was correct, there's no small-file amplification here.

Because of that I can't produce a PoC where one of these counts causes an allocation failure, or a decode that runs long, that a reasonable imageSizeLimit or the file size itself wouldn't already bound. The justification for the count limits doesn't hold up.

So I'm fine closing this. If there's a specific box you'd still want a hard cap on, tell me which one and I'll cut the PR down to just that with a real repro.