validation improvements, fixes #301 #331
Draft
+410
−201
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Was: Became: |
sscobici
force-pushed
the
mp4-panic-fixes
branch
2 times, most recently
from
941897b to
5d913fd
Compare
|
added a new commit to fix aac panics. refactored aac aproximate_frame_count() method. It had some issues which causes panic. Should fix cases from https://github.com/qarmin/Automated-Fuzzer/actions/runs/12516290494 (REPORTS___SYMPHONIA*)
I haven't tested on VBR aac to observe any improvements, It was mention that aproximate method is not perfect there @dedobbin please review. |
sscobici
changed the title
sscobici
force-pushed
the
mp4-panic-fixes
branch
from
f970b72 to
2c2a8e4
Compare
|
fixed elst atom entry count sizes |
|
Added commit: core: advance reader's position if pattern cannot be found This addresses out-of-memory errors mentioned in the following comment: Issue #301 Comment. Issue description:
This commit ensures that the reader's position is always advanced, resolving the issue. |
|
Added commit: convert FourCc assertions to decode_error This addresses some panic errors mentioned in the following comment: #301 (comment). |
|
Added commit: mkv: report decode error on invalid vint width This addresses some panic errors mentioned in the following comment: #301 (comment). |
|
Happy new year, @sscobici! I'll be back to working on Symphonia in a few days, however, I wanted to drop in here to give you a heads up since you're touching a lot of files. I have a WIP change that rewrites MKV's EBML iterator entirely. The previous code was not fully compliant with the spec, but also had many fundamental issues. So please avoid modifying the MKV reader as any changes there will become a conflict. While working on those changes, I also realized that MP4, RIFF, and MKV are all roughly the same (i.e., nested chunk based structure) and thus all had roughly the same problems (e.g., reading out of bounds, etc.). Therefore, I'd like to apply a similar approach as the new EBML iterator to MP4's Atom iterator, and RIFF's Chunk iterator. In this PR I noticed that you've added size checks to some atom. I'm not very fond of this solution because it's not a very scalable solution, and is difficult for parent atoms. Also, it involves a lot of "magic numbers". I think the new EBML iterator provides a good solution to this problem that's more general. So, I'd also recommend avoiding making further changes like this as well. The new EBML iterator requires some additional polishing before I'm ready to merge it, but I just wanted to give you this heads up. |
sscobici
force-pushed
the
mp4-panic-fixes
branch
from
ee647d2 to
d3cd3fd
Compare
|
Added commit: aac: add validation for Ics.info.max_sfb, fixes out of range panic during decoding. |
sscobici
force-pushed
the
mp4-panic-fixes
branch
from
d3cd3fd to
8a693db
Compare
|
added commits:
|
|
Checking in since its been more than a few days. I'm almost ready to push those changes mentioned, however I still have one more issue to solve before I can do that. |
sscobici
force-pushed
the
mp4-panic-fixes
branch
from
8a693db to
d8fb220
Compare
|
Added: metadata: validate min chunk length for jpeg Feedback - error message from the new ebml iterator is very generic and it would be good if this can be improved (reader position offset, element name (path)). But this is for future. |
sscobici
force-pushed
the
mp4-panic-fixes
branch
from
d8fb220 to
4e5795f
Compare
|
added: mkv: detect parent overrun after the header parsing |
sscobici
changed the title
|
Regarding the latest commit about overrun in mkv, should this ever be allowed? What is the overall strategy for demuxer? Validation of correctness or max effort for packet and format extraction? When it should error, when it should warn and skip or try to recover? |
sscobici
force-pushed
the
mp4-panic-fixes
branch
from
4e5795f to
6788d7c
Compare
|
added: alac: validate mid_side_shift to not overflow |
|
Hey, let's start trying to get some of these fixes merged in. I think we will need to split this PR. My recommendation would be to do one PR per major component (i.e., core, riff, aac, etc.) while holding off on the MP4 changes for now. I can leave more pointed feedback on those smaller PRs.
Nice find. The EBML spec is a bit open-ended about this. However, I think this is just a special case of checking if the child element exceeds the parent element. In this case the EBML iterator just skips to the end of the parent element. However, the difference in this case is that we already consumed bytes to read the header and so we would need to rewind the reader to undo those reads. So, I have two recommendations: First, you can handle the overrun like this instead: if header.data_pos() > parent_end {
self.reader.seek_buffered(header.pos());
log::debug!("element out-of-bounds, ignoring");
return Ok(None);
}This rewinds back to the start of the header, and then ends the iteration of the parent element. Second, add a third condition here to check if Symphonia/symphonia-format-mkv/src/ebml.rs Lines 484 to 496 in 5a9bc17
The reference implementation does not clamp the value, nor does it consider it an error. Yay, undefined behaviour! However, to prevent a panic, we should clamp it to a max of 31. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Added More Validations for isomp4 to prevent some panics, Fixes #301, #300
Note: This update does not resolve all potential panic scenarios or address issues related to high memory consumption.
Requires a full regression run on as many files as possible to revalidate the constants put for validations.