The implicit checks via v_data_size and a_data_size don't work in the case
'(hdr_size > 7) && !ctx->alpha_info'.
This fixes segmentation faults due to invalid reads.
This problem was introduced in commit
547c2f002a.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 1e33035ee7)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This was broken by the following Libav commit:
4c387c7 ppc: dsputil: do unaligned block accesses correctly
The following tests fail due to this:
fate-checkasm
fate-vsynth1-dnxhd-2k-hr-hq fate-vsynth1-dnxhd-edge1-hr
fate-vsynth1-dnxhd-edge2-hr fate-vsynth1-dnxhd-edge3-hr
fate-vsynth1-dnxhd-hr-sq-mov fate-vsynth1-dnxhd-hr-hq-mov
fate-vsynth2-dnxhd-2k-hr-hq fate-vsynth2-dnxhd-edge1-hr
fate-vsynth2-dnxhd-edge2-hr fate-vsynth2-dnxhd-edge3-hr
fate-vsynth2-dnxhd-hr-sq-mov fate-vsynth2-dnxhd-hr-hq-mov
fate-vsynth3-dnxhd-2k-hr-hq fate-vsynth3-dnxhd-edge1-hr
fate-vsynth3-dnxhd-edge2-hr fate-vsynth3-dnxhd-edge3-hr
fate-vsynth3-dnxhd-hr-sq-mov fate-vsynth3-dnxhd-hr-hq-mov
Fixes trac ticket #5508.
Reviewed-by: Carl Eugen Hoyos <ceffmpeg@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 3932ccc472)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If realloc fails, the pointer is overwritten and the previously allocated
buffer is leaked, which goes against the expected behavior of keeping the
packet unchanged in case of error.
Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 574929d8b6)
This function must be called from the mb or slice encoding loop and MMX state may not
be clean there
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 03ec6b780c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The calculation of width/height should round up, not round down to
prevent setting width or height to 0.
Also image->comps[compno].w is unsigned (at least in openjpeg2), so the
calculation could silently wrap around without the explicit cast to int.
Reviewed-by: Michael Bradshaw <mjbshaw@gmail.com>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 56706ac0d5)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Use check_lib2 to test the header together with the function. This is
necessary, because '-DOPJ_STATIC' changes what the included header does.
Also add '-DOPJ_STATIC' to CPPFLAGS, so that it isn't necessary to
hardcode this in libavcodec/libopenjpeg{dec,enc}.c.
Finally, check for non-static openjpeg 2.1, too.
Reviewed-by: Michael Bradshaw <mjbshaw@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 7a65aef00d)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This is simpler and fixes an out of array read, fixing it with AVBuffers
would be more complex
Fixes: e00d9e6e50e5495cc93fea41147b97bb/asan_heap-oob_12dcdbb_8798_b32a97ea722dd37bb5066812cc674552.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 752e6dfa3e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: 1642cd3962249d6aaf0eec2836023fb6/signal_sigsegv_2557a72_2995_04efaf2ff57a052f609a3b4a2ea4e622.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2d3099ad8e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This also disabled the case of mixed x/ymin with tiles, the code
handles these cases inconsistent for the 2 coordinate axis and is
unlikely working correctly.
Fixes crash
Fixes: poc1.exr, poc2.exr
Found-by: Yaoguang Chen of Aliapy unLimit Security Team
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 01aee8148d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes occurance of NaN/Inf leading to assertion failures and out of array access
Fixes: d1c38a09acc34845c6be3a127a5aacaf/signal_sigsegv_3982225_6121_d18bd5451d4245ee09408f04badd1b83.wmv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 77bf96b047)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: f29f134ea5f5590df554a7733294a587/asan_stack-oob_309d14e_9188_ea01743d6355aff20530f3d4cdaa841a.wav
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f2a9a30fd6)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: 09f46aa2175cade93e3e3932646a56a9/asan_heap-oob_4a5385_2995_498f6abfdc0248288cefe5f4b7ad316c.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2624695484)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Reading it from any other position would result in a wrong size being
read, instead fallback to the re-sync mechanic in the else clause.
(cherry picked from commit c3e9b098e1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
We know the maximum size of an AnnexB NAL, signaling it as the maximum
NAL size allows ff_h2645_extract_rbsp to determine the correct size.
(cherry picked from commit 83a940e7fb)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
