30588 Commits

Author SHA1 Message Date
James Almer
ee902d3d2d x86/lossless_audiodsp: fix compilation with --disable-yasm
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: James Almer <jamrial@gmail.com>
(cherry picked from commit 383fddeec6)

Found-by: jamrial
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-12 20:04:09 +01:00
Michael Niedermayer
0f671dfeac avcodec/arm/videodsp_armv5te: Fix linking failure with "g++ -shared -D__STDC_CONSTANT_MACROS -o test.so ... libavcodec.a"
Tested-by: Andreas Haupt
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cab6302534)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-12 17:10:36 +01:00
Michael Niedermayer
345962121d avcodec/mjpegdec: Skip blocks which are outside the visible area
Fixes out of array accesses
Fixes: ffmpeg_mjpeg_crash.avi

Found-by: Thomas Lindroth <thomas.lindroth@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 08509c8f86)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-12 17:10:36 +01:00
Michael Niedermayer
0cbf53bdf5 avcodec/h264_slice: assert that reinit does not occur after the first slice
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2fd9ce92af)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-12 17:10:36 +01:00
Michael Niedermayer
b20409c690 avcodec/h264_slice: ignore SAR changes in slices after the first
Fixes race condition and null pointer dereference
Fixes: signal_sigsegv_1472ac3_468_cov_2915641226_CABACI3_Sony_B.jsv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 38d5241b7f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-12 17:10:36 +01:00
Michael Niedermayer
7997ec54c9 avcodec/h264_slice: Check picture structure before setting the related fields
This might fix a hypothetical race condition

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f111831ed6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-12 17:10:36 +01:00
Michael Niedermayer
e805826903 avcodec/h264_slice: Do not change frame_num after the first slice
Fixes potential race condition
Fixes: signal_sigsegv_1472ac3_468_cov_2915641226_CABACI3_Sony_B.jsv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f906982c94)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-12 17:10:36 +01:00
Michael Niedermayer
756d85dc14 avcodec/h264: Be more strict on rejecting pps/sps changes
Fixes race condition
Fixes: signal_sigsegv_1472ac3_468_cov_2915641226_CABACI3_Sony_B.jsv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6fafc62b0b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-12 17:10:36 +01:00
Michael Niedermayer
eeab3e1b20 avcodec/h264: Be more strict on rejecting pps_id changes
Fixes race condition
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 31cc9c04ca)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-12 17:10:36 +01:00
Michael Niedermayer
a75787a71a avcodec/h264_ps: More completely check the bit depths
Fixes out of array read
Fixes: asan_static-oob_30328b6_719_cov_3325483287_H264_artifacts_motion.h264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 69aa79365c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-12 17:10:36 +01:00
Michael Niedermayer
09425294c9 Revert "avcodec/x86/lossless_audiodsp: Make scalarproduct_and_madd_int16 prototypes more similar"
This reverts commit 3b4ffba3af.

Unbreaks the SSSE3 code on mingw32

Conflicts:

	libavcodec/x86/lossless_audiodsp.asm

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a6c2c8fe3f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-12 17:10:36 +01:00
Michael Niedermayer
3572eaaf02 avcodec/x86/lossless_audiodsp: Move order&8 fallback into C code
This is simpler and more robust, and fixes mismatching XMM save restore
mismatches

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f1214763af)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-12 17:10:36 +01:00
Michael Niedermayer
25da8d84a4 avcodec/x86/lossless_audiodsp: Make scalarproduct_and_madd_int16 prototypes more similar
This is needed as the mmx code is used as fallback from the ssse3 code

Suggested-by: jamrial
Tested-by: wm4
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3b4ffba3af)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-12 17:10:36 +01:00
Michael Niedermayer
c65a731b6f avcodec/mpegvideo_motion: Fix gmc chroma dimensions
Fixes integer overflow and out of array read
Fixes: asan_heap-oob_1fb2f9b_3780_cov_3984375136_usf.mkv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fd52d2d3d1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-12 17:10:36 +01:00
Michael Niedermayer
58096b70fa avcodec/mjpegdec: Check number of components for JPEG-LS
Fixes out of array accesses
Fixes: asan_heap-oob_1c1a4ea_1242_cov_2274415971_TESTcmyk.jpg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fabbfaa095)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-12 17:10:36 +01:00
Michael Niedermayer
0ae93844d0 avcodec/mjpegdec: Check escape sequence validity
Fixes assertion failure
Fixes: asan_heap-oob_1c1a4ea_1242_cov_2274415971_TESTcmyk.jpg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit afa92907f3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-12 17:10:36 +01:00
Rong Yan
b0b6d8de7e avcodec/ppc/idctdsp.c: POWER LE support in idct_add_altivec()
also add GET_TMP2() macro

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fc35df8931)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-12 17:10:36 +01:00
Michael Niedermayer
2f5c5767d1 avcodec/mpegvideo_enc: Fix number suffixes in rc_buffer_size calculation
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4531e2c489)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-12 17:10:36 +01:00
Michael Niedermayer
979a54ed18 avcodec/h264_cabac: use int instead of long for mbb_xy
The mb address fits in int

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 592ba6ec10)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-12 17:10:36 +01:00
Michael Niedermayer
bac6554c74 avcodec/dxtory: Use LL instead of L number suffix
This is probably unneeded and normal int would be fine, but its
safer to use LL and this isnt speed relevant

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b4ad2853c5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-12 17:10:35 +01:00
Andreas Cadhalpun
a45b8af839 libavcodec/ppc/mpegvideoencdsp.c: fix stack smashing in pix_norm1_altivec() and pix_sum_altivec()
The vec_ste calls were mistakenly changed to vec_vsx_st in c5ca76a, which
caused stack smashing.

Changing them back fixes crashes on ppc64el, when configured with
--toolchain=hardened.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 840c3c0531)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-12 17:10:35 +01:00
Vittorio Giovara
0bdc64e8b9 hevc: always clip luma_log2_weight_denom
Its value shall be between 0 and 7 according to the specifications.

Bug-Id: CID 1257502
2015-02-12 17:10:35 +01:00
Vittorio Giovara
4d74bb24e3 aacenc: correctly check returned value
(cherry picked from commit 971099ff5a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-12 17:10:35 +01:00
Michael Niedermayer
edec2a4da3 avcodec/flac_parser: fix handling EOF if no headers are found
Fixes assertion failure
Fixes Ticket4269

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c4d85fc23c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-12 17:10:35 +01:00
Vittorio Giovara
8acbba0ec3 vp8: improve memory allocation checks
Check memory earlier, check one more allocation and clean up on error.

CC: libav-stable@libav.org
Bug-Id: CID 1257773
(cherry picked from commit 014b6b416f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-12 17:10:35 +01:00
Michael Niedermayer
4f8814964c avcodec/hevc: Fix handling of skipped_bytes() reallocation failures
Fixes CID1260704

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e172f5e53a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-12 17:10:35 +01:00
wm4
8a6770a214 qpeg: avoid pointless invalid memcpy()
If refdata was NULL, the memcpy() ended up copying the same memory
block onto itself, which is not only pointless, but also undefined
behavior.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 921706691a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-02-12 17:10:35 +01:00
Carl Eugen Hoyos
763e6ecf83 lavc/aarch64: Do not use the neon horizontal chroma loop filter for H.264 4:2:2.
(cherry picked from commit 4faea46bd9)
2015-02-03 23:08:15 +01:00
wm4
3032291b3a vp9: fix parser return values in error case
The parser must always set the out_size and out_data pointers. The API
seems to require it, and the common code in parser.c also relies on it.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b88e80589b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-01-09 17:19:10 +01:00
wm4
b895e29941 avcodec/dvdsubdec: fix accessing dangling pointers
dvdsub_decode() can call append_to_cached_buf() 2 times, the second time
with ctx->buf as argument. If the second append_to_cached_buf() reallocs
ctx->buf, the argument will be a pointer to the previous, freed block.
This can cause invalid reads at least with some fuzzed files - and
possibly with valid files.

Since packets can apparently not be larger than 64K (even if packets are
combined), just use a fixed size buffer. It will be allocated as part of
the DVDSubContext, and although some memory is "wasted", it's relatively
minimal by modern standards and should be acceptable.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 816577716b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-01-09 17:19:10 +01:00
wm4
3d71024f8a avcodec/dvdsubdec: error on bitmaps with size 0
Attemtping to decode them could lead to invalid writes with some fuzzed
samples.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bcaa9099b3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-01-09 17:19:10 +01:00
wm4
e0a12b3dc3 avcodec/dvdsubdec: fix out of bounds accesses
The code blindly trusted buffer offsets read from the file in the RLE
decoder. Explicitly check the offset. Also error out on other RLE
decoding errors.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c9151de7c4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2015-01-09 17:19:10 +01:00
Michael Niedermayer
9f8cdd520b Add FFMPEG_VERSION into the binary libs
This simplifies identifying from which revision a binary of a lib came from

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 649c158e8c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-22 03:17:56 +01:00
Anton Khirnov
f5631d23e0 mmvideo: check frame dimensions
The frame size must be set by the caller and each dimension must be a
multiple of 2.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
See: 8b0e96e1f2
These should be redundant, but are backported for saftey anyway
2014-12-22 03:17:56 +01:00
Anton Khirnov
50f4543c6b jvdec: check frame dimensions
The frame size must be set by the caller and each dimension must be a
multiple of 8.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
See: 105654e376
These should be redundant, but are backported for saftey anyway
2014-12-22 03:17:56 +01:00
Michael Niedermayer
1344e91f33 avcodec/indeo3: ensure offsets are non negative
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 368642361f)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-22 03:17:55 +01:00
Michael Niedermayer
f13e6ec7a6 avcodec/h264: Check *log2_weight_denom
Fixes undefined behavior
Fixes: signal_sigsegv_14768d2_2248_cov_3629497219_h264_h264___pi_20070614T182942.h264
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 61296d41e2)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-22 03:17:55 +01:00
Michael Niedermayer
bf2c9e1ad4 avcodec/hevc_ps: Check diff_cu_qp_delta_depth
Fixes undefined behavior
Fixes: asan_static-oob_17aa046_582_cov_1577759978_DBLK_G_VIXS_1.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3281fa8925)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-22 03:17:55 +01:00
Michael Niedermayer
0663aab1d9 avcodec/h264: Clear delayed_pic on deallocation
Fixes use of freed memory

Fixes: case5_av_frame_copy_props.mp4
Found-by: Michal Zalewski <lcamtuf@coredump.cx>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e8714f6f93)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-22 03:17:55 +01:00
Michael Niedermayer
e911f125fc avcodec/hevc: clear filter_slice_edges() on allocation
This avoids use of uninitialized memory
Fixes: asan_static-oob_17aa046_582_cov_212287884_DBLK_G_VIXS_1.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8aa8d12554)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-22 03:17:55 +01:00
Michael Niedermayer
5aead5ee05 avcodec/dcadec: Check that the added xch channel isnt already there
Fixes null pointer dereference
Fixes: signal_sigsegv_369609d_623_cov_2008234281_ES_6.1_16bit.dts
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7d593495e4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-22 03:17:55 +01:00
Michael Niedermayer
3a5b749d7c avcodec/indeo3: use signed variables to avoid underflow
Fixes out of array read
Fixes: signal_sigsegv_1b0a4da_1865_cov_2167818389_computer_anger.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3305acdc92)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-22 03:17:55 +01:00
Michael Niedermayer
4b4d0b0290 avcodec/h264: make the first field of H264Context an AVClass
Fixes use of freed memory
Fixes: asan_heap-uaf_3660f67_757_cov_1257014655_Hi422FR1_SONY_A.jsv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f3b5b139ad)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-22 03:17:55 +01:00
Michael Niedermayer
3d1972d182 avcodec/utvideodec: Fix handling of slice_height=0
Fixes out of array accesses
Fixes: asan_heap-oob_25bcd7e_3783_cov_3553517262_utvideo_rgba_median.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3881606240)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-22 03:17:55 +01:00
Michael Niedermayer
71b1abe638 avcodec/xface: Add asserts to limit nb_words from becoming too large
Approved-by: Stefano Sabatini <stefasab@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 211200e0c0)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-22 03:17:55 +01:00
Michael Niedermayer
991ef3a67e avcodec/xface: correct the XFACE_MAX_* values
Fixes out of array access

Fixes: asan_stack-oob_32c12e5_2536_cov_2442316831_lena.xface
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 93a5a16f13)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-22 03:17:55 +01:00
Michael Niedermayer
b850b01533 avcodec/vmdvideo: Check len before using it in method 3
Fixes out of array access
Fixes: asan_heap-oob_4d23ba_91_cov_3853393937_128.vmd

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3030fb7e0d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-22 03:17:55 +01:00
Rong Yan
6ec5a199ea avcodec/ppc/vp3dsp_altivec: POWER LE support to vp3_idct_add_altivec()
add GET_VDST16() macro

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9bd8f2cc32)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-08 15:30:03 +01:00
Michael Niedermayer
aa24dd487f avcodec/mpegaudiodec_template: only allocate fdsp when its used
Fixes memleak

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a7ebd0b011)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-05 05:16:14 +01:00
Rong Yan
aac467ae17 avcodec/ppc/vp8dsp_altivec.c: POWER LE support put_vp8_epel_h_altivec_core() put_vp8_epel_v_altivec_core() put_vp8_pixels16_altivec() add marcos GET_PIXHL() GET_OUTER() LOAD_HL()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d23e883248)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-12-04 14:48:19 +01:00