Michael Niedermayer
cea2106fb2
avcodec/h264_slice: More complete cleanup in h264_slice_header_init()
...
Fixes null pointer dereference
Fixes Ticket3873
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
(cherry picked from commit 1fa35e4352 )
Conflicts:
libavcodec/h264_slice.c
2015-06-17 21:50:09 +02:00
Michael Niedermayer
443b9a7d08
Merge commit '964fef3f3ced60e67831549df223bc177e1537c9' into release/2.2
...
* commit '964fef3f3ced60e67831549df223bc177e1537c9':
h264: Make sure reinit failures mark the context as not initialized
See: e8714f6f93
Merged-by: Michael Niedermayer <michaelni@gmx.at >
2015-06-01 13:56:18 +02:00
Luca Barbato
964fef3f3c
h264: Make sure reinit failures mark the context as not initialized
...
Bug-Id: CVE-2015-3417
CC: libav-stable@libav.org
(cherry picked from commit 3b69f245db )
Signed-off-by: Reinhard Tartler <siretart@tauware.de >
Conflicts:
libavcodec/h264_slice.c
2015-05-31 11:35:35 -04:00
Michael Niedermayer
82d3dd44aa
avcodec/h264: Be more tolerant to changing pps id between slices
...
Fixes Ticket4446
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
(cherry picked from commit 98d0c4236c )
Conflicts:
libavcodec/h264.c
(cherry picked from commit 0cd0fa9d0baabd2dc0442ed8b53ba65282733b61)
Conflicts:
libavcodec/h264.c
2015-04-16 16:03:24 +02:00
Michael Niedermayer
2b69da7b5a
avcodec/h264: reset the counts in the correct context
...
Fixes null pointer dereference
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
(cherry picked from commit 8f8d632220100bfde26587b27da73901b05cb774)
Conflicts:
libavcodec/h264.c
2015-04-16 16:03:24 +02:00
Michael Niedermayer
0df90898f5
avcodec/h264_slice: Dont reset mb_aff_frame per slice
...
Fixes null pointer dereference
Fixes Ticket4440
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
(cherry picked from commit 386601286f )
Conflicts:
libavcodec/h264_slice.c
(cherry picked from commit ce6d38e9ed0842870f3cd5414937bb6d1f2417d9)
Conflicts:
libavcodec/h264_slice.c
2015-04-16 16:03:24 +02:00
Michael Niedermayer
8f026e2b38
avcodec/h264: finish previous slices before switching to single thread mode
...
Fixes null pointer dereference
Fixes Ticket4438
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
(cherry picked from commit c4b2017ba6 )
Conflicts:
libavcodec/h264.c
(cherry picked from commit 09cc7aee3f4d1bd1d7107d38520f782c62c14036)
Conflicts:
libavcodec/h264.c
2015-04-07 12:43:06 +02:00
Michael Niedermayer
3852b172e7
avcodec/h264: Fix race between slices where one overwrites data from the next
...
Fixes non deterministic crash in ticket4408/fuzz2.264
Likely fixes other samples as well
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
(cherry picked from commit 43b434210e )
Conflicts:
libavcodec/h264.h
libavcodec/h264_slice.c
(cherry picked from commit dbbc42858e87cdd04e6c3b7694f8b394d4bfcdc6)
Conflicts:
libavcodec/h264_slice.c
2015-04-07 12:43:06 +02:00
Michael Niedermayer
5b4e58ed4c
avcodec/h264: Fail for invalid mixed IDR / non IDR frames in slice threading mode
...
Fixes Ticket4408
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
(cherry picked from commit fc58d5c43b )
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2015-04-07 12:43:06 +02:00
Michael Niedermayer
ec7c1cd733
avcodec/h264: Only reinit quant tables if a new PPS is allowed
...
Fixes null pointer dereference
Fixes: signal_sigsegv_3042097_3007_cov_1741463594_non_monotone_timestamps1.mkv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
(cherry picked from commit c23a0e77dd )
Conflicts:
libavcodec/h264_slice.c
2015-03-14 14:27:23 +01:00
Michael Niedermayer
019b4b0650
Merge commit '3670942fae7beb2bfde52557ee95eab5f536e624' into release/2.2
...
* commit '3670942fae7beb2bfde52557ee95eab5f536e624':
h264: initialize H264Context.avctx in init_thread_copy
Merged-by: Michael Niedermayer <michaelni@gmx.at >
2015-03-09 12:24:50 +01:00
Michael Niedermayer
e3654b25cf
Merge commit 'fa4604d80580dde45bfce32ebe04a5c13c233895' into release/2.2
...
* commit 'fa4604d80580dde45bfce32ebe04a5c13c233895':
h264: Do not share rbsp_buffer across threads
Conflicts:
libavcodec/h264.c
See: ecbf838c7d
Merged-by: Michael Niedermayer <michaelni@gmx.at >
2015-03-09 12:24:04 +01:00
Michael Niedermayer
b837d5bf83
Merge commit '03fbb6ff3d28f639ea5a35aba3c6dca09c17225d' into release/2.2
...
* commit '03fbb6ff3d28f639ea5a35aba3c6dca09c17225d':
h264: only ref cur_pic in update_thread_context if it is initialized
Conflicts:
libavcodec/h264.c
See: 0fc01ae33c
Merged-by: Michael Niedermayer <michaelni@gmx.at >
2015-03-09 12:11:40 +01:00
Anton Khirnov
3670942fae
h264: initialize H264Context.avctx in init_thread_copy
...
This prevents using a wrong (first thread's) AVCodecContext if decoding
a frame in the first pass over all threads fails.
(cherry picked from commit a06b0b1295 )
Signed-off-by: Anton Khirnov <anton@khirnov.net >
(cherry picked from commit 2686dab45e )
Signed-off-by: Anton Khirnov <anton@khirnov.net >
2015-02-21 09:41:30 +01:00
Michael Niedermayer
fa4604d805
h264: Do not share rbsp_buffer across threads
...
Signed-off-by: Luca Barbato <lu_zero@gentoo.org >
CC: libav-stable@libav.org
(cherry picked from commit 61928b68dc )
Signed-off-by: Anton Khirnov <anton@khirnov.net >
(cherry picked from commit 06d433366c )
Signed-off-by: Anton Khirnov <anton@khirnov.net >
2015-02-21 09:41:24 +01:00
Anton Khirnov
03fbb6ff3d
h264: only ref cur_pic in update_thread_context if it is initialized
...
It may be empty if the previous thread's decode call did not contain a
valid frame.
(cherry picked from commit 0dea4c77cc )
Signed-off-by: Anton Khirnov <anton@khirnov.net >
(cherry picked from commit 1dbfaa34e6 )
Signed-off-by: Anton Khirnov <anton@khirnov.net >
Conflicts:
libavcodec/h264_slice.c
2015-02-21 09:40:43 +01:00
Michael Niedermayer
f0526bc21e
avcodec/h264_slice: ignore SAR changes in slices after the first
...
Fixes race condition and null pointer dereference
Fixes: signal_sigsegv_1472ac3_468_cov_2915641226_CABACI3_Sony_B.jsv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
(cherry picked from commit 38d5241b7f )
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
Conflicts:
libavcodec/h264_slice.c
2015-02-17 19:43:19 +01:00
Michael Niedermayer
0afe061f28
avcodec/h264_slice: Check picture structure before setting the related fields
...
This might fix a hypothetical race condition
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
(cherry picked from commit f111831ed6 )
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
Conflicts:
libavcodec/h264_slice.c
2015-02-17 19:43:19 +01:00
Michael Niedermayer
e6093f5b85
avcodec/h264_slice: Do not change frame_num after the first slice
...
Fixes potential race condition
Fixes: signal_sigsegv_1472ac3_468_cov_2915641226_CABACI3_Sony_B.jsv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
(cherry picked from commit f906982c94 )
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
Conflicts:
libavcodec/h264_slice.c
2015-02-17 19:43:19 +01:00
Michael Niedermayer
0c9d465e98
avcodec/h264: Be more strict on rejecting pps/sps changes
...
Fixes race condition
Fixes: signal_sigsegv_1472ac3_468_cov_2915641226_CABACI3_Sony_B.jsv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
(cherry picked from commit 6fafc62b0b )
Conflicts:
libavcodec/h264_slice.c
2015-02-17 19:43:18 +01:00
Michael Niedermayer
a3dca10470
avcodec/h264: Be more strict on rejecting pps_id changes
...
Fixes race condition
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
(cherry picked from commit 31cc9c04ca )
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2015-02-17 19:43:18 +01:00
Michael Niedermayer
f1d59a207f
avcodec/h264: Check *log2_weight_denom
...
Fixes undefined behavior
Fixes: signal_sigsegv_14768d2_2248_cov_3629497219_h264_h264___pi_20070614T182942.h264
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
(cherry picked from commit 61296d41e2 )
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2015-01-20 03:27:17 +01:00
Michael Niedermayer
25dc978bb1
avcodec/h264: Clear delayed_pic on deallocation
...
Fixes use of freed memory
Fixes: case5_av_frame_copy_props.mp4
Found-by: Michal Zalewski <lcamtuf@coredump.cx >
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
(cherry picked from commit e8714f6f93 )
Conflicts:
libavcodec/h264.c
2015-01-20 03:27:17 +01:00
Michael Niedermayer
8b8d794800
avcodec/h264_slice: Clear table pointers to avoid stale pointers
...
Might fix Ticket3889
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
(cherry picked from commit 547fce9585 )
Conflicts:
libavcodec/h264_slice.c
2014-12-08 19:08:30 +01:00
Michael Niedermayer
2d1d053c5d
Merge commit 'c7caed88a03567e8777a606f4bd42f093c6b302c' into release/2.2
...
* commit 'c7caed88a03567e8777a606f4bd42f093c6b302c':
h264: Always invoke the get_format() callback
Conflicts:
libavcodec/h264.c
Merged-by: Michael Niedermayer <michaelni@gmx.at >
2014-11-01 15:08:51 +01:00
Michael Niedermayer
26da47a09b
avcodec/h264: Check mode before considering mixed mode intra prediction
...
Fixes out of array read
Fixes: asan_heap-oob_e476fc_2_asan_heap-oob_1333ec6_61_CAMACI3_Sony_C.jsv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
(cherry picked from commit 9734a7a1de )
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-10-05 14:22:09 +02:00
Rémi Denis-Courmont
c7caed88a0
h264: Always invoke the get_format() callback
...
Signed-off-by: Luca Barbato <lu_zero@gentoo.org >
2014-09-27 15:49:06 +02:00
Michael Niedermayer
9e1ce9a8ee
avcodec/h264: Allow partial escaping
...
Fixes Ticket3923
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
(cherry picked from commit 033a5334ba )
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-09-20 18:40:33 +02:00
Michael Niedermayer
0dc5868f14
Merge commit '67134ad31f1f3bc1515eae129e4368401f7c3342' into release/2.2
...
* commit '67134ad31f1f3bc1515eae129e4368401f7c3342':
h264: fix interpretation of interleaved stereo modes
Merged-by: Michael Niedermayer <michaelni@gmx.at >
2014-08-11 18:51:29 +02:00
Felix Abecassis
67134ad31f
h264: fix interpretation of interleaved stereo modes
...
Column and row frame packing arrangements were inverted.
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com >
2014-08-07 11:40:08 +01:00
Michael Niedermayer
989adf5ee5
avcodec/h264: in the absence of recovery points, be more tolerant on accepting plain I frames
...
Fixes: Ticket3652
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
(cherry picked from commit 19c9d1e8e7 )
2014-06-09 23:50:12 +02:00
Michael Niedermayer
02bae9f013
avcodec/h264: clear cur_pic structure instead of duplicating it in ff_h264_update_thread_context()
...
Fixes crash
Found-by: iive
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
(cherry picked from commit 8710ee11d7 )
Conflicts:
libavcodec/h264_slice.c
2014-04-10 04:53:00 +02:00
Michael Niedermayer
d8fe695779
avcodec/h264: be more tolerant on what pixel format changes trigger reinits
...
Fixes Ticket3260
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
(cherry picked from commit 8e92ff2546 )
2014-03-17 15:45:36 +01:00
Luca Barbato
de187e3e9e
h264: Fix a typo from the previous commit
...
f777504f64 changed a - in +
CC: libav-stable@libav.org
(cherry picked from commit d922c5a5fb )
2014-03-02 11:42:36 -05:00
Vittorio Giovara
63169474b3
h264: Lower bound check for slice offsets
...
And use the value from the specification.
Sample-Id: 00000451-google
Found-by: Mateusz j00ru Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Luca Barbato <lu_zero@gentoo.org >
(cherry picked from commit f777504f64 )
2014-03-02 11:42:36 -05:00
Michael Niedermayer
5c634cbeb7
Merge remote-tracking branch 'qatar/master'
...
* qatar/master:
Give IDCT matrix transpose macro a more descriptive name
Merged-by: Michael Niedermayer <michaelni@gmx.at >
2014-02-28 02:08:11 +01:00
Diego Biurrun
f2408ec9d7
Give IDCT matrix transpose macro a more descriptive name
...
This also avoids a macro name clash and related warning on ARM.
2014-02-27 13:38:00 -08:00
Michael Niedermayer
4f4cc43fd8
avcodec/h264: allow mixing idr and non idr slices with frame threading again
...
This combination exists in the wild
Fixes Ticket3131
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-02-27 15:49:25 +01:00
Michael Niedermayer
64bb64f704
avcodec/h264: fix droped frame handling also for threads > 1
...
Seems i mistakely tested just with threads=1
Fixes part of Ticket3386
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-02-27 08:07:46 +01:00
Michael Niedermayer
b5005def8a
avcodec/h264: avoid using lost frames as references
...
Fixes Ticket3386
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-02-27 03:26:03 +01:00
Michael Niedermayer
72e6913140
avcodec/h264: clear chroma planes when flags gray is used
...
Fixes Ticket3397
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-02-23 18:22:13 +01:00
Luca Barbato
d922c5a5fb
h264: Fix a typo from the previous commit
...
f777504f64 changed a - in +
CC: libav-stable@libav.org
2014-02-22 12:26:32 +01:00
Michael Niedermayer
8c55ff3933
avcodec/h264: use subsample factors of the used pixel format
...
Fixes out of array read
Fixes: 1cb91c36c4e55463f14aacb9bdf55b38-asan_heap-oob_106cbce_5617_cov_11212800_h264_mmx_chroma_intra_lf.mp4
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-02-21 23:53:49 +01:00
Michael Niedermayer
76dd01ecd4
avcodec/h264: fix sign error
...
regression since f777504f64
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-02-21 00:33:57 +01:00
Michael Niedermayer
de7b50e9cd
Merge remote-tracking branch 'qatar/master'
...
* qatar/master:
h264: Lower bound check for slice offsets
Conflicts:
libavcodec/h264.c
See: 91253839e1
Merged-by: Michael Niedermayer <michaelni@gmx.at >
2014-02-21 00:20:55 +01:00
Vittorio Giovara
f777504f64
h264: Lower bound check for slice offsets
...
And use the value from the specification.
Sample-Id: 00000451-google
Found-by: Mateusz j00ru Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Luca Barbato <lu_zero@gentoo.org >
2014-02-20 18:58:38 +01:00
Michael Niedermayer
d0e236292d
Merge remote-tracking branch 'qatar/master'
...
* qatar/master:
h264: informative error reporting in decode_slice_header()
Merged-by: Michael Niedermayer <michaelni@gmx.at >
2014-02-19 02:25:02 +01:00
Luca Barbato
fea6db064b
h264: informative error reporting in decode_slice_header()
...
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com >
2014-02-18 23:47:55 +01:00
Luca Barbato
96f9fbe109
h264: fix slice_type value reported in decode_slice_header()
...
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com >
2014-02-18 23:47:32 +01:00
Michael Niedermayer
91253839e1
avcodec/h264: more completely check the loop filter parameters
...
Fixes out of array read
Fixes: caa65cc01655505705129b677189f036-signal_sigsegv_fdcc43_2681_cov_3043376737_PPH422I5_Panasonic_A.264
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-02-17 01:29:34 +01:00