Commit Graph

1997 Commits

Author SHA1 Message Date
Michael Niedermayer
0225aa6255 swscale/slice: clear allocated memory in alloc_lines()
Fixes: use of uninitialized memory in hScale16To15_c()
Fixes: 373924007/clusterfuzz-testcase-minimized-ffmpeg_SWS_fuzzer-5841199968092160

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit aeec39f3c1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2024-12-24 00:36:20 +01:00
Michael Niedermayer
3cdb0aa5a0 swscale/swscale: Use unsigned operation to avoid undefined behavior
I have not checked that the constant is correct, this just fixes the undefined behavior

Fixes: signed integer overflow: -646656 * 3517 cannot be represented in type 'int
Fixes: 70559/clusterfuzz-testcase-minimized-ffmpeg_SWS_fuzzer-5209368631508992

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 44c5641ae8)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2024-09-26 00:30:51 +02:00
Michael Niedermayer
d968f84435 swscale/output: Avoid undefined overflow in yuv2rgb_write_full()
Fixes: signed integer overflow: -140140 * 16525 cannot be represented in type 'int'
Fixes: 68859/clusterfuzz-testcase-minimized-ffmpeg_SWS_fuzzer-4516387130245120

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c221c7422f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2024-07-21 17:36:46 +02:00
Michael Niedermayer
4f59700877 swscale/output: alpha can become negative after scaling, use multiply
Fixes: left shift of negative value -3245
Fixes: 69047/clusterfuzz-testcase-minimized-ffmpeg_SWS_fuzzer-6571511551950848

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9e6c5b6e86)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2024-07-21 17:36:13 +02:00
Michael Niedermayer
ec2d476bb0 swscale/yuv2rgb: Use 64bit for brightness computation
This will not overflow for normal values
Fixes: CID1500280 Unintentional integer overflow

Sponsored-by: Sovereign Tech Fund
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bfc22f364d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2024-06-13 18:30:14 +02:00
Michael Niedermayer
c657698796 swscale/utils: Fix xInc overflow
Fixes: signed integer overflow: 2 * 1073741824 cannot be represented in type 'int'
Fixes: 67802/clusterfuzz-testcase-minimized-ffmpeg_SWS_fuzzer-6249515855183872

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1a9eda65d0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2024-04-04 21:10:00 +02:00
Michael Niedermayer
7d3c0fbfba libswscale/utils: Fix bayer to yuvj
Fixes: out of array access.

Earlier code assumes that a unscaled bayer to yuvj420 converter exists
but the later code then skips yuvj420

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e9cc9e492f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2024-03-26 20:05:26 +01:00
Michael Niedermayer
b4a7c55f79 swscale/swscale: Check srcSliceH for bayer
Fixes: Assertion srcSliceH > 1 failed at libswscale/swscale_unscaled.c:1359
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 64098d0cd8)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2024-03-26 18:52:32 +01:00
Michael Niedermayer
4f7d68172b swscale/utils: Allocate more dithererror
Fixes: out of array read
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 18f26f8a2f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2024-03-26 18:46:54 +01:00
Michael Niedermayer
dcc369b8f5 swscale/input: Use more unsigned intermediates
Same principle as previous commit, with sufficiently huge rgb2yuv table
values this produces wrong results and undefined behavior.
The unsigned produces the same incorrect results. That is probably
ok as these cases with huge values seem not to occur in any real
use case.

Fixes: signed integer overflow
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ba209e3d51)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2023-02-21 00:37:18 +01:00
Michael Niedermayer
764de1f6d8 swscale/slice: Fix wrong return on error
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7874d40f10)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2021-10-17 21:34:53 +02:00
Michael Niedermayer
f9d94e32e5 swscale/slice: Check slice for allocation failure
Fixes: null pointer dereference
Fixes: alloc_slice.mp4

Found-by: Rafael Dutra <rafael.dutra@cispa.de>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 997f9cfc12)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2021-10-17 21:34:53 +02:00
Michael Niedermayer
1ff19987be swscale/yuv2rgb: Fix vertical dither offset with slices
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit be3c29e379)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-04-23 21:29:01 +02:00
Michael Niedermayer
4359dddb72 swscale/output: Fix integer overflow in yuv2rgb_write_full() with out of range input
Fixes: signed integer overflow: 1169365504 + 981452800 cannot be represented in type 'int'
Fixes: ticket8293

Found-by: Suhwan
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e057e83a4f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-04-23 21:29:01 +02:00
Michael Niedermayer
23a0acd4d1 swscale/input: Fix several invalid shifts related to rgb2yuv constants
Fixes: Invalid shifts
Fixes: #8140
Fixes: #8146

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d48e510124)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-04-23 21:29:01 +02:00
Michael Niedermayer
b1e6aaa735 swscale/output: Fix several invalid shifts in yuv2rgb_full_1_c_template()
Fixes: Invalid shifts
Fixes: #8320

Reviewed-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7b7f97532b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-04-23 21:29:01 +02:00
Michael Niedermayer
f355bb59d8 swscale/swscale: Fix several invalid shifts related to vChrDrop
Fixes: Invalid shifts
Fixes: #8166
Fixes: filter-crop_scale_vflip FATE-test

Reviewed-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a6ca22c118)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2020-04-23 21:29:01 +02:00
Adam Richter
b55cf5f53d libswcale: Fix possible string overflow in test.
In libswcale/tests/swcale.c, the function fileTest() calls sscanf in
an argument of "%12s" on character srcStr[] and dstStr[], which are
only 12 bytes.  So, if the input string is 12 characters, a
terminating null byte can be written past the end of these arrays.

This bug was found by cppcheck.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b8ed493061)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2019-12-02 19:41:48 +01:00
Michael Niedermayer
42f2cf1597 libswscale/tests/swscale: Fix uninitialized variables
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7796f29065)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2017-05-20 03:41:33 +02:00
Michael Niedermayer
48c51b796a swscale/swscale_unscaled: Try to fix Rgb16ToPlanarRgb16Wrapper() with slices
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e57d99dd4e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-09-18 00:18:11 +02:00
Michael Niedermayer
26eccf4bd8 swscale/swscale_unscaled: Fix packed_16bpc_bswap() with slices
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 47bc1bdafb)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-09-18 00:18:11 +02:00
Michael Niedermayer
4ccb97650a swscale/utils: Fix chrSrcHSubSample for GBRAP16
Fixes part of Ticket5264

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 67e5bd0c50)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-02-24 00:04:51 +01:00
Michael Niedermayer
a3d698dcb1 swscale/input: Fix GBRAP16 input
Fixes part of Ticket5264

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit df36257a53)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-02-24 00:04:17 +01:00
Michael Niedermayer
21a6b7930e swscale/x86/output: Fix yuv2planeX_16* with unaligned destination
Reviewed-by: BBB
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f6492a2ea8)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-02-17 05:17:35 +01:00
Michael Niedermayer
530192b0e0 swscale/x86/output: Move code into yuv2planeX_mainloop
Reviewed-by: BBB
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d07f6e5f1c)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-02-17 05:17:18 +01:00
Michael Niedermayer
6897859b5a swscale/swscale: Add some sanity checks for srcSlice* parameters
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 321e85e176)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-28 15:53:53 +01:00
Michael Niedermayer
f121ed611e swscale/x86/rgb2rgb_template: Fix planar2x() for short width
Fixes: 451b3e0cf956c0bd2f27ed753ac24050/asan_heap-oob_2873c01_3231_7ed10a9464d15f0d57277f5917c566a8.AVI

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c8a9aaab26)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-28 15:53:53 +01:00
Michael Niedermayer
6eb76b34ca swscale/swscale_unscaled: Fix odd height inputs for bayer_to_yv12_wrapper()
Fixes: 372d2df1f04b49e25f109f07f90b1505/asan_heap-oob_2835d2e_8501_99e0114d7ba3a6db885d0b4684d200c1.cine
Fixes out of array read

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 757248ea3c)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-28 15:53:53 +01:00
Michael Niedermayer
034edcec6d swscale/swscale_unscaled: Fix odd height inputs for bayer_to_rgb24_wrapper()
Fixes: 372d2df1f04b49e25f109f07f90b1505/asan_heap-oob_2835d2e_8501_99e0114d7ba3a6db885d0b4684d200c1.cine
Fixes out of array read

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ad3b6fa7d8)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-28 15:53:53 +01:00
Michael Niedermayer
07da25548f swscale/vscale: Check that 2 tap filters are bilinear before using bilinear code
Fixes: out of array reads
Fixes: 07e8b9c5d348ccdf7add0f37de20cf6c/asan_heap-oob_27e8df7_6849_e56653f768070ec8cb52f587048444c2.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit eb7802afef)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 12:30:41 +01:00
Michael Niedermayer
be9b89314f swscale: Move VScalerContext into vscale.c
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7b6e6895f2)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 12:30:41 +01:00
Michael Niedermayer
cdac68fbab swscale/utils: Detect and skip unneeded sws_setColorspaceDetails() calls
This avoids running various table inits unnecessarily

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cc538e9dbd)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 12:30:41 +01:00
Michael Niedermayer
1fd9556062 swscale/yuv2rgb: Increase YUV2RGB table headroom
This makes SWS more robust
Fixes: 07650a772d98aa63b0fed6370dc89037/asan_heap-oob_27ddeaf_2657_2c81ff264dee5d9712cb3251fb9c3bbb.264
Fixes: out of array read

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8f3a9a8c27)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 12:30:40 +01:00
Michael Niedermayer
c781a1e14b swscale/yuv2rgb: Factor YUVRGB_TABLE_LUMA_HEADROOM out
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5e5f82a287)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 12:30:40 +01:00
Michael Niedermayer
14c63dc9b0 swscale/utils: Use normal bilinear scaler if fast cannot be used due to tiny dimensions
Fixes Ticket4960

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1edf129cbc)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 12:30:40 +01:00
Michael Niedermayer
0a6324fe93 swscale/utils: Fix intermediate format for cascaded alpha downscaling
Fixes Ticket4926

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b83d8be6bf)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-01-15 12:30:40 +01:00
Michael Niedermayer
20a48eaaf1 swscale/x86/rgb2rgb_template: Fallback to mmx in interleaveBytes() if the alignment is insufficient for SSE*
This also as a sideeffect fixes the non aligned case

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a066ff89bc)
2015-12-15 11:16:22 +01:00
Michael Niedermayer
31aeb96533 swscale/x86/rgb2rgb_template: Do not crash on misaligend stride
Fixes Ticket5013

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 80bfce35cc)
2015-12-15 11:16:10 +01:00
Michael Niedermayer
af44545613 swscale/utils: Fix for runtime error: left shift of negative value -1
Fixes: c106b36fa36db8ff8f3ed0c82be7bea2/asan_heap-oob_32699f0_6321_467b9a1d7e03d7cfd310b7e65dc53bcc.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 325b59368d)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-06 02:51:27 +01:00
Pedro Arthur
bb3b4ad460 swscale: fix ticket #4881
When scaling only a slice of a frame the output was written always
in the first lines leaving the rest of the frame black.
(cherry picked from commit 5bd62a1b3c)
2015-10-13 20:11:42 +02:00
Pedro Arthur
01bf0a178d swscale: fix ticket #4877
(cherry picked from commit a8602dde5e)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-10-08 13:15:42 +02:00
Pedro Arthur
a8d0dcbafa swscale: fix ticket 4850
(cherry picked from commit 77367f61b3)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-10-08 13:15:42 +02:00
Michael Niedermayer
9801c9524a swscale/swscale: Fix "unused variable" warning
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0ae40c5a70)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-10-08 13:15:42 +02:00
Pedro Arthur
3059562aa1 swscale: re-enable gamma
+added gamma conversion to refactored code
2015-09-04 19:00:20 -03:00
Pedro Arthur
f67aff3ad7 swscale: added proper error check for ff_init_filters 2015-09-04 19:00:20 -03:00
Michael Niedermayer
1acd6311a1 swscale/utils: If cascaded contexts are used forward sws_setColorspaceDetails() to the first context
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-08-31 02:00:55 +02:00
Michael Niedermayer
8e05f9217a swscale/utils: Split scaling if possible and yuv->yuv with different matrixes is requested
This uses a RGB intermediate, a more optimal solution would be to perform the rematrixing
directly in subsampled YUV, this is quite a bit more complicated though

Fixes Ticket4805

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-08-31 02:00:19 +02:00
Timothy Gu
68a9c8ac77 swscale: Silence an unused variable warning
Also remove a pair of extraneous ifdeffery.
2015-08-25 17:40:06 -07:00
Ganesh Ajjanagadde
eb3cfa7dd3 swscale/alphablend: fix pointer cast
Commit 7c72a4bbd3 was slightly wrong

Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
2015-08-22 20:11:48 +00:00
Michael Niedermayer
7c72a4bbd3 swscale/alphablend: Fix pointer type warnings
Found-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-08-21 12:14:08 +02:00