swscale/output: Use 64bit in addition in yuv2gbrp16_full_X_c() for RGB + Y

Fixes: signed integer overflow: -1159988356 + -1082982400 cannot be represented in type 'int'
Fixes: 461519938/clusterfuzz-testcase-minimized-ffmpeg_SWS_fuzzer-4777382021234688

Found-by:  continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This commit is contained in:
Michael Niedermayer
2025-12-22 23:44:54 +01:00
committed by michaelni
parent 09d8300779
commit d41b8d95a7

View File

@@ -2513,9 +2513,9 @@ yuv2gbrp16_full_X_c(SwsInternal *c, const int16_t *lumFilter,
G = V * c->yuv2rgb_v2g_coeff + U * c->yuv2rgb_u2g_coeff;
B = U * c->yuv2rgb_u2b_coeff;
dest16[2][i] = av_clip_uintp2(((Y + R) >> 14) + (1<<15), 16);
dest16[0][i] = av_clip_uintp2(((Y + G) >> 14) + (1<<15), 16);
dest16[1][i] = av_clip_uintp2(((Y + B) >> 14) + (1<<15), 16);
dest16[2][i] = av_clip_uintp2(((Y + (int64_t)R) >> 14) + (1<<15), 16);
dest16[0][i] = av_clip_uintp2(((Y + (int64_t)G) >> 14) + (1<<15), 16);
dest16[1][i] = av_clip_uintp2(((Y + (int64_t)B) >> 14) + (1<<15), 16);
if (hasAlpha)
dest16[3][i] = av_clip_uintp2(A, 30) >> 14;