Commit Graph

80931 Commits

Author SHA1 Message Date
Andreas Cadhalpun
e70caba384 libopusdec: default to stereo for invalid number of channels
This fixes an out-of-bounds read if avc->channels is 0.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 8c8f543b81)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:08 +01:00
Andreas Cadhalpun
d0f8741a5a flvdec: require need_context_update when changing codec id
Otherwise the codec context and codecpar might disagree on the codec id,
triggering asserts in av_parser_parse2.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 98b3a7979f)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:07 +01:00
Andreas Cadhalpun
9b506280dd pgssubdec: only set w/h/linesize when allocating data
Rects with positive w/h/linesize but no data are invalid.

Reviewed-by: Petri Hintukainen <phintuka@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 995512328e)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:07 +01:00
Andreas Cadhalpun
312757eb84 sbgdec: prevent NULL pointer access
Reviewed-by: Josh de Kock <josh@itanimul.li>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit dbefbb61b7)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:07 +01:00
Andreas Cadhalpun
e2de6f31c0 rmdec: validate block alignment
This fixes division by zero crashes.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit de4ded0636)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:07 +01:00
Andreas Cadhalpun
53e1493cb5 smacker: limit recursion depth of smacker_decode_bigtree
This fixes segmentation faults due to stack-overflow caused by too deep
recursion.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 946ecd19ea)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:07 +01:00
Andreas Cadhalpun
315f1dea84 mxfdec: fix NULL pointer dereference in mxf_read_packet_old
Metadata streams have priv_data set to NULL.

Reviewed-by: Josh de Kock <josh@itanimul.li>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit fdb8c455b6)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:07 +01:00
Andreas Cadhalpun
b4f42e5c85 ffmdec: validate codec parameters
A negative extradata size for example gets passed to memcpy in
avcodec_parameters_from_context causing a segmentation fault.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 1c7da19a4b)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:07 +01:00
Andreas Cadhalpun
cb936d6266 exr: reindent after previous commit
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit ce3147eb19)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:06 +01:00
Andreas Cadhalpun
71378e7937 exr: fix out-of-bounds read
channel_index can be -1.

This problem was introduced in commit
2dd7b46132.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit ffdc5d09e4)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:06 +01:00
Andreas Cadhalpun
f70e9726dc libschroedingerdec: fix leaking of framewithpts
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 3c0328d58d)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:06 +01:00
Andreas Cadhalpun
89a22d3fbf libschroedingerdec: don't produce empty frames
They are not valid and can cause problems/crashes for API users.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit a86ebbf7f6)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:06 +01:00
Andreas Cadhalpun
d000e66c4f softfloat: handle -INT_MAX correctly
This is similar to commit 9ac61e73d0.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 0edd569466)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:06 +01:00
Andreas Cadhalpun
52d8c1e474 filmstripdec: correctly check image dimensions
This prevents a division by zero in read_packet.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 25012c5644)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:06 +01:00
Andreas Cadhalpun
a5ba9eab44 pnmdec: make sure v is capped by maxval
Otherwise put_bits can be called with a value that doesn't fit in the
sample_len, causing an assertion failure.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit cdb5479c9d)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:06 +01:00
Andreas Cadhalpun
eaf79ac2d9 smvjpegdec: make sure cur_frame is not negative
This fixes a heap-buffer-overflow detected by AddressSanitizer.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 360bc0d90a)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:05 +01:00
Andreas Cadhalpun
c35a140e71 icodec: correctly check avio_read return value
It can read less than the requested amount, in which case buf contains
uninitialized data, causing problems like segmentation faults later on.

Also make sure that image->size is positive, so that it can't match a
negative error code.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 89eb398c7f)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:05 +01:00
Andreas Cadhalpun
5c2e26275c dvbsubdec: fix division by zero in compute_default_clut
This problem was introduced in commit
4b90dcb849.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit c82b8ef0e4)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:05 +01:00
Andreas Cadhalpun
727ec4acc4 proresdec_lgpl: explicitly check coff[3] against slice_data_size
The implicit checks via v_data_size and a_data_size don't work in the case
'(hdr_size > 7) && !ctx->alpha_info'.

This fixes segmentation faults due to invalid reads.

This problem was introduced in commit
547c2f002a.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 1e33035ee7)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:05 +01:00
Andreas Cadhalpun
1499f65ad4 escape124: reject codebook size 0
It causes a cb_depth of 32, leading to assertion failures in get_bits.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 226d35c845)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:05 +01:00
Andreas Cadhalpun
6a7f0585ab icodec: add ico_read_close to fix leaking ico->images
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit d54c95a143)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:05 +01:00
Andreas Cadhalpun
356e035773 icodec: fix leaking pkt on error
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 467eece1be)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:05 +01:00
Andreas Cadhalpun
e1c1cb4aa1 mpegts: prevent division by zero
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 1bbb18fe82)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:05 +01:00
Andreas Cadhalpun
c19e965704 matroskadec: fix NULL pointer dereference in webm_dash_manifest_read_header
The code assumes that s->streams[0] is valid.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit ff100c9dd9)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:04 +01:00
Andreas Cadhalpun
a401893487 mpegaudio_parser: don't return AVERROR_PATCHWELCOME
The API does not allow returning AVERROR codes.

It triggers an assert in av_parser_parse2.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 5249706e9d)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:04 +01:00
Andreas Cadhalpun
50d34cbf5a mxfdec: fix NULL pointer dereference
Metadata streams have priv_data set to NULL.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 0efb610611)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:04 +01:00
Andreas Cadhalpun
1af13ea539 lzf: update pointer p after realloc
This fixes heap-use-after-free detected by AddressSanitizer.

Reviewed-by: Luca Barbato <lu_zero@gentoo.org>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit bb6a7b6f75)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:04 +01:00
Andreas Cadhalpun
cb0b818244 diracdec: check return code of get_buffer_with_edge
If it fails, buffers aren't allocated, causing NULL pointer dereferencing.

Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit db79dedb1a)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:04 +01:00
Andreas Cadhalpun
e3f671b101 ppc: pixblockdsp: do unaligned block accesses correctly again
This was broken by the following Libav commit:
4c387c7 ppc: dsputil: do unaligned block accesses correctly

The following tests fail due to this:
fate-checkasm
fate-vsynth1-dnxhd-2k-hr-hq fate-vsynth1-dnxhd-edge1-hr
fate-vsynth1-dnxhd-edge2-hr fate-vsynth1-dnxhd-edge3-hr
fate-vsynth1-dnxhd-hr-sq-mov fate-vsynth1-dnxhd-hr-hq-mov
fate-vsynth2-dnxhd-2k-hr-hq fate-vsynth2-dnxhd-edge1-hr
fate-vsynth2-dnxhd-edge2-hr fate-vsynth2-dnxhd-edge3-hr
fate-vsynth2-dnxhd-hr-sq-mov fate-vsynth2-dnxhd-hr-hq-mov
fate-vsynth3-dnxhd-2k-hr-hq fate-vsynth3-dnxhd-edge1-hr
fate-vsynth3-dnxhd-edge2-hr fate-vsynth3-dnxhd-edge3-hr
fate-vsynth3-dnxhd-hr-sq-mov fate-vsynth3-dnxhd-hr-hq-mov

Fixes trac ticket #5508.

Reviewed-by: Carl Eugen Hoyos <ceffmpeg@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 3932ccc472)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:04 +01:00
Andreas Cadhalpun
5a1433b19a interplayacm: increase bitstream buffer size by AV_INPUT_BUFFER_PADDING_SIZE
This fixes out-of-bounds reads by the bitstream reader.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 60178e78f2)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:03 +01:00
Andreas Cadhalpun
d6fbc7a2da interplayacm: validate number of channels
The number of channels is used as divisor in decode_frame, so it must
not be zero to avoid SIGFPE crashes.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 5540d6c134)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:03 +01:00
Andreas Cadhalpun
5ede8a9d8c interplayacm: check for too large b
This fixes out-of-bounds reads.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 14e4e26559)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:03 +01:00
Andreas Cadhalpun
facf964d37 mpeg12dec: unref discarded picture from extradata
Otherwise another frame gets referenced into picture, triggering an assert
(from commit 13aae8) in av_frame_ref.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit a92f8edf0c)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:03 +01:00
Andreas Cadhalpun
72f1701c92 cavsdec: unref frame before referencing again
This fixes asserts (from commit 13aae8) in av_frame_ref and
av_frame_move_ref.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 1966ea012f)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:03 +01:00
Andreas Cadhalpun
d77684b853 dcstr: fix division by zero
Also check for possible overflows.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit b0a043f51b)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:03 +01:00
Andreas Cadhalpun
2c52b74980 aiff: check block_align in aiff_read_packet
It can be unset in avcodec_parameters_from_context and a value of 0
causes SIGFPE crashes.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 93c39db5f1)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:03 +01:00
Andreas Cadhalpun
13f032abbb rsd: limit number of channels
Negative values don't make sense and too large values can cause
overflows. For AV_CODEC_ID_ADPCM_THP this leads to a too small extradata
buffer being allocated, causing out-of-bounds writes.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit ee5f0f1d35)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:02 +01:00
Andreas Cadhalpun
d69dc10466 avformat: prevent triggering request_probe assert in ff_read_packet
If probe_codec is called with pkt == NULL, it sets probe_packets to 0
and request_probe to -1.
However, request_probe can change when calling s->iformat->read_packet
and thus a probe_packets value of 0 doesn't guarantee a request_probe
value of -1.
In that case calling probe_codec again is necessary to prevent
triggering the assert.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit a5b4476a60)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:02 +01:00
Andreas Cadhalpun
d4f64a0f54 westwood_aud: prevent division by zero
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit bc7e128a6e)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:02 +01:00
Andreas Cadhalpun
b3991ccd11 astdec: fix division by zero
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 9959a52b14)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:28:02 +01:00
Andreas Cadhalpun
230c04e3f6 aiffdec: fix division by zero
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit c143a9c96f)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-11-27 00:27:56 +01:00
James Almer
c3f97bf544 avcodec/avpacket: fix leak on realloc in av_packet_add_side_data()
If realloc fails, the pointer is overwritten and the previously allocated
buffer is leaked, which goes against the expected behavior of keeping the
packet unchanged in case of error.

Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>

(cherry picked from commit 574929d8b6)
2016-11-19 20:24:44 -03:00
Michael Niedermayer
2a5c41e3e4 Chagelog: update
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
n3.1.5
2016-10-22 01:37:37 +02:00
Michael Niedermayer
9e6586ceb2 avformat/mxfdec: Check size to avoid integer overflow in mxf_read_utf16_string()
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit fecb3e82a4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-10-21 20:26:00 +02:00
Michael Niedermayer
6456a7416e avcodec/mpegvideo_enc: Clear mmx state in ff_mpv_reallocate_putbitbuffer()
This function must be called from the mb or slice encoding loop and MMX state may not
be clean there

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 03ec6b780c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-10-21 19:33:04 +02:00
Michael Niedermayer
de487cb765 avcodec/utils: Clear MMX state before returning from avcodec_default_execute*()
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4f96f9d111)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-10-21 19:33:04 +02:00
Michael Niedermayer
2fece989f8 doc/examples/demuxing_decoding: Drop AVFrame->pts use
This code is not correct for git master

Reviewed-by: Stefano Sabatini <stefasab@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2bd9956454)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2016-10-21 19:33:04 +02:00
Andreas Cadhalpun
a2d3e7392d Changelog: update for recent commits
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-10-17 18:13:44 +02:00
Andreas Cadhalpun
d391719be1 libopenjpegenc: fix out-of-bounds reads when filling the edges
The calculation of width/height should round up, not round down to
prevent setting width or height to 0.

Also image->comps[compno].w is unsigned (at least in openjpeg2), so the
calculation could silently wrap around without the explicit cast to int.

Reviewed-by: Michael Bradshaw <mjbshaw@gmail.com>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 56706ac0d5)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-10-17 17:21:35 +02:00
Andreas Cadhalpun
a22155dacd libopenjpegenc: stop reusing image data buffer for openjpeg 2
openjpeg 2 sets the data pointers of the image components to NULL,
causing segfaults if the image is reused.

Reviewed-by: Michael Bradshaw <mjbshaw@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 69c8505f3b)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
2016-10-17 17:21:30 +02:00