summaryrefslogtreecommitdiff
path: root/libavcodec
Commit message (Collapse)AuthorAge
* avcodec/videotoolboxenc: fix undefined behavior with rc_max_rate=0Thomas Guillem2018-07-19
| | | | | | | | | | | On macOS, a zero rc_max_rate cause an error from VTSessionSetProperty(kVTCompressionPropertyKey_DataRateLimits). on iOS (depending on device/version), a zero rc_max_rate cause invalid arguments from the vtenc_output_callback after few frames and then a crash within the VideoToolbox library. Signed-off-by: Aman Gupta <aman@tmm1.net>
* avcodec: parse options from AVCodec.bsfsAman Gupta2018-07-19
| | | | | | | | | Fixes a bug that would prevent using multiple comma-separated filters, and allows options to be passed to each filter. Based on similar loop in ffmpeg_opt.c's new_output_stream(). Signed-off-by: Aman Gupta <aman@tmm1.net>
* avcodec/dvbsub_parser: Do not discard dataMichael Niedermayer2018-07-18
| | | | | | May fix: oss fuzz issue 9220 Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/cfhd: Use bytestream2 for peaksMichael Niedermayer2018-07-18
| | | | | | | This fixes out of array accesses No testcase known Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/tscc: Move reget buffer to after decodingMichael Niedermayer2018-07-18
| | | | | | | | | | reget buffer can be time consuming, theres no need to do it if failures occur Testcase: 9010/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TSCC_fuzzer-6042614817095680 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/lagarith: Remove redundant varianble initMichael Niedermayer2018-07-18
| | | | Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/dvdsub_parser: Allocate input paddingMichael Niedermayer2018-07-15
| | | | | | | | Fixes: out of array read Fixes: 9350/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DVDSUB_fuzzer-5746777750765568 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/vp8_parser: Remove redundant output initializationMichael Niedermayer2018-07-15
| | | | Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/dvdsub_parser: Init output buf/sizeMichael Niedermayer2018-07-15
| | | | | | No testcase Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/lagarith: Check that the range coded data stream is consistent when ↵Michael Niedermayer2018-07-15
| | | | | | | | | | | the probabilities indicate no data could have been coded. Fixes: Timeout Fixes: 8638/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_LAGARITH_fuzzer-5132046098759680 Fixes: 8943/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_LAGARITH_fuzzer-4883030219948032 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* libavcodec/cfhd: Fix signed overflow in shiftMichael Niedermayer2018-07-15
| | | | | | | Fixes: 8695/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_CFHD_fuzzer-4906172426485760 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/dirac_dwt_template: Fix signedness regression in interleave()Michael Niedermayer2018-07-14
| | | | | | Found-by: <jdarnley> Tested-by: James Darnley <james.darnley@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/mips: fix conflicting types error of ff_vc1_h_s_overlap_mmi.Shiyou Yin2018-07-14
| | | | | | | | | | In commit 975a1a8,function ff_vc1_h_s_overlap_mmi was refactored, but the declaration in libavcodec/mips/vc1dsp_mips.h was unchanged. Change-Id: I90beae683511622a0cc1130ab1660ac8669ec3ef Signed-off-by: Shiyou Yin <yinshiyou-hf@loongson.cn> Reviewed-by: Jerome Borsboom <jerome.borsboom@carpalis.nl> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/internal: increase FF_SANE_NB_CHANNELS to 256Marton Balint2018-07-13
| | | | | | | | | | This was reduced from 128 in libav commit 192f1984b1a93aa08af053b8f9ab4950f307bd5d, but since we support unknown channel layouts, we can increase this limit. Fixes ticket #6332. Signed-off-by: Marton Balint <cus@passwd.hu>
* avcodec/utils: report insane channel count errorsMarton Balint2018-07-13
| | | | | | More than 64 is not *that* insane, so let's report the error at least. Signed-off-by: Marton Balint <cus@passwd.hu>
* lavc/aarch64/h264dsp_init_aarch64: Fix weight function prototypes.Carl Eugen Hoyos2018-07-13
| | | | | | | | | | | | | | | | | | | | | | | Fixes the following warnings: libavcodec/aarch64/h264dsp_init_aarch64.c: In function ‘ff_h264dsp_init_aarch64’: libavcodec/aarch64/h264dsp_init_aarch64.c:84:38: warning: assignment from incompatible pointer type [enabled by default] c->weight_h264_pixels_tab[0] = ff_weight_h264_pixels_16_neon; ^ libavcodec/aarch64/h264dsp_init_aarch64.c:85:38: warning: assignment from incompatible pointer type [enabled by default] c->weight_h264_pixels_tab[1] = ff_weight_h264_pixels_8_neon; ^ libavcodec/aarch64/h264dsp_init_aarch64.c:86:38: warning: assignment from incompatible pointer type [enabled by default] c->weight_h264_pixels_tab[2] = ff_weight_h264_pixels_4_neon; ^ libavcodec/aarch64/h264dsp_init_aarch64.c:88:40: warning: assignment from incompatible pointer type [enabled by default] c->biweight_h264_pixels_tab[0] = ff_biweight_h264_pixels_16_neon; ^ libavcodec/aarch64/h264dsp_init_aarch64.c:89:40: warning: assignment from incompatible pointer type [enabled by default] c->biweight_h264_pixels_tab[1] = ff_biweight_h264_pixels_8_neon; ^ libavcodec/aarch64/h264dsp_init_aarch64.c:90:40: warning: assignment from incompatible pointer type [enabled by default] c->biweight_h264_pixels_tab[2] = ff_biweight_h264_pixels_4_neon; ^
* lavc/hevc_ps: use skip_bits instead of get_bits when skip bits.Jun Zhao2018-07-13
| | | | | | use skip_bits when want to skip some bits. Signed-off-by: Jun Zhao <mypopydev@gmail.com>
* lavc/hevc_ps: Refine sps_range_extension parse.Jun Zhao2018-07-13
| | | | Signed-off-by: Jun Zhao <mypopydev@gmail.com>
* libavcodec/mpegaudiodecheader.h: fix version check patternKarsten Otto2018-07-13
| | | | | | | This fixes the check for the reserved MPEG audio version ID, used to detect an invalid frame header. Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* lavc/h263dec: Remove a variable declaration that can lead to a warning.Carl Eugen Hoyos2018-07-13
|
* lavc/qsv: handle MFX_FRAMETYPE_UNKNOWN caseZhong Li2018-07-12
| | | | Signed-off-by: Zhong Li <zhong.li@intel.com>
* libavcodec/mpegaudiodecheader.h : detect reserved mpeg idKarsten Otto2018-07-09
| | | | | | | | Check the MPEG version ID for the reserved bit pattern 01, and abort the header check in that case. This reduces the chance of misinterpreting arbitrary data as a valid header, and prevents resulting audio artifacts. Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/mjpegdec: Check for odd progressive RGBMichael Niedermayer2018-07-08
| | | | | | | | Fixes: out of array access Fixes: 9225/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_JPEGLS_fuzzer-5684770334834688 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* libavcodec/vp8: Do not compute line pointers per pixel in fade()Michael Niedermayer2018-07-07
| | | | | | | 72->60 seconds Testcase: 8680/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VP7_fuzzer-5861504418054144 Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/vp8_parser: Do not leave data/size uninitializedMichael Niedermayer2018-07-07
| | | | | | | | | | | | This is identical to what the VP9 parser does Fixes: 9215/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_LIBVPX_VP8_fuzzer-5768227253649408 Fixes: out of memory access This may also fix oss fuzz issue 9212 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/mpeg4videodec: Check for bitstream end in read_quant_matrix_ext()Michael Niedermayer2018-07-04
| | | | | | | | Fixes: out of array read Fixes: asff-crash-0e53d0dc491dfdd507530b66562812fbd4c36678 Found-by: Paul Ch <paulcher@icloud.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/atrac9tab: add missing header includeJames Almer2018-07-04
| | | | | | Fixes make checkheaders Signed-off-by: James Almer <jamrial@gmail.com>
* lavc/atrac9tab: Add inclusion guards.Carl Eugen Hoyos2018-07-04
| | | | Fixes fate-source.
* avcodec/indeo4: Check for end of bitstream in decode_mb_info()Michael Niedermayer2018-07-04
| | | | | | | | Fixes: Timeout Fixes: 8776/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_INDEO4_fuzzer-5361788798369792 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/truemotion2: Check len in tm2_read_stream()Michael Niedermayer2018-07-04
| | | | | | | Fixes: Timeout Fixes: 8774/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_TRUEMOTION2_fuzzer-5942199639343104 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
* avcodec/ac3dec: Check channel_map indexMichael Niedermayer2018-07-04
| | | | | | | | Fixes: out of array read Fixes: 8924/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_EAC3_fuzzer-5851861780267008 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/mpeg4videodec: Remove use of FF_PROFILE_MPEG4_SIMPLE_STUDIO as ↵Michael Niedermayer2018-07-04
| | | | | | | | | | | | | | | indicator of studio profile The profile field is changed by code inside and outside the decoder, its not a reliable indicator of the internal codec state. Maintaining it consistency with studio_profile is messy. Its easier to just avoid it and use only studio_profile Fixes: assertion failure Fixes: ffmpeg_crash_9.avi Found-by: Thuan Pham, Marcel Böhme, Andrew Santosa and Alexandru Razvan Caciulescu with AFLSmart Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/shorten: Fix undefined addition in shorten_decode_frame()Michael Niedermayer2018-07-04
| | | | | | | | Fixes: signed integer overflow: 1139785606 + 1454196085 cannot be represented in type 'int' Fixes: 8937/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SHORTEN_fuzzer-6202943597445120 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/shorten: Fix undefined integer overflowMichael Niedermayer2018-07-04
| | | | | | | | Fixes: signed integer overflow: 8454144 * 256 cannot be represented in type 'int' Fixes: 8788/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SHORTEN_fuzzer-5728205041303552 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/jpeg2000dec: Fixes invalid shifts in ↵Michael Niedermayer2018-07-04
| | | | | | | | | | jpeg2000_decode_packets_po_iteration() Fixes: shift exponent 47 is too large for 32-bit type 'int' Fixes: 9163/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_JPEG2000_fuzzer-5661750182543360 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/jpeg2000dec: Check that there are enough bytes for all tilesMichael Niedermayer2018-07-04
| | | | | | | | Fixes: OOM Fixes: 8781/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_JPEG2000_fuzzer-5810709081358336 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* lavc: implement an ATRAC9 decoderRostislav Pehlivanov2018-07-03
| | | | | | | | | | | | | | | | | | | | This commit implements a full ATRAC9 decoder, a simple low-delay codec developed by Sony and used in most PSVita games, some PS3 games and some PS4 games. Its similar to AAC in that it uses Huffman coded scalefactors but instead of vector quantization it just Huffman codes the spectral coefficients (in a way similar to how Opus splits band energy coding into coarse and fine precision). It opts to write rather large Huffman codes by packing several small coefficients into one Huffman coded symbol, though I don't believe this increases efficiency at all. Band extension implements SBC in a simple way, first it mirrors the lower spectrum onto the higher frequencies and then it uses one of 5 filters to shape it. Noise substitution is implemented via 2 of them. Unlike previous ATRAC codecs, there's no QMF, this is a standard MDCT codec. Based off of the reverse engineering work of Alex Barney. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
* avcodec/dvdsubdec: Check for fully transparent rectangles earlierMichael Niedermayer2018-07-02
| | | | | | | | | | Testcase with large transparent rectangles changes from 67 sec to 3 sec decode time Fixes: Timeout Fixes: 8728/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DVDSUB_fuzzer-5190088756559872 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/vc1_block: Fix mqaunt check for negative valuesMichael Niedermayer2018-06-30
| | | | | | | | | | | Fixes: out of array access Fixes: ffmpeg_bof_4.avi Fixes: ffmpeg_bof_5.avi Fixes: ffmpeg_bof_6.avi Found-by: Thuan Pham, Marcel Böhme, Andrew Santosa and Alexandru Razvan Caciulescu with AFLSmart Reviewed-by: Jerome Borsboom <jerome.borsboom@carpalis.nl> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/vc1: fix check for missing CBPTABJerome Borsboom2018-06-29
| | | | | | CBPTAB must be present in (non skipped) P and B pictures. Signed-off-by: Jerome Borsboom <jerome.borsboom@carpalis.nl>
* avcodec/vc1: fix overlap filter for frame interlaced picturesJerome Borsboom2018-06-29
| | | | | | | | | | The overlap filter is not correct for vertical edges in frame interlaced I and P pictures. When filtering macroblocks with different FIELDTX values, we have to match the lines at both sides of the vertical border. In addition, we have to use the correct rounding values, depending on the line we are filtering. Signed-off-by: Jerome Borsboom <jerome.borsboom@carpalis.nl>
* avcodec/vc1: change the internal ordering of blocks within a macroblockJerome Borsboom2018-06-29
| | | | | | | | | | The overlap filter needs to cover a full macroblock vertical edge when the FIELDTX value for two neighbouring macroblocks is not equal. By changing the internal ordering of the blocks from row major to column major, we do not need to reinterlace a FIELDTX coded macroblock before running the overlap filter. Signed-off-by: Jerome Borsboom <jerome.borsboom@carpalis.nl>
* avcodec/vc1: fix condition guarding overlap filter on I pictureJerome Borsboom2018-06-29
| | | | | | | The overlap filter needs to run when PQUANT is 9 or higher, irrespective of CONDOVER. Signed-off-by: Jerome Borsboom <jerome.borsboom@carpalis.nl>
* h264_slice: Fix return of incomplete frames from decoderJohn Stebbins2018-06-28
| | | | | | | | | | | When not using libavformat for demuxing, AVCodecContext.has_b_frames gets set too late causing the recovery frame heuristic in h264_refs to incorrectly flag an early frame as recovered. This patch sets has_b_frames earlier to prevent improperly flagging the frame as recovered. Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/mpeg4videodec: Check read profile before setting itMichael Niedermayer2018-06-28
| | | | | | | | Fixes: null pointer dereference Fixes: ffmpeg_crash_7.avi Found-by: Thuan Pham, Marcel Böhme, Andrew Santosa and Alexandru Razvan Caciulescu with AFLSmart Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/ac3_parser: Check init_get_bits8() for failureMichael Niedermayer2018-06-28
| | | | | | | | | Fixes: null pointer dereference Fixes: ffmpeg_crash_6.avi Found-by: Thuan Pham, Marcel Böhme, Andrew Santosa and Alexandru Razvan Caciulescu with AFLSmart Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/dpx: Check elements in 12bps planar pathMichael Niedermayer2018-06-28
| | | | | | | | | Fixes: null pointer dereference Fixes: 8946/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DPX_fuzzer-5078915222601728 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Carl Eugen Hoyos <ceffmpeg@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/qsvenc: fix version detection on cygwinTimo Rothenpieler2018-06-27
|
* avcodec/escape124: Fix spelling errors in commentMichael Niedermayer2018-06-27
| | | | Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/dvbsubdec: Compute scoretab without iterating over pixels multiple ↵Michael Niedermayer2018-06-27
| | | | | | | | | | | times in compute_default_clut() Improves speed 102->2 sec Testcase: 8655/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DVBSUB_fuzzer-6277869285146624 Fixes: Timeout Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>