summaryrefslogtreecommitdiff
path: root/libavcodec/mpeg4videodec.c
Commit message (Collapse)AuthorAge
* lavc: deprecate AV_CODEC_(FLAG|CAP)_TRUNCATEDAnton Khirnov2021-09-20
| | | | | | | | | | It is supported only by a few decoders (h263, h263p, mpeg(1|2|)video and mpeg4) and is entirely redundant with parsers. Furthermore, using it leads to missing frames, as flushing the decoder at the end does not work properly. Co-authored-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
* avcodec/mpegvideo: Move startcodes to mpeg12.hAndreas Rheinhardt2021-09-20
| | | | | | | And remove the MPEG-4-specific SLICE_START_CODE, which duplicates SLICE_STARTCODE. Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
* avcodec: Constify AVCodecsAndreas Rheinhardt2021-04-27
| | | | | | | | | | Given that the AVCodec.next pointer has now been removed, most of the AVCodecs are not modified at all any more and can therefore be made const (as this patch does); the only exceptions are the very few codecs for external libraries that have a init_static_data callback. Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com> Signed-off-by: James Almer <jamrial@gmail.com>
* avcodec/mpeg4video: Make initializing RLTable thread-safeAndreas Rheinhardt2021-04-27
| | | | | | | | | | | Up until now the RLTable ff_mpeg4_rl_intra was initialized by both mpeg4 decoder and encoder (except the VLCs that are only used by the decoder). This is an obstacle to making these codecs init-threadsafe, so move initializing this to a single function that is guarded by a dedicated AVOnce. Reviewed-by: Anton Khirnov <anton@khirnov.net> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/mpeg4videodec: update exported AVOptions in the user-facing contextJames Almer2021-04-14
| | | | | | | This prevents bogus values being reported on frame multithreaded decoding scenarios. Signed-off-by: James Almer <jamrial@gmail.com>
* Revert "avcodec: add FF_CODEC_CAP_INIT_CLEANUP for all codecs which use ↵Andreas Rheinhardt2021-04-08
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ff_mpv_common_init()" This mostly reverts commit 4b2863ff01b1fe93d9a518523c9098d17a9d8c6f. Said commit removed the freeing code from ff_mpv_common_init(), ff_mpv_common_frame_size_change() and ff_mpeg_framesize_alloc() and instead added the FF_CODEC_CAP_INIT_CLEANUP to several codecs that use ff_mpv_common_init(). This introduced several bugs: a) Several decoders using ff_mpv_common_init() in their init function were forgotten: This affected FLV, Intel H.263, RealVideo 3.0 and V4.0 as well as VC-1/WMV3. b) ff_mpv_common_init() is not only called from the init function of codecs, it is also called from AVCodec.decode functions. If an error happens after an allocation has succeeded, it can lead to memleaks; furthermore, it is now possible for the MpegEncContext to be marked as initialized even when ff_mpv_common_init() returns an error and this can lead to segfaults because decoders that call ff_mpv_common_init() when decoding a frame can mistakenly think that the MpegEncContext has been properly initialized. This can e.g. happen with H.261 or MPEG-4. c) Removing code for freeing from ff_mpeg_framesize_alloc() (which can't be called from any init function) can lead to segfaults because the check for whether it needs to allocate consists of checking whether the first of the buffers allocated there has been allocated. This part has already been fixed in 76cea1d2ce3f23e8131c8664086a1daf873ed694. d) ff_mpv_common_frame_size_change() can also not be reached from any AVCodec.init function; yet the changes can e.g. lead to segfaults with decoders using ff_h263_decode_frame() upon allocation failure, because the MpegEncContext will upon return be flagged as both initialized and not in need of reinitialization (granted, the fact that ff_h263_decode_frame() clears context_reinit before the context has been reinited is a bug in itself). With the earlier version, the context would be cleaned upon failure and it would be attempted to initialize the context again in the next call to ff_h263_decode_frame(). While a) could be fixed by adding the missing FF_CODEC_CAP_INIT_CLEANUP, keeping the current approach would entail adding cleanup code to several other places because of b). Therefore ff_mpv_common_init() is again made to clean up after itself; the changes to the wmv2 decoder and the SVQ1 encoder have not been reverted: The former fixed a memleak, the latter allowed to remove cleanup code. Fixes: double free Fixes: ff_free_picture_tables.mp4 Fixes: ff_mpeg_update_thread_context.mp4 Fixes: decode_colskip.mp4 Fixes: memset.mp4 Reviewed-by: Michael Niedermayer <michael@niedermayer.cc> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/mpeg4videodec: add forgotten flags to mpeg4_optionsPaul B Mahol2021-03-08
|
* Revert "mpeg4videodec: raise an error if sprite_trajectory.table is NULL"Andreas Rheinhardt2021-01-27
| | | | | | | | | | | | | | | | | | | | This reverts commit 6ac0e7818399a57e4684202bac79f35b3561ad1e. The mpeg4video parser can reach code that presumes that a certain VLC has been initialized; yet Libav did not ensure this and Libav bug #1012 [1] is about an ensuing crash. Instead of fixing the root cause a simple check for whether said VLC has already been initialized was added; said check is inherently racy. The proper fix is of course to ensure that the VLC is initialized and commit 7c76eaeca2791261d3f4f5c98c95f44abdbd879a already ensured this, so there was no need to merge 6ac0e7818399a57e4684202bac79f35b3561ad1e at all. This commit therefore reverts said commit. [1]: https://bugzilla.libav.org/show_bug.cgi?id=1012 Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/mpeg4videodec: Fix indentationAndreas Rheinhardt2021-01-26
| | | | | | It was wrong since e03bf251d8784f4d1df2c22381c902087e151e31. Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/mpeg4videodec: Move code around to avoid forward declarationAndreas Rheinhardt2021-01-26
| | | | | | | Also fix the indentation of decode_studio_vol_header while at it; it was wrong since 177133a0f4b41b3c98b9cbc7f8f45755412c537b. Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/mpeg4videodec: Don't initialize unused RL VLCsAndreas Rheinhardt2021-01-25
| | | | | | | | | | The RLTables whose VLC tables are only used for intra blocks only use the very first VLC table; yet all 32 have been initialized. This commit stops this by switching to INIT_FIRST_VLC_RL. This saves 201624B from the .bss segment; in case the decoder is actually used, this translates into less memory used. Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* Mark some pointers as constAndreas Rheinhardt2021-01-01
| | | | | Reviewed-by: Lynne <dev@lynne.ee> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/mpeg4videodec: Make studio VLCs staticAndreas Rheinhardt2020-12-08
| | | | | Reviewed-by: Michael Niedermayer <michael@niedermayer.cc> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/mpeg4video: Make tables used to initialize VLCs smallerAndreas Rheinhardt2020-12-08
| | | | | | | | | | Switching from ff_init_vlc_sparse() to ff_init_vlc_from_lengths() allows to replace codes which are so long that they need to be stored in an uint16_t by symbols which fit into an uint8_t; and even these can be avoided in case of the sprite trajectory VLC. Reviewed-by: Michael Niedermayer <michael@niedermayer.cc> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/mpeg4videodec: Don't check for errors for complete VLCsAndreas Rheinhardt2020-10-27
| | | | Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/mpeg4videodec: Use smallest max_depth in get_vlc2()Andreas Rheinhardt2020-10-24
| | | | | | | The longest code here is 12 bits long and can be read in two attempts. Reviewed-by: Michael Niedermayer <michael@niedermayer.cc> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* lavc, lavf: Remove newlines from calls to avpriv_request_sample().Carl Eugen Hoyos2020-10-11
|
* avcodec/mpeg4videodec: Fix 2 integer overflows in get_amv()Michael Niedermayer2020-06-14
| | | | | | | | Fixes: signed integer overflow: -144876608 * 16 cannot be represented in type 'int' Fixes: 22782/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-6039584977977344 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/mpeg4videodec: avoid invalid values and reinitialize in format ↵Michael Niedermayer2020-06-14
| | | | | | | | | | changes for studio profile Fixes: out of array access Fixes: 23327/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-5134822992510976 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec: add FF_CODEC_CAP_INIT_CLEANUP for all codecs which use ↵Limin Wang2020-06-13
| | | | | | ff_mpv_common_init() Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
* lavc: Rename hwaccel.h to hwconfig.hMark Thompson2020-04-26
| | | | | This already applied to decoders as well as hwaccels, and adding encoder support was going to make the name even more inaccurate.
* pthread_frame: merge the functionality for normal decoder init and ↵Anton Khirnov2020-04-10
| | | | | | | | | | | | | | | | init_thread_copy The current design, where - proper init is called for the first per-thread context - first thread's private data is copied into private data for all the other threads - a "fixup" function is called for all the other threads to e.g. allocate dynamically allocated data is very fragile and hard to follow, so it is abandoned. Instead, the same init function is used to init each per-thread context. Where necessary, AVCodecInternal.is_copy can be used to differentiate between the first thread and the other ones (e.g. for decoding the extradata just once).
* lavc: replace AVCodecInternal.allocate_progress with an internal capAnton Khirnov2020-04-10
| | | | This is a constant codec property, so a capability flag is more appropriate.
* mpeg4videodec: do not copy a range of fields at onceAnton Khirnov2020-04-10
| | | | | This is extremely fragile against reordering and hides what is actually being copied. Copy all the fields manually instead.
* avcodec: Replace get_bits_long() by get_bits() where possibleMichael Niedermayer2019-12-31
| | | | Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec: Replace show_bits_long() by show_bits() where possibleMichael Niedermayer2019-12-31
| | | | Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* repeat an even number of characters in occuredMichael Niedermayer2019-09-16
| | | | Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/mpeg4videodec: Fix integer overflow in mpeg4_decode_studio_block()Michael Niedermayer2019-09-11
| | | | | | | | | Fixes: signed integer overflow: 24023040 * 112 cannot be represented in type 'int' Fixes: 16570/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-5173275211071488 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Kieran Kunhya <kierank@obe.tv> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/mpeg4videodec: Check idx in mpeg4_decode_studio_block()Michael Niedermayer2019-03-12
| | | | | | | | | Fixes: Out of array access Fixes: 13500/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-5769760178962432 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Kieran Kunhya <kierank@obe.tv> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/mpeg4videodec: Fix nonsense warningAndreas Rheinhardt2019-03-12
| | | | | | | | | | | | Since db772308941a2a338c7809f90d347219a6a93074 parsing of mpeg4-extradata lead to a "Failed to parse extradata" warning, because ff_mpeg4_decode_picture_header returns AVERROR_INVALIDDATA in case that no VOP was found. This patch adds a parameter to signify whether a header (where the absence of a VOP does not raise an error) or not is parsed. The first mode is of course used for parsing headers. Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@googlemail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/mpeg4videodec: Clear interlaced_dct for studio profileMichael Niedermayer2019-02-16
| | | | | | | | | Fixes: Out of array access Fixes: 13090/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-5408668986638336 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Kieran Kunhya <kierank@obe.tv> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/mpeg4videodec: Clear partitioned frame in decode_studio_vop_header()Michael Niedermayer2018-11-04
| | | | | | | | | | partitioned_frame is also set/cleared in decode_vop_header() Fixes: out of array read Fixes: 9789/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-5638681627983872 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/mpeg4videodec: Fix typo in sprite delta checkMichael Niedermayer2018-10-22
| | | | | | | | Fixes: Integer overflow Fixes: 10890/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-5636062181851136 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/mpeg4videodec: Fix undefined shift in get_amv()Michael Niedermayer2018-09-19
| | | | | | | | Fixes: runtime error: shift exponent -1 is negative Fixes: 9938/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-5653783529914368 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/mpeg4videodec: Check rice_prefix_codeMichael Niedermayer2018-09-19
| | | | | | | | | | Fixes: out of array read Fixes: 10064/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-5766801384800256 Fixes: 10225/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-5071833448054784 Fixes: 10261/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-5115048024866816 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* mpeg4video: Add Studio DPCM supportKieran Kunhya2018-08-25
|
* avcodec/mpeg4videodec: Fix slice end detection in mpeg4_decode_studio_mb()Michael Niedermayer2018-08-25
| | | | Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/mpeg4videodec: Check for bitstream end in read_quant_matrix_ext()Michael Niedermayer2018-07-04
| | | | | | | | Fixes: out of array read Fixes: asff-crash-0e53d0dc491dfdd507530b66562812fbd4c36678 Found-by: Paul Ch <paulcher@icloud.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/mpeg4videodec: Remove use of FF_PROFILE_MPEG4_SIMPLE_STUDIO as ↵Michael Niedermayer2018-07-04
| | | | | | | | | | | | | | | indicator of studio profile The profile field is changed by code inside and outside the decoder, its not a reliable indicator of the internal codec state. Maintaining it consistency with studio_profile is messy. Its easier to just avoid it and use only studio_profile Fixes: assertion failure Fixes: ffmpeg_crash_9.avi Found-by: Thuan Pham, Marcel Böhme, Andrew Santosa and Alexandru Razvan Caciulescu with AFLSmart Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/mpeg4videodec: Check read profile before setting itMichael Niedermayer2018-06-28
| | | | | | | | Fixes: null pointer dereference Fixes: ffmpeg_crash_7.avi Found-by: Thuan Pham, Marcel Böhme, Andrew Santosa and Alexandru Razvan Caciulescu with AFLSmart Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/mpeg4videodec: Clear bits_per_raw_sample if it has originated from a ↵Michael Niedermayer2018-06-13
| | | | | | | | | | previous instance Fixes: assertion failure Fixes: ffmpeg_crash_5.avi Found-by: Thuan Pham <thuanpv@comp.nus.edu.sg>, Marcel Böhme, Andrew Santosa and Alexandru Razvan Caciulescu with AFLSmart Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/mpeg4video: Detect reference studio streams as studio streamsMichael Niedermayer2018-05-17
| | | | Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/mpeg4videodec: Check bps (VOL header) before VOP for studio profileMichael Niedermayer2018-05-17
| | | | | | | | | | | Fixes: runtime error: shift exponent -1 is negative Fixes: 7486/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-4977380939530240 Fixes: runtime error: index 36 out of bounds for type 'const uint8_t [32]' Fixes: 7566/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-6536620682510336 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/mpeg4videodec: Do not corrupt bits_per_raw_sampleMichael Niedermayer2018-05-17
| | | | | Reviewed-by: Kieran Kunhya <kierank@obe.tv> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/mpeg4videode: Eliminate out of loop VOP startcode reading for studio ↵Michael Niedermayer2018-05-17
| | | | | | profile Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/mpeg4videodec: Split decode_studio_vol_header() out of ↵Michael Niedermayer2018-05-03
| | | | | | decode_studiovisualobject() Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/mpeg4videodec: Move decode_studiovisualobject() parsing in the ↵Michael Niedermayer2018-05-03
| | | | | | | | | | branch for visual object parsing Fixes: runtime error: shift exponent -1 is negative Fixes: 7510/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-5024523356209152 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/mpeg4videodec: unbreak multithreading decodingJames Almer2018-04-04
| | | | | | Should fix double free related crashes. Signed-off-by: James Almer <jamrial@gmail.com>
* avcodec/mpeg4videodec: free studio profile VLCs when closing the decoderJames Almer2018-04-04
| | | | | | Fixes memleaks. Signed-off-by: James Almer <jamrial@gmail.com>
* mpeg4video: Add support for MPEG-4 Simple Studio Profile.Kieran Kunhya2018-04-02
| | | | This is a profile supporting > 8-bit video and has a higher quality DCT
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-27 03:07:52 +0000