summaryrefslogtreecommitdiff
path: root/libavcodec/magicyuv.c
Commit message (Collapse)AuthorAge
* avcodec/magicyuv: Check slice size before reading flags and predMichael Niedermayer2020-10-24
| | | | | | | | | Fixes: heap-buffer-overflow Fixes: 26487/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MAGICYUV_fuzzer-5742553675333632 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/magicyuv: Don't waste stack spaceAndreas Rheinhardt2020-09-26
| | | | | | | | | | | | | | | Now that the HuffEntries are no longer sorted by the MagicYUV decoder, their symbols are trivial: The symbol of the element with index i is i. They can therefore be removed. Furthermore, despite the length of the codes being in the range 1..32 bits, the actual value of the codes is <= 4096 (for 12 bit content). The reason for this is that the longer codes are on the left side of the tree, so that the higher bits of these codes are simply zero. By using an uint16_t for the codes and removing the symbols entry, the size of each HuffEntry is decreased from eight to four, saving 16KB of stack space. Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/magicyuv: Avoid AV_QSORT when creating Huffman tableAndreas Rheinhardt2020-09-26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The MagicYUV format stores Huffman tables in its bitstream by coding the length of a given symbol; it does not code the actual code directly, instead this is to be inferred by the rule that a symbol is to the left of every shorter symbol in the Huffman tree and that for symbols of the same length the symbol is ascending from left to right. Our decoder implemented this by first sorting the array containing length and symbol of each element according to descending length and for equal length, according to ascending symbol. Afterwards, the current state in the tree got encoded in a variable code; if the next array entry had length len, then the len most significant bits of code contained the code of this entry. Whenever an entry of the array of length len was processed, code was incremented by 1U << (32 - len). So two entries of length len have the same effect as incrementing code by 1U << (32 - (len - 1)), which corresponds to the parent node of length len - 1 of the two nodes of length len etc. This commit modifies this to avoid sorting the entries before calculating the codes. This is done by calculating how many non-leaf nodes there are on each level of the tree before calculating the codes. Afterwards every leaf node on this level gets assigned the number of nodes already on this level as code. This of course works only because the entries are already sorted by their symbol initially, so that this algorithm indeed gives ascending symbols from left to right on every level. This offers both speed- as well as (obvious) codesize advantages. With Clang 10 the number of decicycles for build_huffman decreased from 1561987 to 1228405; for GCC 9 it went from 1825096 decicyles to 1429921. These tests were carried out with a sample with 150 frames that was looped 13 times; and this was iterated 10 times. The earlier reference point here is from the point when the loop generating the codes was traversed in reverse order (as the patch reversing the order led to performance penalties). Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/magicyuv: Fix edge case of building Huffman tableAndreas Rheinhardt2020-09-26
| | | | | | | | | | | | | | | | | | | | The MagicYUV format stores Huffman tables in its bitstream by coding the length of a given symbol; it does not code the actual code directly, instead this is to be inferred by the rule that a symbol is to the left of every shorter symbol in the Huffman tree and that for symbols of the same length the symbol is ascending from left to right. With one exception, this is also what our decoder did. The exception only matters when there are codes of length 32, because in this case the first symbol of this length did not get the code 0, but 1; e.g. if there were exactly two nodes of length 32, then they would get assigned the codes 1 and 2 and a node of length 31 will get the 31-bit code 1 which is a prefix of the 32 bit code 2, making the Huffman table invalid. On the other hand, if there were only one symbol with the length 32, the earlier code would accept this un-Huffman-tree. Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/magicyuv: Don't invert order unnecessarilyAndreas Rheinhardt2020-09-26
| | | | | | | | | | | | | | The MagicYUV decoder currently sets both the length and the symbol field of an array of HuffEntries; hereby the symbol of the ith entry (0-based) is just i. Then said array gets sorted so that entries with greater length are at the end and entries with the same length are ordered so that those with smaller symbols are at the end. Afterwards the newly sorted array is traversed in reverse order. This commit instead inverts the ordering and traverses the array in its ordinary order in order to simplify understanding. Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/magicyuv: Replace implicit checks for overread by explicit onesAndreas Rheinhardt2020-09-26
| | | | | Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/magicyuv: Use const uint8_t* for pointer to immutable dataAndreas Rheinhardt2020-09-26
| | | | | Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/magicyuv: Don't use GetBit API for byte-aligned readsAndreas Rheinhardt2020-09-26
| | | | | Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/magicyuv: Check early for invalid slicesAndreas Rheinhardt2020-09-26
| | | | | | | | Every plane of each slice has to contain at least two bytes for flags and the type of prediction used. Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/magicyuv: Improve overread check when parsing Huffman tablesAndreas Rheinhardt2020-09-26
| | | | | Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/magicyuv: add support for recently added new formatPaul B Mahol2020-09-22
|
* avcodec/magicyuv: Avoid intermediate array when parsing Huffman tablesAndreas Rheinhardt2020-09-01
| | | | | Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/magicyuv: Unify creating Huffman tablesAndreas Rheinhardt2020-09-01
| | | | | Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/magicyuv: Reuse array instead of using multiple arraysAndreas Rheinhardt2020-09-01
| | | | | | | | | | The lengths of the VLC codes are implicitly contained in the VLC tables itself; apart from that they are not used lateron. So it is unnecessary to store them and the very same array can be reused to parse the Huffman table for the next plane. Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/magicyuv: Don't zero unnecessarilyAndreas Rheinhardt2020-09-01
| | | | | | | | The code already checks that exactly the expected amount of entries are read and set. Ergo it is unnecessary to zero them at the beginning. Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/magicyuv: Simplify check for invalid Huffman codesAndreas Rheinhardt2020-09-01
| | | | | Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/magicyuv: Unify qsort comparison functionsAndreas Rheinhardt2020-09-01
| | | | | | | | | | Up until now, there were three comparison functions depending upon bitness. But they all are actually the same, namely a lexical ordering: entry a > entry b iff a.len > b.len or a.len == b.len and a.sym < b.sym. So they can be easily unified. Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/magicyuv: Avoid copying values around pointlesslyAndreas Rheinhardt2020-09-01
| | | | | | | | | | | | | | | | | | When parsing Huffman tables, an array of HuffEntries (a struct containing a code's bitlength, its bits and its symbol) is used as intermediate tables in order to sort the entries (the order depends on both the length of the entries as well as on their symbols). After sorting them, the symbol and len components are copied into other arrays (the HuffEntries' code has never been set or used, despite using quite a lot of stack space) and the codes are generated. Afterwards, the VLC is created. Yet ff_init_vlc_sparse() can handle non-continuous arrays as input; there is no need to copy the entries at all. This commit implements this. Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/magicyuv: Don't invert Huffman table symbols twiceAndreas Rheinhardt2020-09-01
| | | | | | | | | | | | | | | | | | | | When the MagicYUV decoder builds Huffman tables from an array of code lengths, it proceeds as follows: First it copies the entries of the array of lengths into an array of HuffEntries (a struct which contains a length and a symbol field); it also sets the symbol field in descending order from nb_elem - 1 to 0, where nb_elem is the common number of elements of the length and HuffEntry arrays. Then the HuffEntry array is sorted lexicographically: a > b iff a.len > b.len or a.len == b.len and a.sym > b.sym. Afterwards the symbols of the so sorted array are inverted again (i.e. each symbol sym is replaced by nb_elem - sym). Yet inverting can easily be avoided altogether: Just modify the order so that smaller symbols correspond to bigger HuffEntries. This leads to the same permutation as the current code does and given that the two inversions just cancel each other out, the result is the same. Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/magicyuv: change bits used by 12bit tablesPaul B Mahol2020-08-31
| | | | Higher number slows decoder.
* avcodec/magicyuv: use cached bitstream reader except for x32Paul B Mahol2020-08-31
|
* avcodec/magicyuv: invert symbols when building vlcPaul B Mahol2020-08-31
| | | | Instead at every decoded symbol.
* pthread_frame: merge the functionality for normal decoder init and ↵Anton Khirnov2020-04-10
| | | | | | | | | | | | | | | | init_thread_copy The current design, where - proper init is called for the first per-thread context - first thread's private data is copied into private data for all the other threads - a "fixup" function is called for all the other threads to e.g. allocate dynamically allocated data is very fragile and hard to follow, so it is abandoned. Instead, the same init function is used to init each per-thread context. Where necessary, AVCodecInternal.is_copy can be used to differentiate between the first thread and the other ones (e.g. for decoding the extradata just once).
* avcodec/magicyuv: Check that there are enough lines for interlacing to be ↵Michael Niedermayer2020-02-25
| | | | | | | | | | | possible Fixes: out of array access Fixes: 20763/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MAGICYUV_fuzzer-5759562508664832 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/magicyuv: remove duplicate codeLimin Wang2019-10-10
| | | | | | Signed-off-by: Limin Wang <lance.lmwang@gmail.com> Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/magicyuv: add support for recently added YUV444P10Paul B Mahol2019-07-11
|
* avcodec/magicyuv: Check bits left in flags&1 branchMichael Niedermayer2018-06-25
| | | | | | | | | Fixes: Timeout Fixes: 8690/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MAGICYUV_fuzzer-6542020913922048 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/magicyuv : use gradient_pred dsp func for 8 bits gradient modeMartin Vignali2017-12-19
|
* libavcodec/magicyuv : remove unneed variable assignmentMartin Vignali2017-10-29
|
* avcodec/magicyuv: Check that vlc len is not too largeMichael Niedermayer2017-07-12
| | | | | | | | Fixes: runtime error: shift exponent -95 is negative Fixes: 2568/clusterfuzz-testcase-minimized-4926115716005888 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/magicyuv: add 12 bit formatsPaul B Mahol2017-07-10
| | | | Signed-off-by: Paul B Mahol <onemda@gmail.com>
* avcodec/magicyuv: make RLE table reading match referencePaul B Mahol2017-07-10
| | | | Signed-off-by: Paul B Mahol <onemda@gmail.com>
* avcodec/magicyuv: Check len to be supportedMichael Niedermayer2017-05-07
| | | | | | | | Fixes: shift exponent -1 is negative Fixes: 1390/clusterfuzz-testcase-minimized-5452757630713856 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* lossless_videodsp: rename add_hfyu_left_pred_int16 to add_left_pred_int16James Almer2017-01-12
| | | | Signed-off-by: James Almer <jamrial@gmail.com>
* huffyuvdsp: move functions only used by huffyuv from lossless_videodspJames Almer2017-01-12
| | | | Signed-off-by: James Almer <jamrial@gmail.com>
* lossless_videodsp: move shared functions from huffyuvdspJames Almer2017-01-12
| | | | | | Several codecs other than huffyuv use them. Signed-off-by: James Almer <jamrial@gmail.com>
* avcodec/magicyuv: export colorspace and color_range for YUVPaul B Mahol2016-12-26
| | | | Signed-off-by: Paul B Mahol <onemda@gmail.com>
* avcodec/magicyuv: add 10 bit supportPaul B Mahol2016-12-20
| | | | Signed-off-by: Paul B Mahol <onemda@gmail.com>
* Merge commit 'd78fd2fa21cde28465e40dd0be4446b1387d22a6'Clément Bœsch2016-07-14
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * commit 'd78fd2fa21cde28465e40dd0be4446b1387d22a6': Add MagicYUV decoder Changes observed from Libav: - many cosmetics (function renames/move, spacing, line breaks) - MagicYUVContext.slices_size is now unsigned - use of pixdesc (include fixed in FFmpeg) - mention of "Lossless" in the long name dropped (also removed from general.texi in FFmpeg) - addition of the FF_CODEC_CAP_INIT_THREADSAFE caps - use of qsort() instead of AV_QSORT() (NOT MERGED) - use of AVCodecContext.{width,height} instead of AVCodecContext.coded_{width,height} (NOT MERGED) See also 77f9c4b7aa9eb793b3019025e177245896821816 Merged-by: Clément Bœsch <u@pkh.me>
| * Add MagicYUV decoderPaul B Mahol2016-06-20
| | | | | Signed-off-by: Paul B Mahol <onemda@gmail.com> Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
* lavc/magicyuv: fix undefined behaviour introduced in 8a135a55bClément Bœsch2016-06-19
| | | | Order of evaluation of parameters in C is not defined.
* avcodec/magicyuv: check dimensionsPaul B Mahol2016-06-19
| | | | Signed-off-by: Paul B Mahol <onemda@gmail.com>
* avcodec/magicyuv: set correct size of last slice for each planePaul B Mahol2016-06-02
| | | | | | Fixes invalid read. Signed-off-by: Paul B Mahol <onemda@gmail.com>
* avcodec/magicyuv: fix decoding of raw slicesPaul B Mahol2016-06-02
| | | | Signed-off-by: Paul B Mahol <onemda@gmail.com>
* avocdec: add MagicYUV decoderPaul B Mahol2016-05-31
Signed-off-by: Paul B Mahol <onemda@gmail.com>
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-30 04:42:54 +0000