summaryrefslogtreecommitdiff
path: root/libavcodec/atrac9dec.c
Commit message (Collapse)AuthorAge
* avcodec/avcodec: Stop including channel_layout.h in avcodec.hAndreas Rheinhardt2021-07-22
| | | | | | Also include channel_layout.h directly wherever used. Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
* avcodec: Constify AVCodecsAndreas Rheinhardt2021-04-27
| | | | | | | | | | Given that the AVCodec.next pointer has now been removed, most of the AVCodecs are not modified at all any more and can therefore be made const (as this patch does); the only exceptions are the very few codecs for external libraries that have a init_static_data callback. Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com> Signed-off-by: James Almer <jamrial@gmail.com>
* lavu/mem: move the DECLARE_ALIGNED macro family to mem_internal on next+1 bumpAnton Khirnov2021-01-01
| | | | They are not properly namespaced and not intended for public use.
* avcodec: set AV_CODEC_CAP_CHANNEL_CONF on decoders which set their own channelsHendrik Leppkes2020-12-10
| | | | | | | | | | | | The decoders in this set either have a fixed channel count, or read it from the bitstream, and thus do not require the channel count as external information. Fixes various regressions since 81503ac58a763a36b1f57264013b1e76acb62b68, which requires a valid channel count for decoders which do not set this capability. Signed-off-by: Hendrik Leppkes <h.leppkes@gmail.com>
* avcodec/atrac9dec: Make VLCs staticAndreas Rheinhardt2020-12-08
| | | | | | | Also remove code duplication and use a named constant for the number of VLC bits while just at it. Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/atrac9dec: Don't create VLCs that are never usedAndreas Rheinhardt2020-12-08
| | | | | | | | | | | | | | | | | | | | | The ATRAC9 decoder creates VLCs with parameters contained in HuffmanCodebooks; some of these HuffmanCodebooks are empty and yet VLCs (that were completely unused*) were created from them. Said VLC contained a single table with 512 VLC_TYPE[2] entries, each of which indicated that this is an invalid code. This commit stops creating said VLCs. *: read_coeffs_coarse() uses the HuffmanCodebook at9_huffman_coeffs[cb][prec][cbi]. prec is c->precision_coarse[i] + 1 and every precision_coarse entry is in the 1..15 range after calc_precision(), so prec is >= 2 (all codebooks with prec < 2 are empty). The remaining empty codebooks are those with cb == 1 and cbi == 0, yet this is impossible, too: cb is given by c->codebookset[i] and this is always 0 if i < 8 (because those are never set to anything else in calc_codebook_idx()) and cbi is given by at9_q_unit_to_codebookidx[i] which is never zero if i >= 8. Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/atrac9tab: Unify tables used to initialize VLCsAndreas Rheinhardt2020-12-08
| | | | | | | | Using separate tables has the downside that one needs a big number of pointers to the separate tables (currently 77); unifying them avoids this. Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/atrac9dec: Make tables used to initialize VLCs smallerAndreas Rheinhardt2020-12-08
| | | | | | | | | | | | | | | | | | | | | | The ATRAC9 decoder uses VLCs which are currently initialized with static length tables of type uint8_t and code tables of type uint16_t. Furthermore, in one case the actually desired symbols are in the range -16..15 and in order to achieve this an ad-hoc symbols table of type int16_t is calculated. This commit modifies this process by replacing the codes tables by symbols tables and switching to ff_init_vlc_from_lengths(); the signed symbols are stored in the table after having been shifted by 16 to fit into an uint8_t and are shifted back when the VLC is created. This makes all symbols fit into an uint8_t, saving space. Furthermore, the earlier tables had holes in them (entries with length zero that were inserted because the actually used symbols were not contiguous); these holes are unnecessary in the new approach, leading to further saving. Finally, given that now both lengths as well as symbols are of the same type, they can be combined; this saves a pointer for each VLC. Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/atrac9dec: Replace av_free() by av_freep() in close functionAndreas Rheinhardt2020-11-24
| | | | Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/atrac9dec: Don't use unnecessarily large VLC tablesAndreas Rheinhardt2020-11-24
| | | | | | Using more bits for a VLC than the longest code has has no advantage. Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/atrac9dec: Use least max_depth in calls to get_vlc2()Andreas Rheinhardt2020-11-24
| | | | Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/atrac9dec: Don't confuse max_depth of VLC with max codelengthAndreas Rheinhardt2020-11-24
| | | | | | | | | | | | | | | The whole point of VLCs with their tables is to read more than one bit at a time; therefore max_depth, the number of times one has to (maximally) read further bits is given by ceil(max_code_length / table_bits) which in the case of ATRAC9's coefficient VLCs gives an upper bound of two. Instead the maximum length of a code of the given VLC has been used (which is not even a compile-time constant). Use two instead. Furthermore, given that this was the only usage of the field containing the maximum of all the code lengths of a given VLC the field has been removed from its containing struct. Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
* avcodec/atrac9dec: Clamp band_ext_data to max that can be read if skipped.Michael Niedermayer2019-12-28
| | | | | | | | | Fixes: out of array read Fixes: 19327/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ATRAC9_fuzzer-5679823087468544 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Lynne <dev@lynne.ee> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/atrac9dec: Check q_unit_cnt more completely before using it to ↵Michael Niedermayer2019-12-07
| | | | | | | | | | | access at9_tab_band_ext_group Fixes: index 8 out of bounds for type 'const uint8_t [8][3]' Fixes: 19127/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ATRAC9_fuzzer-5709394985091072 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Lynne <dev@lynne.ee> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/atrac9dec: Check precision_fine/coarseMichael Niedermayer2019-11-20
| | | | | | | | | | Clipping is done as it was preferred in review See: [FFmpeg-devel] [PATCH 1/5] avcodec/atrac9dec: Check precision_fine/coarse Fixes: out of array access Fixes: 18330/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ATRAC9_fuzzer-5641113058148352 Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/atrac9dec: Set channelsMichael Niedermayer2019-10-21
| | | | | | | | | Fixes: null pointer dereference Fixes: 18341/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ATRAC9_fuzzer-5681203490848768 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Lynne <dev@lynne.ee> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/atrac9dec: Check block_alignMichael Niedermayer2019-08-27
| | | | | | | | | Fixes: Infinite loop Fixes: 16260/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ATRAC9_fuzzer-5676365617037312 Fixes: 16260/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ATRAC9_fuzzer-5768093879500800 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/atrac9dec: Check grad_range[1] more tightlyMichael Niedermayer2019-08-05
| | | | | | | | | | | | Alternatively the array could be made bigger but the extra values would not be read without other changes. Fixes: Out of array access Fixes: 15658/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ATRAC9_fuzzer-5738260074070016 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Reviewed-by: Lynne <dev@lynne.ee> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/atrac9dec: Remove impossible conditionMichael Niedermayer2019-08-05
| | | | | | Suggested-by: Lynne <dev@lynne.ee> Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/atrac9dec: Check conditions before apply_band_extension() to avoid ↵Michael Niedermayer2019-07-19
| | | | | | | | | | out of array read in initialization of unused variables Fixes: global-buffer-overflow Fixes: 15247/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ATRAC9_fuzzer-5671602181636096 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/atrac9dec: Check q_unit_cnt in parse_band_ext()Michael Niedermayer2019-06-29
| | | | | | | | Fixes: global-buffer-overflow Fixes: 15247/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ATRAC9_fuzzer-5671602181636096 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* avcodec/atrac9dec: Check that the reused block has succeeded initilizationMichael Niedermayer2019-06-29
| | | | | | | | | Fixes: global-buffer-overflow Fixes: 15247/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_ATRAC9_fuzzer-5671602181636096 Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* libavcodec: Reduce the size of some arraysAndreas Rheinhardt2019-06-20
| | | | | | | | | | This commit uses smaller types for some static const arrays to reduce their size in case the entries can be represented in the smaller type. The biggest savings came from inv_map_table in vp9.c. Reviewed-by: Michael Niedermayer <michael@niedermayer.cc> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com> Signed-off-by: James Almer <jamrial@gmail.com>
* atrac9dec: clean up code slightlyRostislav Pehlivanov2018-08-28
| | | | | | | Just remove some dead variable assignments, unneeded variables and change the FFMAX order to something more readable. Still identical. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
* atrac9dec: implement LFE channel decodingRostislav Pehlivanov2018-08-27
| | | | | | | Much simpler than regular decoding, does allow for 5.1 and 7.1 streams to be decoded without desync. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
* atrac9dec: relax gradient value requirementsRostislav Pehlivanov2018-08-27
| | | | | | | | Unlike the range, the gradient start value does not have to be lower than the end value. Does allow more files to be correctly decoded without errors. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
* lavc: implement an ATRAC9 decoderRostislav Pehlivanov2018-07-03
This commit implements a full ATRAC9 decoder, a simple low-delay codec developed by Sony and used in most PSVita games, some PS3 games and some PS4 games. Its similar to AAC in that it uses Huffman coded scalefactors but instead of vector quantization it just Huffman codes the spectral coefficients (in a way similar to how Opus splits band energy coding into coarse and fine precision). It opts to write rather large Huffman codes by packing several small coefficients into one Huffman coded symbol, though I don't believe this increases efficiency at all. Band extension implements SBC in a simple way, first it mirrors the lower spectrum onto the higher frequencies and then it uses one of 5 filters to shape it. Noise substitution is implemented via 2 of them. Unlike previous ATRAC codecs, there's no QMF, this is a standard MDCT codec. Based off of the reverse engineering work of Alex Barney. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>