avformat/matroskadec: Don't output uninitialized data for RealAudio 28.8

The Matroska demuxer splits every sequence of h Matroska Blocks into
h * w / cfs packets of size cfs; here h (sub_packet_h), w (frame_size)
and cfs (coded_framesize) are parameters from the track's CodecPrivate.

It does this by splitting the Block's data in h/2 pieces of size cfs each
and putting them into a buffer at offset m * 2 * w + n * cfs where
m (range 0..(h/2 - 1)) indicates the index of the current piece in the
current Block and n (range 0..(h - 1)) is the index of the current Block
in the current sequence of Blocks. The data in this buffer is then used
for the output packets.

The problem is that there is currently no check to actually guarantee
that no uninitialized data will be output. One instance where this is
trivially so is if h == 1; another is if cfs * h is so small that the
input pieces do not cover everything that is output. In order to
preclude this, rmdec.c checks for h * cfs == 2 * w and h >= 2. The
former requirement certainly makes much sense, as it means that for
every given m the input pieces (corresponding to the h different values
of n) form a nonoverlapping partition of the two adjacent frames of size w
corresponding to m. But precluding h == 1 is not enough, other odd
values can cause problems, too. That is because the assumption behind
the code is that h frames of size w contain data to be output, although
the real number is h/2 * 2. E.g. for h = 3, cfs = 2 and w = 3 the
current code would output four (== h * w / cfs) packets. although only
data for three (== h/2 * h) packets has been read.

(Notice that if h * cfs == 2 * w, h being even is equivalent to
cfs dividing w; the latter condition also seems very reasonable:
It means that the subframes are a partition of the frames.)

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

1 files changed, 2 insertions, 2 deletions

diff --git a/libavformat/matroskadec.c b/libavformat/matroskadec.c
index 951695b5b5..1dc0b77962 100644
--- a/libavformat/matroskadec.c
+++ b/libavformat/matroskadec.c
@@ -2612,8 +2612,8 @@ static int matroska_parse_tracks(AVFormatContext *s)
                 return AVERROR_INVALIDDATA;
 
             if (codec_id == AV_CODEC_ID_RA_288) {
-                if ((int64_t)track->audio.sub_packet_h * track->audio.coded_framesize
-                    > (2 + (track->audio.sub_packet_h & 1)) * track->audio.frame_size)
+                if (track->audio.sub_packet_h & 1 || 2 * track->audio.frame_size
+                    != (int64_t)track->audio.sub_packet_h * track->audio.coded_framesize)
                     return AVERROR_INVALIDDATA;
                 st->codecpar->block_align = track->audio.coded_framesize;
                 track->codec_priv.size = 0;