avformat/avidec: Fix memleak when allocating DVDemuxContext fails

An AVIStream (intended to be used as private data for an AVStream) would
leak in this scenario.

Also return a more fitting error code instead of AVERROR_INVALIDDATA.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>

1 files changed, 6 insertions, 3 deletions

diff --git a/libavformat/avidec.c b/libavformat/avidec.c
index 18402f0032..87298513c2 100644
--- a/libavformat/avidec.c
+++ b/libavformat/avidec.c
@@ -605,9 +605,12 @@ static int avi_read_header(AVFormatContext *s)
                 ast = s->streams[0]->priv_data;
                 st->priv_data = NULL;
                 ff_free_stream(s, st);
-                    avi->dv_demux = avpriv_dv_init_demux(s);
-                    if (!avi->dv_demux)
-                        goto fail;
+
+                avi->dv_demux = avpriv_dv_init_demux(s);
+                if (!avi->dv_demux) {
+                    av_free(ast);
+                    return AVERROR(ENOMEM);
+                }
 
                 s->streams[0]->priv_data = ast;
                 avio_skip(pb, 3 * 4);