diff options
author | Andreas Rheinhardt <andreas.rheinhardt@outlook.com> | 2021-04-05 02:05:58 +0200 |
---|---|---|
committer | Andreas Rheinhardt <andreas.rheinhardt@outlook.com> | 2021-04-08 01:17:24 +0200 |
commit | 9abda1365c5e2d827eb673b6d98245163c868bf1 (patch) | |
tree | e0c3436b48ef988b37fd0a861fdba4790ad88cb9 /libavcodec/mpegvideo.c | |
parent | 8ffd3ef9d94f33b411348c594a49d994b55c9550 (diff) |
avcodec/rv34, mpegvideo: Fix segfault upon frame size change error
The RealVideo 3.0 and 4.0 decoders call ff_mpv_common_init() only during
their init function and not during decode_frame(); when the size of the
frame changes, they call ff_mpv_common_frame_size_change(). Yet upon
error, said function calls ff_mpv_common_end() which frees the whole
MpegEncContext and not only those parts that
ff_mpv_common_frame_size_change() reinits. As a result, the context will
never be usable again; worse, because decode_frame() contains no check
for whether the context is initialized or not, it is presumed that it is
initialized, leading to segfaults. Basically the same happens if
rv34_decoder_realloc() fails.
This commit fixes this by only resetting the parts that
ff_mpv_common_frame_size_change() changes upon error and by actually
checking whether the context is in need of reinitialization in
ff_rv34_decode_frame().
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Diffstat (limited to 'libavcodec/mpegvideo.c')
-rw-r--r-- | libavcodec/mpegvideo.c | 6 |
1 files changed, 4 insertions, 2 deletions
diff --git a/libavcodec/mpegvideo.c b/libavcodec/mpegvideo.c index 7eddbdcc37..5de0719f83 100644 --- a/libavcodec/mpegvideo.c +++ b/libavcodec/mpegvideo.c @@ -555,7 +555,6 @@ int ff_mpeg_update_thread_context(AVCodecContext *dst, } if (s->height != s1->height || s->width != s1->width || s->context_reinit) { - s->context_reinit = 0; s->height = s1->height; s->width = s1->width; if ((ret = ff_mpv_common_frame_size_change(s)) < 0) @@ -1099,10 +1098,12 @@ int ff_mpv_common_frame_size_change(MpegEncContext *s) if (err < 0) goto fail; } + s->context_reinit = 0; return 0; fail: - ff_mpv_common_end(s); + free_context_frame(s); + s->context_reinit = 1; return err; } @@ -1149,6 +1150,7 @@ void ff_mpv_common_end(MpegEncContext *s) av_frame_free(&s->new_picture.f); s->context_initialized = 0; + s->context_reinit = 0; s->last_picture_ptr = s->next_picture_ptr = s->current_picture_ptr = NULL; |