diff options
| author | Ronald S. Bultje <rsbultje@gmail.com> | 2012-02-09 22:57:01 -0800 |
|---|---|---|
| committer | Ronald S. Bultje <rsbultje@gmail.com> | 2012-02-09 22:57:01 -0800 |
| commit | 45b7bd7c53b41bc5ff6fc2158831f2b1b1256113 (patch) | |
| tree | 51923d7569c3c823380fcca869fa8361055dcc3e /libavcodec/h264.c | |
| parent | 81749f30cd84b35f774d7d1bbe6bf3f96e2362c8 (diff) | |
h264: disallow constrained intra prediction modes for luma.
Conversion of the luma intra prediction mode to one of the constrained
("alzheimer") ones can happen by crafting special bitstreams, causing
a crash because we'll call a NULL function pointer for 16x16 block intra
prediction, since constrained intra prediction functions are only
implemented for chroma (8x8 blocks).
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Diffstat (limited to 'libavcodec/h264.c')
| -rw-r--r-- | libavcodec/h264.c | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/libavcodec/h264.c b/libavcodec/h264.c index cf409c0978..a80183b4a1 100644 --- a/libavcodec/h264.c +++ b/libavcodec/h264.c @@ -105,7 +105,7 @@ int ff_h264_check_intra4x4_pred_mode(H264Context *h){ * Check if the top & left blocks are available if needed and * change the dc mode so it only uses the available blocks. */ -int ff_h264_check_intra_pred_mode(H264Context *h, int mode){ +int ff_h264_check_intra_pred_mode(H264Context *h, int mode, int is_chroma){ MpegEncContext * const s = &h->s; static const int8_t top [7]= {LEFT_DC_PRED8x8, 1,-1,-1}; static const int8_t left[7]= { TOP_DC_PRED8x8,-1, 2,-1,DC_128_PRED8x8}; @@ -125,7 +125,7 @@ int ff_h264_check_intra_pred_mode(H264Context *h, int mode){ if((h->left_samples_available&0x8080) != 0x8080){ mode= left[ mode ]; - if(h->left_samples_available&0x8080){ //mad cow disease mode, aka MBAFF + constrained_intra_pred + if(is_chroma && (h->left_samples_available&0x8080)){ //mad cow disease mode, aka MBAFF + constrained_intra_pred mode= ALZHEIMER_DC_L0T_PRED8x8 + (!(h->left_samples_available&0x8000)) + 2*(mode == DC_128_PRED8x8); } if(mode<0){ |