diff options
| author | Christophe Gisquet <christophe.gisquet@gmail.com> | 2015-10-12 19:37:45 +0200 |
|---|---|---|
| committer | Michael Niedermayer <michael@niedermayer.cc> | 2015-10-13 02:10:51 +0200 |
| commit | 2fd14dd8eb66dc5dd14254d0b758fb80d44b3140 (patch) | |
| tree | 34a086df2390e67180c11276193f258ae4b540e3 /libavcodec/simple_idct.c | |
| parent | e9a68b0316ab127098ac4c24a6762ce68980bd23 (diff) | |
avcodec/simple_idct10: improve precision
omse goes from 0.03060703 (which fails for dct-test) to 0.01663750.
This also actually improve the error of decoding the sample generated
by fate-vsynth3-dnxhd1080i-10bit using simple_idct10 to FAANI, which
goes (when resampled to yuv422p) from:
stddev: 0.06 PSNR: 72.28 MAXDIFF: 1
to identical.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Diffstat (limited to 'libavcodec/simple_idct.c')
| -rw-r--r-- | libavcodec/simple_idct.c | 9 |
1 files changed, 7 insertions, 2 deletions
diff --git a/libavcodec/simple_idct.c b/libavcodec/simple_idct.c index eeb627999c..4d6d20df5a 100644 --- a/libavcodec/simple_idct.c +++ b/libavcodec/simple_idct.c @@ -36,6 +36,11 @@ #define BIT_DEPTH 10 #include "simple_idct_template.c" + +#define EXTRA_SHIFT 2 +#include "simple_idct_template.c" + +#undef EXTRA_SHIFT #undef BIT_DEPTH #define BIT_DEPTH 12 @@ -230,10 +235,10 @@ void ff_prores_idct(int16_t *block, const int16_t *qmat) block[i] *= qmat[i]; for (i = 0; i < 8; i++) - idctRowCondDC_10(block + i*8, 2); + idctRowCondDC_extrashift_10(block + i*8, 2); for (i = 0; i < 8; i++) { block[i] += 8192; - idctSparseCol_10(block + i); + idctSparseCol_extrashift_10(block + i); } } |