avcodec/jpeg2000: Move gainb handling into the quantization code

thats how the specification defines it, this also improves numerical
accuracy of the integer wavelet implementation. It otherwise should
be equivalent, in case of overflows this can be reverted.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>

4 files changed, 16 insertions, 16 deletions

diff --git a/tests/ref/vsynth/vsynth1-jpeg2000-97 b/tests/ref/vsynth/vsynth1-jpeg2000-97
index 15ca696a07..20ff0733cf 100644
--- a/tests/ref/vsynth/vsynth1-jpeg2000-97
+++ b/tests/ref/vsynth/vsynth1-jpeg2000-97
@@ -1,4 +1,4 @@
-4cac63aaf880ab0ee199ac44e520640a *tests/data/fate/vsynth1-jpeg2000-97.avi
-2188956 tests/data/fate/vsynth1-jpeg2000-97.avi
-d9227e7a7413cb48362654d879df985a *tests/data/fate/vsynth1-jpeg2000-97.out.rawvideo
-stddev:    6.18 PSNR: 32.30 MAXDIFF:   57 bytes:  7603200/  7603200
+0c704936bf54fad2b072dc0c8c289bb7 *tests/data/fate/vsynth1-jpeg2000-97.avi
+4336566 tests/data/fate/vsynth1-jpeg2000-97.avi
+1207ed65ca04d2c91936c4bc73c3a562 *tests/data/fate/vsynth1-jpeg2000-97.out.rawvideo
+stddev:    4.36 PSNR: 35.34 MAXDIFF:   51 bytes:  7603200/  7603200
diff --git a/tests/ref/vsynth/vsynth2-jpeg2000-97 b/tests/ref/vsynth/vsynth2-jpeg2000-97
index f6f4505bc5..caeb9adaab 100644
--- a/tests/ref/vsynth/vsynth2-jpeg2000-97
+++ b/tests/ref/vsynth/vsynth2-jpeg2000-97
@@ -1,4 +1,4 @@
-352d3dc81b5c78638a2e7b575ce468c2 *tests/data/fate/vsynth2-jpeg2000-97.avi
-1448342 tests/data/fate/vsynth2-jpeg2000-97.avi
-d7b18b6dcac0139858e64e00073db1c6 *tests/data/fate/vsynth2-jpeg2000-97.out.rawvideo
-stddev:    5.35 PSNR: 33.56 MAXDIFF:   56 bytes:  7603200/  7603200
+7bb7ccf4e9743888a5bda48f431630cf *tests/data/fate/vsynth2-jpeg2000-97.avi
+3124708 tests/data/fate/vsynth2-jpeg2000-97.avi
+fe0f3a97aa8c0b2f02dd11ccd1b77ca6 *tests/data/fate/vsynth2-jpeg2000-97.out.rawvideo
+stddev:    3.26 PSNR: 37.86 MAXDIFF:   26 bytes:  7603200/  7603200
diff --git a/tests/ref/vsynth/vsynth3-jpeg2000-97 b/tests/ref/vsynth/vsynth3-jpeg2000-97
index 236519863f..eff6945989 100644
--- a/tests/ref/vsynth/vsynth3-jpeg2000-97
+++ b/tests/ref/vsynth/vsynth3-jpeg2000-97
@@ -1,4 +1,4 @@
-36dddf487c4919d890db3484ab5d6486 *tests/data/fate/vsynth3-jpeg2000-97.avi
-66522 tests/data/fate/vsynth3-jpeg2000-97.avi
-332c0924bc1477a41eb413455c5b9d90 *tests/data/fate/vsynth3-jpeg2000-97.out.rawvideo
-stddev:    6.17 PSNR: 32.32 MAXDIFF:   51 bytes:    86700/    86700
+d551b1b1bfc691356e6f45f4386aff1d *tests/data/fate/vsynth3-jpeg2000-97.avi
+92144 tests/data/fate/vsynth3-jpeg2000-97.avi
+533535dad2ccc7aef3eb0e7ccb239bf2 *tests/data/fate/vsynth3-jpeg2000-97.out.rawvideo
+stddev:    4.60 PSNR: 34.87 MAXDIFF:   46 bytes:    86700/    86700
diff --git a/tests/ref/vsynth/vsynth_lena-jpeg2000-97 b/tests/ref/vsynth/vsynth_lena-jpeg2000-97
index 21160ef061..426787c0cd 100644
--- a/tests/ref/vsynth/vsynth_lena-jpeg2000-97
+++ b/tests/ref/vsynth/vsynth_lena-jpeg2000-97
@@ -1,4 +1,4 @@
-a39e052e7d109e33714329800c9df90f *tests/data/fate/vsynth_lena-jpeg2000-97.avi
-1105046 tests/data/fate/vsynth_lena-jpeg2000-97.avi
-c151476d2ed807575122064ada37d5ef *tests/data/fate/vsynth_lena-jpeg2000-97.out.rawvideo
-stddev:    4.85 PSNR: 34.41 MAXDIFF:   59 bytes:  7603200/  7603200
+6600b19f6c2e84c599f37d33a295ee35 *tests/data/fate/vsynth_lena-jpeg2000-97.avi
+2459730 tests/data/fate/vsynth_lena-jpeg2000-97.avi
+9d9c07b0d1936810032820a08d0b0b5d *tests/data/fate/vsynth_lena-jpeg2000-97.out.rawvideo
+stddev:    3.01 PSNR: 38.55 MAXDIFF:   27 bytes:  7603200/  7603200