diff options
| author | Lynne <dev@lynne.ee> | 2021-01-09 20:41:25 +0100 |
|---|---|---|
| committer | Lynne <dev@lynne.ee> | 2021-01-09 20:54:56 +0100 |
| commit | 91e1625db15fe8853ceedca9eed14307aaa514c7 (patch) | |
| tree | 4b81ac9c4098b35eea16df71a852aedd56e73197 /libavutil/tx_priv.h | |
| parent | 29993b2947a99806cf41dd58853af510b0ce152c (diff) | |
lavu/tx: clip when converting table values to fixed-point
INT32_MAX (2147483647) isn't exactly representable by a floating point
value, with the closest being 2147483648.0. So when rescaling a value
of 1.0, this could overflow when casting the 64-bit value returned from
lrintf() into 32 bits.
Unfortunately the properties of integer overflows don't match up well
with how a Fourier Transform operates. So clip the value before
casting to a 32-bit int.
Should be noted we don't have overflows with the table values we're
currently using. However, converting a Kaiser-Bessel window function
with a length of 256 and a parameter of 5.0 to fixed point did create
overflows. So this is more of insurance to save debugging time
in case something changes in the future.
The macro is only used during init, so it being a little slower is
not a problem.
Diffstat (limited to 'libavutil/tx_priv.h')
| -rw-r--r-- | libavutil/tx_priv.h | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/libavutil/tx_priv.h b/libavutil/tx_priv.h index a3738f68bd..0ace3e90dc 100644 --- a/libavutil/tx_priv.h +++ b/libavutil/tx_priv.h @@ -85,7 +85,7 @@ typedef void FFTComplex; (dim) = (int)(((accu) + 0x40000000) >> 31); \ } while (0) -#define RESCALE(x) (lrintf((x) * 2147483648.0)) +#define RESCALE(x) (av_clip64(lrintf((x) * 2147483648.0), INT32_MIN, INT32_MAX)) #define FOLD(x, y) ((int)((x) + (unsigned)(y) + 32) >> 6) |