libswresample/swresample: avoid s16p internal transfer format

Instead use float one by default for sample rate conversions.
The s16p internal transfer format produces visible and hearable
quantization artifacts.

Signed-off-by: Paul B Mahol <onemda@gmail.com>

for S8 we continue to use S16 as it should have enough precision
Fate is adjusted so bitexactness is maintained between mips/arm/x86
if more tests became bit-inexact on some platform, the same change
can be done to them

The use of higher precision and float intermediates inevitably
leads to more differences between platforms.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This commit is contained in:
Paul B Mahol
2023-01-05 13:40:12 +01:00
committed by Michael Niedermayer
parent d897f82cf8
commit 3bef108fa9
3 changed files with 16 additions and 10 deletions

View File

@@ -331,8 +331,14 @@ av_cold int swr_init(struct SwrContext *s){
s->rematrix_custom;
if(s->int_sample_fmt == AV_SAMPLE_FMT_NONE){
// 16bit or less to 16bit or less with the same sample rate
if( av_get_bytes_per_sample(s-> in_sample_fmt) <= 2
&& av_get_bytes_per_sample(s->out_sample_fmt) <= 2){
&& av_get_bytes_per_sample(s->out_sample_fmt) <= 2
&& s->out_sample_rate==s->in_sample_rate) {
s->int_sample_fmt= AV_SAMPLE_FMT_S16P;
// 8 -> 8, 16->8, 8->16bit
} else if( av_get_bytes_per_sample(s-> in_sample_fmt)
+av_get_bytes_per_sample(s->out_sample_fmt) <= 3 ) {
s->int_sample_fmt= AV_SAMPLE_FMT_S16P;
}else if( av_get_bytes_per_sample(s-> in_sample_fmt) <= 2
&& !s->rematrix