swscale/ops: correctly truncate on ff_sws_apply_op_q(SWS_OP_RSHIFT)

Instead of using a "precise" division, simulate the actual truncation.

Note that the division by `den` is unneeded in principle because the
denominator *should* always be 1 for an integer, but this way we don't
explode if the user should happen to pass `4/2` or something.

Fixes a lot of unnecessary clamps w.r.t. xv36, e.g.:

 xv36be -> yuv444p12be:
   [u16 XXXX -> ++++] SWS_OP_READ         : 4 elem(s) packed >> 0
   [u16 ...X -> ++++] SWS_OP_SWAP_BYTES
   [u16 ...X -> ++++] SWS_OP_SWIZZLE      : 1023
   [u16 ...X -> ++++] SWS_OP_RSHIFT       : >> 4
-  [u16 ...X -> ++++] SWS_OP_CONVERT      : u16 -> f32
-  [f32 ...X -> ++++] SWS_OP_MIN          : x <= {4095 4095 4095 _}
-  [f32 ...X -> ++++] SWS_OP_CONVERT      : f32 -> u16
   [u16 ...X -> ++++] SWS_OP_SWAP_BYTES
   [u16 ...X -> ++++] SWS_OP_WRITE        : 3 elem(s) planar >> 0
     (X = unused, + = exact, 0 = zero)
This commit is contained in:
Niklas Haas
2025-12-16 00:36:55 +01:00
committed by Niklas Haas
parent d1eaea1a03
commit 75ba2bf457
2 changed files with 2 additions and 3 deletions

View File

@@ -1 +1 @@
708f10e4de79450729ab2120e4e44ec9
e910ff7ceaeb64bfdbac3f652b67403f