configure,swscale/x86/Makefile: Remove special red-zone handling

ff_h[yc]scale_fast_mmxext() call other functions from inline assembly;
these functions look like leaf functions to GCC, so it may use the
red zone to avoid modifying the stack. But this makes the call
instructions in the inline asm corrupt the stack.

In order to fix this 424bcc46b5
made libswscale/x86/swscale_mmx.o be compiled with -mno-red-zone.
Later Libav fixed it in their version in commit
b14fa5572c by saving and restoring
the memory clobbered by the call (as is still done now). This was
merged into FFmpeg in 0e7fc3cafe,
without touching the -mno-red-zone hack.

Libav later renamed swscale_mmx.c to just swscale.c in
16d2a1a51c which was merged into FFmpeg
in commit 2cb4d51654, without
removing the -mno-red-zone hack, although the file it applies to
no longer existed.

This commit removes the special red-zone handling given that it is
inactive anyway.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
This commit is contained in:
Andreas Rheinhardt
2026-03-09 15:35:07 +01:00
parent 9bbe1ec86f
commit 835781af23
2 changed files with 0 additions and 8 deletions

6
configure vendored
View File

@@ -8014,11 +8014,6 @@ fi
enabled ftrapv && check_cflags -ftrapv
test_cc -mno-red-zone <<EOF && noredzone_flags="-mno-red-zone"
int x;
EOF
if enabled icc; then
# Just warnings, no remarks
check_allcflags -w1
@@ -8678,7 +8673,6 @@ SLIB_INSTALL_EXTRA_LIB=${SLIB_INSTALL_EXTRA_LIB}
SLIB_INSTALL_EXTRA_SHLIB=${SLIB_INSTALL_EXTRA_SHLIB}
VERSION_SCRIPT_POSTPROCESS_CMD=${VERSION_SCRIPT_POSTPROCESS_CMD}
SAMPLES:=${samples:-\$(FATE_SAMPLES)}
NOREDZONE_FLAGS=$noredzone_flags
LIBFUZZER_PATH=$libfuzzer_path
IGNORE_TESTS=$ignore_tests
VERSION_TRACKING=$version_tracking

View File

@@ -1,5 +1,3 @@
$(SUBDIR)x86/swscale_mmx.o: CFLAGS += $(NOREDZONE_FLAGS)
OBJS += x86/rgb2rgb.o \
x86/swscale.o \
x86/yuv2rgb.o \