lavc/dxv: fix incorrect back-reference index calculation in DXT5 decoding

This bug causes the DXT5 decoder to produce incorrect block texture data.
After the fix, textures are visually correct and match data decoded by
Resolume Alley (extracted with Nvida Nsight for comparison). Current FATE DXT5
samples did not cover this case.

Signed-off-by: Connor Worley <connorbworley@gmail.com>
This commit is contained in:
Connor Worley
2024-01-29 22:26:26 -08:00
committed by Lynne
parent 3b6a515c5f
commit 1eeee68d8e

View File

@@ -955,7 +955,7 @@ static int dxv_decompress_dxt5(AVCodecContext *avctx)
break;
case 2:
/* Copy two dwords from a previous index */
idx = 8 + bytestream2_get_le16(gbc);
idx = 8 + 4 * bytestream2_get_le16(gbc);
if (idx > pos || (unsigned int)(pos - idx) + 2 > ctx->tex_size / 4)
return AVERROR_INVALIDDATA;
prev = AV_RL32(ctx->tex_data + 4 * (pos - idx));