bpf, x86: Swap 16-bit values with bswap and shr, not ror and movzx

The 16-bit byte swap was emitted as a 16-bit rotate followed by a zero
extending move. bswap of the 32-bit register followed by a right shift
by 16 has the same result, is two bytes shorter, and avoids the operand
size prefix and the 16-bit partial register write. Both sequences are
two single cycle uops on Intel and AMD. LLVM emits be16 for bpf_htons()
and bpf_ntohs() with cpu v3 and bswap16 with cpu v4, both of which
take this path.

Before, for r1 = be16 r1:

  66 c1 cf 08  rorw $0x8,%di
  0f b7 ff     movzwl %di,%edi

After:

  0f cf        bswap %edi
  c1 ef 10     shrl $0x10,%edi

Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20261013171230.523677-10-daniel@iogearbox.net
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>

1 files changed, 7 insertions, 9 deletions

diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index 1c84d2a02aba..50158d4b85b2 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -2596,18 +2596,16 @@ static int do_jit(struct bpf_verifier_env *env, struct bpf_prog *bpf_prog, int *
 		case BPF_ALU64 | BPF_END | BPF_FROM_LE:
 			switch (imm32) {
 			case 16:
-				/* Emit 'ror %ax, 8' to swap lower 2 bytes */
-				EMIT1(0x66);
+				/* Emit 'bswap eax' to swap lower 4 bytes */
 				if (is_ereg(dst_reg))
 					EMIT1(0x41);
-				EMIT3(0xC1, add_1reg(0xC8, dst_reg), 8);
+				EMIT2(0x0F, add_1reg(0xC8, dst_reg));
 
-				/* Emit 'movzwl eax, ax' */
-				if (is_ereg(dst_reg))
-					EMIT3(0x45, 0x0F, 0xB7);
-				else
-					EMIT2(0x0F, 0xB7);
-				EMIT1(add_2reg(0xC0, dst_reg, dst_reg));
+				/*
+				 * Emit 'shr eax, 16' to move the swapped lower
+				 * 2 bytes back down and zero extend
+				 */
+				emit_shift_imm(&prog, BPF_RSH, dst_reg, 16, false);
 				break;
 			case 32:
 				/* Emit 'bswap eax' to swap lower 4 bytes */