bpf: Treat load and store through a number as arena access

There are no address spaces in Rust. LLVM emits plain loads and stores
for arena memory, without cast_kern:

  r3 = 0x20 ll            // R_BPF_64_64 .bss, libbpf puts it into arena
  r2 = *(u64 *)(r3 + 0)   // R3 invalid mem access 'scalar'
  lock *(u64 *)(r2 + 8) += r1

Add BPF_F_ARENA_SCALAR flag of BPF_PROG_LOAD. When the program is loaded
with it and has an arena treat ldx, stx, st and atomics through a number
as arena access. It's as safe as access through PTR_TO_ARENA: JIT adds
the base of arena to the low 32 bits of the address. The program has
an arena only with CAP_BPF and CAP_PERFMON.

Registers of the program are not changed, since Rust compares and
stores the address after the access. bpf_do_misc_fixups() copies
the low 32 bits into BPF_REG_AX and the access goes through it:

  r2 = *(u64 *)(r3 + 0)  ->  w12 = w3
                             r2 = *(u64 *)(r12 + 0)

Constant blinding needs BPF_REG_AX for immediates and leaves insns that
use BPF_REG_AX alone. So the immediate of st through a number is not
blinded, the rest of the program is:

  *(u64 *)(r3 + 0) = 1   ->  w12 = w3
                             *(u64 *)(r12 + 0) = 1

The same insn may see PTR_TO_ARENA on another path. It works for both.

It's a flag, since a number is also what the verifier makes of a pointer
that went away when the program has CAP_PERFMON: a ringbuf record after
bpf_ringbuf_submit(), a pointer to the packet after bpf_skb_pull_data().
Programs in C that have an arena keep "invalid mem access 'scalar'" for
such bugs.

JIT tells with bpf_jit_supports_arena_scalar() that it takes BPF_REG_AX
as the address of arena access. With other JITs the flag is rejected
with -EOPNOTSUPP. x86 and arm64 do:
- x86: the fault handler finds the register that holds the address
  through reg2pt_regs[]. Add BPF_REG_AX, r10 of x86, there.
- arm64: nothing else is needed. The JIT takes any register as the
  address and the fault handler reads it by its number. Compile tested
  only.

Any number is an address of arena, NULL and small numbers included, like
it is for PTR_TO_ARENA: arena code in C dereferences NULL and relies on
the fault being handled.

Still rejected:
 - insn that sees a number on one path and a pointer that is not
   PTR_TO_ARENA on another.
 - numbers passed to helpers and kfuncs, except __arena arguments.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20261002124714.180012-4-alexei.starovoitov@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>

1 files changed, 6 insertions, 0 deletions

diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h
index 4687c3310996..e0ed44b1bbcb 100644
--- a/include/uapi/linux/bpf.h
+++ b/include/uapi/linux/bpf.h
@@ -1344,6 +1344,12 @@ enum bpf_perf_event_type {
 /* The verifier internal test flag. Behavior is undefined */
 #define BPF_F_TEST_REG_INVARIANTS	(1U << 7)
 
+/*
+ * Load and store through a number is an access to the arena of the program
+ * at the low 32 bits of the number. It's for programs written in Rust.
+ */
+#define BPF_F_ARENA_SCALAR	(1U << 8)
+
 /* link_create.kprobe_multi.flags used in LINK_CREATE command for
  * BPF_TRACE_KPROBE_MULTI attach type to create return probe.
  */