Merge branch 'bpf-one-may_goto-time-slice-per-prog-invocation'

Alexei Starovoitov says:

====================
bpf: One may_goto time slice per prog invocation

From: Alexei Starovoitov <ast@kernel.org>

may_goto stops a loop that the verifier cannot bound: after 250 ms the loop
must exit. The count and the timestamp are in the stack frame of the
function that contains the loop. So the limit is for one call of a subprog,
not for one invocation of a prog. Loops that nest across calls multiply it,
and 9 insns lock up a cpu:

  main:                 sub:
  1: call sub           1: may_goto 2f
     may_goto 2f           goto 1b
     goto 1b            2: r0 = 0
  2: r0 = 1                exit
     exit

This series uses one count and one timestamp for all may_goto of a prog, in
per-cpu memory of the prog. The main prog sets them in its first insns.
When the time is up each loop ends at its next may_goto, in any function.

There is nothing in the stack frame any more, and nothing at the entry or
at an exit of a subprog. The C code of may_goto, helpers, kfuncs and
trampolines do not change. The callers of async callbacks save and restore
count and timestamp.

 - Patch 1: the interpreter can run a prog with the timed form of may_goto.
   Then the kernel crashes. This fix does not depend on the other patches.
 - Patches 2-4 change three selftests that rely on a time slice for each
   frame.
 - Patch 5 adds per-cpu insns to the JIT of s390. It is the only JIT with
   timed may_goto that doesn't have them.
 - Patch 6 passes a pointer to arch_bpf_timed_may_goto() in place of an
   offset in the stack frame. It removes code from six archs.
 - Patch 7 is the change.
 - Patch 8 ends bpf_loop() and bpf_for_each_map_elem() when the time is up.
 - Patch 9 has the tests.

What changes for a prog:
 - The 250 ms are for all loops of an invocation, not for each frame.
 - When the time is up, a function that is called after that makes no
   iteration. Today each call has a new time slice.

Count and timestamp are per-cpu, so a second invocation of the same prog on
the same cpu shares them with the first one. The error is on the long side:
the first one gets more time. None is stopped before it is 250 ms old.

All progs with may_goto get different code: 56 of 493 progs of scx, none of
Cilium. The series adds 1.8% to their JITed code, and they use 25% less
stack. On x86-64 (Zen 5) a call of a function with may_goto is cheaper than
before, and an iteration of an empty loop costs 0.09 nsec more.

Tested on x86-64: test_progs, test_progs-no_alu32, test_progs-cpuv4,
test_progs-bpf_gcc and test_verifier have no new failure. The same is true
for test_progs with KASAN, and for test_progs and test_verifier with
mitigations on. On arm64 the new tests pass in QEMU. On s390 test_progs did
not run: progs for the per-cpu insn and for may_goto pass in QEMU. riscv,
loongarch and powerpc are compile tested only.

v1 was patches 7 and 8 of [1]. I dropped them from v2 of that series [2].
v1 -> v2:
 - v1 also had a count in per-cpu memory. A subprog took it at its entry
   and gave it back at each exit. The insns at the exit changed R2, which
   is a part of a 16-byte return value (Ihor). Now each may_goto uses the
   per-cpu count, and there are no insns at the entry and at exits.
 - Each read of the clock walked the stack to find the prog. Now may_goto
   has the address.
 - Add patches 1-6 and 8.

[1] https://lore.kernel.org/bpf/20261026025943.404450-1-alexei.starovoitov@gmail.com/
[2] https://lore.kernel.org/bpf/20261027190314.1440199-1-alexei.starovoitov@gmail.com/

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
====================

Link: https://patch.msgid.link/20261031215543.682284-1-alexei.starovoitov@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>

0 files changed, 0 insertions, 0 deletions