| Age | Commit message (Collapse) | Author |
|---|
|
The stack a program may use will depend on the JIT: 2 KiB where the JIT
declares support for large stacks, 512 bytes elsewhere and for
interpreted programs. Tests that probe the limit therefore need to know
which one is in force. Add __load_if_large_stack() and
__load_if_no_large_stack() to test_loader, analogous to the JIT load
conditions, backed by a one-time probe that loads a program storing at
fp-2048. The probe caches only the verifier's verdict on that store: a
load that fails for another reason, such as a missing capability, is
reported and probed again on the next call.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260924165740.2146806-15-memxor@gmail.com
|
|
The verifier checks every stack access and the combined depth of a call
chain against MAX_BPF_STACK, which is also the frame size of the
interpreter and the frame that JITs without subprogram tail call
support set up for tail-call targets. A JIT that lays out frames of any
size and lets a tail-called program set up its own frame does not need
that limit; it only needs the verifier to bound how much stack a
program uses in total.
Add bpf_jit_supports_large_stack() for a JIT to claim that, and give
each program its budget through bpf_prog_stack_limit(): MAX_BPF_STACK_JIT
when the JIT is requested, the program is not offloaded and the JIT
supports large stacks as well as tail calls from subprograms,
MAX_BPF_STACK otherwise. The latter is what lets a tail-called program
set up its own frame: without it, do_misc_fixups() gives every program
with tail calls a MAX_BPF_STACK frame, which a deeper frame verified
against the larger budget would overrun. The verifier keeps the budget
in env->stack_limit and uses it for the bounds of fixed and variable
offset stack accesses, for unprivileged stack pointer arithmetic and its
speculation limit, and for the combined and private stack depth checks.
A frame may use any part of its program's budget. The interpreter paths
keep MAX_BPF_STACK: a program whose main frame is deeper falls back to
the JIT-required path of bpf_prog_select_runtime() and one with deeper
subprogram frames is rejected when patching calls for the interpreter.
The extra stack that may_goto and the timed may_goto instrumentation
add below a frame is, as before, not counted against the budget of a
JITed program and rejected past MAX_BPF_STACK for an interpreted one.
Stack liveness treats a read through a pointer of unknown offset, or a
call passing a frame pointer to a subprogram, as reaching the whole
frame, and widens the masks of that frame to the deepest half-slot such
a read can cover. Bound that by the program's budget too: no access
past it is accepted, so a program kept at MAX_BPF_STACK carries masks
of two words for such frames, as before, instead of the eight that
MAX_BPF_STACK_JIT needs. The three selftests matching a whole-frame
read in the liveness log accept either depth.
bpf_clone_redirect() transmits from inside the program, and a tc egress
or lwt_xmit program that redirects to its own device runs again on top
of its own frame until the datapath's recursion limit drops the packet,
ten frames deep. Ten MAX_BPF_STACK frames fit the kernel stack as they
always did; ten MAX_BPF_STACK_JIT frames would not, so a program that
calls bpf_clone_redirect() keeps MAX_BPF_STACK. The redirect helpers
that transmit after the program has returned leave no frame behind and
do not affect the budget. Nesting through other attach points is not
accounted, as before.
The spill tracker of the liveness analysis follows no slot past the
budget either.
The capability is a boolean and the budget a single constant, in the
style of the other bpf_jit_supports_*() queries, rather than a per JIT
size: the budget is meant to be the same everywhere it is raised, so
that programs verify identically across those architectures.
No JIT declares support yet, so every program keeps its 512-byte budget.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260924165740.2146806-14-memxor@gmail.com
|
|
The verifier keeps a few structures whose size follows the deepest
frame a program may have: the backtracking and scratched-slot bitmaps,
the jump history slot index and the clamp of the liveness masks. They
are all expressed through MAX_BPF_STACK_SLOTS, which derives from
MAX_BPF_STACK, the frame size of the interpreter.
Introduce MAX_BPF_STACK_JIT, the stack budget a program may get on a
JIT that can lay out frames of any size, and derive those structures
from it so that a frame may be as deep as that budget. Nothing grants
the budget yet, so no program verifies differently; the only visible
change is that the liveness log prints a whole-frame read up to the new
depth, so the three selftests matching such reads are updated.
The spill tracker of the liveness analysis keeps a table entry per
instruction and tracked slot, so it follows more than the 64 slots of a
MAX_BPF_STACK frame only while that table stays within what 64 slots
need for the largest program; a subprog of a million instructions keeps
64, one of a quarter million may track all 256. This bounds the table
at its old worst case of 640 MiB instead of letting a single deep store
push it past what kvmalloc() serves.
The backtracking and scratched-slot bitmaps grow from one to four
words per frame, a fixed few hundred bytes per verifier environment.
tmp_str_buf, which formats a frame's slot list for the log, grows from
320 to 1408 bytes so that all 256 slots still fit, and the log's line
buffer from 1 to 2 KiB so that a line built from it is not cut; the
environment stays within its 64 KiB allocation. The liveness masks are
only as wide as the stack a frame uses, so most frames cost the same as
before; a frame that is read as a whole, through a pointer of unknown
offset or by bpf_loop() with two callbacks, now carries masks of eight
words, 192 bytes per instruction per frame instead of 48. Measured over
the 5075 selftest programs, that is 0.2% of the total peak verifier
memory: strobemeta_bpf_loop and pyperf600_bpf_loop grow by 11% (1.2 MiB
and 0.6 MiB), a few dozen small programs by 40 to 100 KiB each,
everything else is unchanged. The next patch bounds such reads by the
program's budget, so this cost is only paid once a JIT grants it.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260924165740.2146806-13-memxor@gmail.com
|
|
The liveness masks of a function instance are as wide as the deepest
half-slot the instance was seen to access. When the same instance is
analyzed again through another call site, the new pass may have settled
on a different width, and merge_instances() has to widen the original
before combining the two. Add a test where the first pass of a callee
reads through a pointer 248 bytes into the main frame and the second
one through a pointer of unknown offset, which reads the whole frame,
and check that the merged result keeps the whole-frame read. The
precise read stays within the first mask word on 64-bit, so the
whole-frame pass is wider under every stack budget and the merge has
to widen the masks; a deeper read would need two words already, which
is all a 512-byte whole-frame read needs on 64-bit, and the widening
would go untested there.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260924165740.2146806-12-memxor@gmail.com
|
|
The stack liveness analysis records a store as a "def" only for the
4-byte half-slots it covers completely, so a one or two byte store near
the top of the frame must not define anything. Add a test that reads
fp-8, stores one byte at fp-1 and two bytes at fp-4, and reads fp-8
again, expecting no def mark on either store and the second read to
still use fp-8.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260924165740.2146806-11-memxor@gmail.com
|
|
A tail call from a subprog only unwinds that subprog's frame, so the
verifier refuses tail calls once the frames of the callers add up to
256 bytes or more. Nothing exercised that rule. Add a pair of tests
with a caller using 240 and 256 bytes of stack respectively, the
latter expecting the "tail_calls are not allowed when call stack of
previous frames is 256 bytes" rejection.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260924165740.2146806-10-memxor@gmail.com
|
|
Add tests where a helper stack buffer, or a load through a pointer into
another frame, overlaps the slots of a fastcall spill/fill pair. The
rewrite must not be applied in these cases, so check that the spill and
fill remain in the translated program and that the final stack depth
still covers the accessed slots. Cover:
- a constant-sized uninitialized output without CAP_PERFMON;
- the same output in the caller's stack, passed to a helper by a callee;
- a helper input whose only initialization is the fastcall spill;
- a callee load from the caller's fastcall spill slot.
Also add a zero-sized buffer, for which the rewrite must still be
applied.
The tests only load the programs and inspect the verifier log and the
translated instructions. Do not run them: without the fixes, the
verifier accepts programs that access memory outside their stack frame.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260924075352.2343553-3-memxor@gmail.com
|
|
Cross-merge BPF and other fixes after downstream PR.
Conflicts:
kernel/bpf/helpers.c
tools/testing/selftests/bpf/prog_tests/cb_refs.c
tools/testing/selftests/bpf/prog_tests/verifier.c
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
|
|
Add a bpftool raw BTF dump test covering LOC_PARAM, LOC_PROTO, and
LOCSEC types. Verify LOC_PARAM expressions, LOC_PROTO parameter values,
and LOCSEC function name, type id, and offset output.
Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260924111428.75957-11-alan.maguire@oracle.com
|
|
When creating distilled BTF, BTF_KIND_FUNC, _LOC_PARAM and _LOC_PROTO
should be added to split BTF. This means potentially some duplication
of location information, but only for out-of-tree modules that use
distilled base/split BTF.
Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260924111428.75957-7-alan.maguire@oracle.com
|
|
Ensure that location params/protos are deduplicated and location
sections are not, and that references to deduplicated locations within
location prototypes and sections are updated after deduplication.
Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260924111428.75957-6-alan.maguire@oracle.com
|
|
BTF_KIND_LOC[_PARAM|_PROTO|SEC] need to work with field iteration, so
extend the selftest to cover these and ensure iteration over all types
and names succeeds.
Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260924111428.75957-5-alan.maguire@oracle.com
|
|
Add support to dump and validate new location-related kinds.
Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260924111428.75957-4-alan.maguire@oracle.com
|
|
The existing cpu_flag subtests always prime a key with BPF_F_ALL_CPUS
before any BPF_F_CPU write, so the create path is never covered.
Add a subtest that creates the element with BPF_F_CPU on a map with
max_entries 1, so the key can only reuse the element the previous key
released, and check that the CPUs the update did not name read back
zero. Run it for PERCPU_HASH preallocated and BPF_F_NO_PREALLOC,
whose per-cpu areas come from different allocators, and for
LRU_PERCPU_HASH.
Under BPF_F_NO_PREALLOC the reuse is only guaranteed on the cpu that
ran the delete, so pin the thread across the pair, and name a cpu other
than that one in map_flags.
Signed-off-by: Donggeun Yoo <donggeunyoo.kernel@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260924102321.2120434-3-donggeunyoo.kernel@gmail.com
|
|
Extend tc_bpf2bpf with two freplace links, one on entry_tc and one on
subprog_tc, and detach the one on the entry: while subprog_tc is still
extended, updating entry_tc into a prog_array map must keep failing
with -EBUSY, and succeed again once the last link detaches. The test
asserts the rejection instead of running the prog, as the update
succeeds and the prog loops unbounded on an unfixed kernel.
Signed-off-by: Yuan Chen <chenyuan@kylinos.cn>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Link: https://lore.kernel.org/bpf/20260924023737.1140521-3-chenyuan_fl@163.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Add verifier_callx_rodata tests where pointers to functions are in
.rodata, libbpf stores offsets there and the kernel recognizes them:
- vtable-like data in .rodata and .data.rel.ro, one of two vtables picked
at run time, const and variable index, array of structs where
the index selects the pointer or the data, address of an element,
table without a symbol. These are executed.
- data next to a pointer is still a constant. The pointer is not:
bpf_prune_dead_branches() must not fold a branch on it.
- all functions the index can select are verified. The rest are not.
- index that may select non-pointer, partial and misaligned reads,
arithmetic on the pointer.
- pointer to a global function is not recognized.
- prog without callx reads the pointer as a number.
- recursion and stack depth via a table, stack depth of a callback
that is read from a table.
- effects of the functions in a table that call each other.
- CAP_BPF without CAP_PERFMON.
- C: array of functions, NULL check of an element, struct ops with
data, switch that may become a table, packed struct with misaligned
pointers in a prog without callx.
Add callx_func_ptr_map test that doesn't use libbpf logic: map that
another prog uses is not scanned, the prog that failed to load is not
a user, the offset in the map is replaced at load, the data is not, no
other prog including second instance of the same one can use the map
after that.
Add callx_rodata_lskel test: two progs with a table of functions in
.rodata and one without callx loaded by light skeleton, with .rodata
set by user space.
callx needs JIT. Skip the tests when it's off.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260924031042.1690890-17-alexei.starovoitov@gmail.com
|
|
Add verifier_callx tests:
- calls via r0, callee saved regs, pointer passed as argument, returned
from subprog, spilled/filled, different callees on different paths.
These are executed to test JIT.
- invalid operands: scalar, other pointers, modified and variable
PTR_TO_FUNC, global functions, reserved fields, JMP32.
- callx under lock.
- bounded and unbounded recursion via callx.
- stack depth of call chains through callx, including address taken
several frames above callx.
- tail calls in the callee and the caller of callx.
- stack liveness: caller slots read by callee stay alive.
- no const propagation across callx.
- precision backtracking through callx.
- address of a function that is never called, with and without callx.
- function pointers in C.
callx needs JIT. Add RUN_JITED() to skip the tests when it's off.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260924031042.1690890-16-alexei.starovoitov@gmail.com
|
|
Introduce BPF_JMP | BPF_CALL | BPF_X (opcode 0x8d) 'callx dst_reg'
instruction: indirect call of bpf subprog with address in dst_reg.
That's the encoding LLVM emits for calls via function pointer.
src_reg, off, imm are reserved and must be zero.
dst_reg must be PTR_TO_FUNC produced by ld_imm64 BPF_PSEUDO_FUNC.
check_ld_imm() allows it for static subprogs only, so callx cannot call
global subprogs or the main prog. Since every callee has its address
taken by ld_imm64, add_subprogs() and check_cfg() see all of them before
the main pass, and might_sleep, changes_pkt_data, might_throw of
the callee are already merged into the subprog that takes the address.
reg->subprogno is the callee. Verify callx as a direct call of that
static subprog: split check_func_call() into check_static_func_call()
that is shared with new check_func_callx(). Different paths through
the same callx may call different subprogs.
Arithmetic on PTR_TO_FUNC is allowed, so check that the pointer wasn't
modified. Allow callx while holding a lock like direct calls of static
subprogs.
The interpreter doesn't support callx. Set jit_required and add
bpf_jit_supports_callx() for JITs to opt in. No JIT does yet, so callx
is still rejected.
Print it as "callx rN" in the verifier log and xlated dump.
Adjust "invalid call insn1" test_verifier test that used opcode 0x8d as
unknown opcode. It fails with "R0 !read_ok" now.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260924031042.1690890-6-alexei.starovoitov@gmail.com
|
|
Add a test for recursion
callback -> static func -> global func -> bpf_loop() -> callback
that used to hang check_max_stack_depth().
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260924031042.1690890-3-alexei.starovoitov@gmail.com
|
|
cb_refs matches verifier messages such as "Unreleased reference id=4
alloc_insn=3". The ID is allocated from env->id_gen, so it changes
whenever the verifier assigns another ID earlier in the program.
In the bpf tree commit 71919742c83c ("bpf: Assign lock identity to
callback map values") gives each callback map value an ID, and the
nested_cb expectation was changed to id=5.
In the bpf-next tree, commit 5dc1549afac9 ("bpf: Consolidate release
argument validation") changed leak_prog to expect id=4 from the same
leak check.
The trees are merged in linux-next, and there leak_prog reports id=5
and the test fails [1]:
Expected: Unreleased reference id=4 alloc_insn=3
...
Unreleased reference id=5 alloc_insn=33
The ID is incidental. The test checks that the verifier rejects the
leak and which acquisition leaked.
Convert the test to test_loader, register the test in
prog_tests/verifier.c and drop unnecessary prog_tests/cb_refs.c file.
[1] https://github.com/kernel-patches/bpf/actions/runs/35923662147/job/107398076527
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Reviewed-by: Amery Hung <ameryhung@gmail.com>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://patch.msgid.link/20260923230717.3156345-1-ihor.solodrai@linux.dev
|
|
A child joins a memcg, fills it with 128M of clean page cache by reading
a sparse temp file (the way the cgroup selftests do), caps memory.max 8M
above its usage and then faults 64M of arena in, which only fits by
reclaiming that cache.
With the fix the arena fault-in reclaims, every fault succeeds and the
child exits 0. Without it the allocation cannot reclaim, fails once the
headroom is used up, and the child dies with SIGSEGV on a valid arena
address, so the test fails.
# test_progs -v -t arena_memcg
serial_test_arena_memcg:PASS:child faulted the arena in
#8 arena_memcg:OK
# without the fix
child killed by signal 11
serial_test_arena_memcg:FAIL:child faulted the arena in
The page cache must be reclaimable, so the temp file has to live on a
disk-backed filesystem, not tmpfs - the same assumption the cgroup
selftests make.
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://patch.msgid.link/20260922101831.192102-4-jiayuan.chen@linux.dev
|
|
cgroup_helpers has write_cgroup_file()/write_cgroup_file_parent() but no
read counterpart. Add read_cgroup_file() and read_cgroup_file_parent() so
a forked child can read a cgroup file (e.g. memory.current) from the work
dir owned by the parent that set the environment up, without hand-building
the /mnt/... path.
Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://patch.msgid.link/20260922101831.192102-3-jiayuan.chen@linux.dev
|
|
csum_partial() computes num_u16 = len >> 1 and only sums that many
16-bit words, so the last byte of an odd-length buffer never gets
added to the checksum. RFC 1071 says it should be padded with a zero
byte and summed as one more word, not dropped.
This backs build_ip_csum(), build_udp_v4_csum() and
build_udp_v6_csum(), used by flow_dissector_classification.c and
xdp_metadata.c to hand-build packets. No current caller builds an
odd-length payload, so nothing fails today, but a future one would
get a silently wrong checksum.
Also bump flow_dissector_classification's TEST_PACKET_LEN from 100 to
99 so this actually gets exercised instead of staying latent.
f4504af68575 wrote the len >> 1 division, but it only ever passed
sizeof(iphdr), always even, so it couldn't hit the bug. Tagging
bcc00987bc56 instead, since it added the first caller,
build_udp_v4_csum()/build_udp_v6_csum(), that can pass an odd length.
Verified with test_progs under vmtest.sh: without the fix, an odd
TEST_PACKET_LEN makes the kernel drop the packet over a bad checksum
and flow_dissector_classification fails; with the fix, both
flow_dissector_classification and xdp_metadata pass.
Signed-off-by: Madhav Khosla <madhav.khoslaa@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260920085549.867099-1-madhav.khoslaa@gmail.com
|
|
Run the same functional test against the tasks trace flavour.
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260922200208.3203834-5-puranjay@kernel.org
|
|
Cover the callback running after a grace period with the right map, key
and value, -EBUSY on a second arm, reuse of the head once disarmed, a
callback arming itself again, and teardown with a callback queued.
struct bpf_rcu_head is not the first member of the map value, so the
callback's recovery of the value from the head is exercised. arm()
wraps both arms in an RCU read section, otherwise a grace period may
elapse between them and the second one legitimately succeeds.
Negative tests: a hash map created with the same BTF, the map used as an
inner map, and an iterator attach, all checked for -EOPNOTSUPP; plus
verifier rejection of a mismatched map, a map with no bpf_rcu_head, a
head at the wrong offset, a head on the stack, and a sleepable callback.
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260922200208.3203834-3-puranjay@kernel.org
|
|
Add a selftest to confirm the verifier rejects ALU operations
that return arena or non-arena results depending on code path.
Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260922172028.6269-9-emil@etsalapatis.com
|
|
Add a selftests that ensures that PTR_TO_PACKET arguments can
only be passed to subprogs that will never adjust the underlying
packet memory, and are rejected otherwise.
Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260922172028.6269-7-emil@etsalapatis.com
|
|
Add tests to ensure the verifier properly tracks the 0 bit state
and width of the rx_queue_mapping field read from struct sock.
Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260922172028.6269-5-emil@etsalapatis.com
|
|
Add a selftest to ensure dynptr slices cannot include
past the end of the linear area of an skb.
Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260922172028.6269-3-emil@etsalapatis.com
|
|
Networking used to print the socket pointer with %p as output in the
/proc interface. This was later changed to %pK in order not to reveal
the actual pointer value. The output format has then been copied to bpf
tests which produce the same format including the %pK format modifier.
Networking recently replaced the socket output with a plain 0 because
the socket pointer added no value to the output and the %pK should be
removed from library handling. The format remained otherwise unchanged
in order not to break any tools which parsing this information. This
was done via 7c0ec6288b49 ("net: Replace %pK output with 0").
This change removes the %pK modifier, following the change in networking.
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20260918102610.gxA3km8_@linutronix.de
|
|
Produce known ring buffer records and check complete plain, JSON and
pretty JSON output. Exercise ID and pinned map selection, SIGINT and
SIGTERM shutdown, empty streams and invalid map types or selectors.
Also produce a perf event sample and check its existing header and raw
payload output. Use a payload whose size plus the raw sample length
field is aligned to eight bytes so the expected bytes exclude implicit
perf padding.
Signed-off-by: Tianyi Chen <hi@tychen.cc>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260921073556.99421-3-hi@tychen.cc
|
|
Add a verifier regression test that initializes a numeric iterator at fp-8
and attempts to destroy it through fp+0. The verifier must reject the
non-negative offset instead of treating it as the initialized stack slot.
Check the offset diagnostic to ensure rejection happens at the stack
object address check. The numeric iterator destroy operation is a no-op;
this test checks verifier rejection and does not run the program.
Signed-off-by: Xu Yunxiang <xyx2021@mail.ustc.edu.cn>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Reviewed-by: Sun Jian <sun.jian.kdev@gmail.com>
Link: https://lore.kernel.org/bpf/20260920210423.345636-3-xyx2021@mail.ustc.edu.cn
|
|
task_work_run() forks a child that blocks reading a pipe until the
parent wakes it. If task_work__open() fails afterwards, the parent
returns without closing either pipe fd or waiting for the child.
Because the parent retains the write end, the child remains blocked in
read() until test_progs exits.
Route this failure through the common cleanup path. At this point
cleanup is safe: pe_fd is -1, link is NULL, task_work__destroy() accepts
NULL, and the pid > 0 branch closes the read end, wakes the child,
closes the write end and waits for it.
This is the last early return after a successful fork. The fork failure
path already closes both pipe fds after commit 5730dacb3f17
("selftests/bpf: Task_work selftest cleanup fixes").
Fixes: 39fd74dfd5d2 ("selftests/bpf: BPF task work scheduling tests")
Signed-off-by: Yun Lu <luyun@kylinos.cn>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Mykyta Yatsenko <yatsenko@meta.com>
Link: https://lore.kernel.org/bpf/20260918024300.18321-1-luyun_611@163.com
|
|
test_fexit_bpf2bpf_common() can jump to the common cleanup path before
the link array is allocated, for example if bpf_prog_get_info_by_fd()
fails. The cleanup loop still indexes link[i] unconditionally, which can
dereference a NULL pointer and hide the original failure.
Guard the loop so the test reports the original failure instead.
Signed-off-by: Zhixing Chen <running910@gmail.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260917093928.48495-1-running910@gmail.com
|
|
Check that fixed-size and sized helper input/output buffers require both
read and write permission. Cover the MTU and FIB helpers, including read-only
and write-only rejection and read/write positive controls. Exercise the XDP
prototypes and the sock_ops header-option input/output argument as well.
Keep write-only maps usable as destinations for partial-output helpers
bpf_snprintf() and bpf_sysctl_get_name(), while rejecting a read-only
snprintf destination. Also retain a write-only-map destination for
bpf_get_current_comm(), whose output is annotated MEM_UNINIT and fully
initialized by the helper.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260921023843.411943-12-memxor@gmail.com
|
|
Exercise generic output buffers with and without CAP_PERFMON, using both
helpers and __uninit kfuncs. Check that initialized bytes stay readable
and lose stale value information, while invalid bytes remain unreadable
without permission to read uninitialized stack memory. Cover constant and
variable sizes, scalar spills, pointer spills, special stack objects, and
privileged variable offsets.
Add a kfunc that writes only the first byte of its output. Its runtime tests
read preinitialized bytes, checking both the written byte and an untouched
tail byte across a liveness checkpoint. Verify that the sysctl name helper
and uninitialized fixed and variable-sized kfunc outputs are accepted when
their contents are not read back.
Update existing __uninit readback expectations and exercise skb_load_bytes
with reduced capabilities. Even fully-writing generic outputs now preserve
invalid bytes in the verifier, so reading those bytes requires prior
initialization by the BPF program.
Use map_update_elem inputs for helper_arg_fallback_keeps_scanning. Its
original snprintf argument no longer reads the buffer, and a privileged
output with an unknown size does not trigger the whole-stack read fallback.
A variable-offset key and parent-frame value retain the intended assertion.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260921023843.411943-10-memxor@gmail.com
|
|
Keep coverage for per-slot output tracking separate from the immediate
single-output regression tests. Check that both constant-size outputs are
initialized, and that a variable-size output does not disable initialization
of an independent constant-size output.
Also exercise an output following a by-value parameter that occupies two
argument slots, and an output pointer passed on the stack. Run each case
with normal capabilities and with CAP_BPF and CAP_NET_ADMIN only. Leave the
stack-passed output uninitialized so its reduced-capability case fails if
the verifier treats it as an ordinary input buffer.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260921023843.411943-8-memxor@gmail.com
|
|
Exercise the struct and sized-buffer cases where stack liveness poisons an
output before a kfunc call. Check that the verifier accepts these outputs
and that the kfunc initializes the memory read after the call.
Verify that an uninitialized input aliasing an output is still rejected
without CAP_PERFMON or CAP_SYS_ADMIN. Include an initialized alias as a
positive control, using an int-width store so its value is independent of
endianness.
Use __prepare_priv to resolve the test module's BTF before dropping to
CAP_BPF and CAP_NET_ADMIN for program loading. Keep multiple-output and
argument-slot coverage separate from these immediate regression tests.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Reviewed-by: Amery Hung <ameryhung@gmail.com>
Link: https://patch.msgid.link/20260921023843.411943-6-memxor@gmail.com
|
|
The annotation-driven loader drops capabilities before libbpf prepares an
object. Resolving bpf_testmod kfuncs requires CAP_SYS_ADMIN to enumerate and
open module BTF, so tests without that capability fail before reaching the
verifier.
Add an opt-in __prepare_priv annotation. Call bpf_object__prepare() with the
fixture's initial capabilities, then apply __caps_unpriv before loading the
programs. This uses libbpf's explicit prepare/load boundary. In particular,
CAP_SYS_ADMIN must be dropped along with CAP_PERFMON to test uninitialized
stack checks, since CAP_SYS_ADMIN satisfies the verifier's CAP_PERFMON check.
Preparation also creates maps and loads BTF. Keep it opt-in so existing tests
continue checking those operations with reduced capabilities. The existing
pre-execution callback runs after program loading and is too late for this.
Allow tests retaining CAP_BPF to run when the unprivileged-BPF sysctl is set.
Check CPU mitigations separately: disabled or undetectable mitigations must
still skip these tests, because CAP_BPF does not restore speculative
execution checks.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260921023843.411943-2-memxor@gmail.com
|
|
Add selftests covering selective kernel module BTF loading through
bpf_object_open_opts.
The tests verify that:
- the existing behavior is preserved when the allowlist is not
specified;
- loading succeeds when the required module BTF is specified in the
allowlist;
- module BTFs not in the allowlist are skipped;
- an empty allowlist skips loading all module BTFs;
- invalid allowlist and module name inputs are rejected.
Signed-off-by: Fuyu Zhao <zhaofuyu@vivo.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260915124104.77287-3-zhaofuyu@vivo.com
|
|
Replace DEFINE_TEST_RUNNER/DEFINE_TEST_RUNNER_RULES with
Makefile.runner, invoked once per test runner instance. Each invocation
uses ordinary make rules in a single-flavor namespace.
The main Makefile owns shared build outputs and the kselftest
run/install rules. Makefile.skel supplies the BPF object and skeleton
rules used by both the main Makefile and the runners. The main Makefile
builds the default flavor before invoking unflavored runners; flavored
runners build in separate directories.
Build shared userspace objects once and link them into every flavor.
Order their compilation after the bpftool sub-build, which installs the
libbpf-internal headers they include. Pass the assembled CFLAGS and
LDFLAGS to runners on their command lines.
Build and copy runtime fixtures alongside runner compilation so module
builds do not delay runner startup. They are prerequisites of the
test_progs, test_progs-<flavor> and all goals rather than of the runner
binaries, so a goal spelled as a binary's path builds the binary alone.
Installation copies the default flavor's BPF objects, preserving the
previous result.
Generate prog_tests/tests.h and map_tests/tests.h through ordinary
recipes, avoiding generation during make -n. Signed skeletons explicitly
depend on the private key. The flavored runners' objects and the test
objects are targets of the sub-makes only, and the bare linked-object
names (make linked_funcs1.bpf.o) are no longer targets. Build-log
details change: skeleton messages adopt the common format and output
stream, the shared objects log as CC, the fixture copy prints no
EXT-COPY line and TEST-HDR carries no runner tag.
Co-developed-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Mykola Lysenko <nickolay.lysenko@gmail.com>
Link: https://lore.kernel.org/bpf/20260921075855.2065871-10-nickolay.lysenko@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Move shared build definitions into Makefile.buildvars, preserving their
order, in preparation for separate runner sub-makes.
Include it after ../lib.mk and before Makefile.feature, so the LLVM
feature probe still sees srctree. Keep the CFLAGS and LDFLAGS additions
before lib.mk to preserve flag ordering; their references to definitions
below the include expand when used. Name the clang warning suppression
CLANG_WARN_CFLAGS so it can be referenced there.
The arena-ASAN probe moves below lib.mk, so it queries the CLANG the BPF
objects are built with. The CPU-v4 probe, BPF_GCC and TEST_KMODS remain
before lib.mk because they determine the target lists. Definitions no
runner reads move too when they are declared beside ones a runner does.
BPFTOOLDIR, HOST_BPFOBJ and BPF_TARGET_ENDIAN stay in the Makefile,
which alone reads them, and follow the include because their :=
assignments read values it defines.
Suggested-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Mykola Lysenko <nickolay.lysenko@gmail.com>
Link: https://lore.kernel.org/bpf/20260921075855.2065871-9-nickolay.lysenko@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Four skeleton recipes duplicate the link, determinism-check and
header-generation pipeline inside DEFINE_TEST_RUNNER_RULES. Extract it
into gen_bpf_skel.sh, with options for light and signed skeletons.
Intermediate names now derive from the output header, retaining separate
linked/llinked infixes for regular and light skeletons.
Keep the permissive-mode missing-input guards in a skip_if_missing
helper, so a skipped skeleton still prints only SKIP-SKEL. On failure,
the script removes intermediates and both output headers; previously,
strict-mode recipes left intermediates behind and .DELETE_ON_ERROR
covered only the target, not its subskeleton.
The determinism check names the skeleton when it fails, and linked
skeletons log GEN-SKEL before linking.
Suggested-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Mykola Lysenko <nickolay.lysenko@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260921075855.2065871-8-nickolay.lysenko@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
bench links every benchs/bench_*.c object. Derive the object list with a
wildcard so adding a benchmark no longer requires updating the link
rule. Skeleton prerequisites remain explicit.
The sorted list changes object link order and the binary's symbol
layout; no benchmark behaves differently.
Signed-off-by: Mykola Lysenko <nickolay.lysenko@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260921075855.2065871-7-nickolay.lysenko@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
The verifier/tests.h recipe is a $(shell ...) expansion: the command
runs while make expands the recipe line - including under make -n - its
exit status is discarded, and the resulting (empty) expansion is what
make actually executes.
Signed-off-by: Mykola Lysenko <nickolay.lysenko@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260921075855.2065871-6-nickolay.lysenko@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
genkey produces the private key and certificate together, but an
ordinary multi-target rule can run it twice concurrently when both
outputs are required. Only the certificate is currently a prerequisite;
the runner split will require both.
Use an implicitly grouped pattern rule, as test_kmods already does, to
support make versions before 4.3.
Signed-off-by: Mykola Lysenko <nickolay.lysenko@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260921075855.2065871-5-nickolay.lysenko@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
With BPF_STRICT_BUILD=0, eleven recipes append the same "|| { remove the
target, print a SKIP marker, report success }" tail, each spelled out
inline. Factor the tail into skip_on_fail; every call site keeps its
exact message and behavior.
Signed-off-by: Mykola Lysenko <nickolay.lysenko@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260921075855.2065871-4-nickolay.lysenko@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Four target-specific lines name objects nothing builds.
Commit afef88e65554 ("selftests/bpf: Store BPF object files with .bpf.o
extension") left three naming the old BPF objects; flow_dissector_load.o
names an intermediate the one-step compile and link rule does not
produce.
The xsk_xdp_progs, xdp_hw_metadata and xdp_features dependency-map
entries are unused: none is listed in LINKED_SKELS, and the regular
skeleton rule does not consult the map.
The test_l4lb_noinline and test_xdp_noinline '-fno-inline' settings had
already stopped taking effect with commit 74b5a5968fe8 ("selftests/bpf:
Replace test_progs and test_maps w/ general rule"). The compile recipe
uses TRUNNER_BPF_CFLAGS, a simply-expanded copy of BPF_CFLAGS that the
target-specific additions cannot affect. The intended functions already
carry noinline annotations; restoring the flag produces byte-identical
objects.
Restore flow_dissector_load.h as a prerequisite of the binary, which is
compiled and linked directly from its .c file. Move
cgroup_getset_retval_hooks.h under progs/, where the BPF rules' blanket
header prerequisite tracks it. Its userspace consumer remains tracked by
compiler-generated dependencies.
Drop the redundant CURDIR assignment and the unused OBJCOPY definition.
Signed-off-by: Mykola Lysenko <nickolay.lysenko@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260921075855.2065871-3-nickolay.lysenko@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
The generic '$(OUTPUT)/%:%.c' rule links with '$(LINK.c) $^', so every
prerequisite reaches the compiler driver. A header argument makes clang
fail with "cannot specify -o when generating multiple output files".
Filter headers out of the link command while retaining them as
prerequisites.
Signed-off-by: Mykola Lysenko <nickolay.lysenko@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/bpf/20260921075855.2065871-2-nickolay.lysenko@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Exercise bpf_inode_init_xattr() in combination with a policy example
via BPF LSM. A program on the inode_init_security hook labels new files
and directories, inherits a zone label from the parent directory, and
has claims refused for names outside the security.bpf. prefix and for
a name that would exceed XATTR_NAME_MAX once the prefix is put back as
well as other corner case tests that should get rejected.
# LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t lsm_inode_init_xattr
[...]
#226/1 lsm_inode_init_xattr/init_labels:OK
#226/2 lsm_inode_init_xattr/inherit_from_parent:OK
#226/3 lsm_inode_init_xattr/refused_claims:OK
#226/4 lsm_inode_init_xattr/null_xattrs:OK
#226/5 lsm_inode_init_xattr/shared_budget:OK
#226/6 lsm_inode_init_xattr/value_shapes:OK
#226 lsm_inode_init_xattr:OK
Summary: 1/6 PASSED, 0 SKIPPED, 0/0 FAILED
Co-developed-by: David Windsor <dwindsor@gmail.com>
Signed-off-by: David Windsor <dwindsor@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260915150739.284189-9-daniel@iogearbox.net
|
