summaryrefslogtreecommitdiff
path: root/kernel/bpf
AgeCommit message (Collapse)Author
14 daysbpf: Fix missing migration protection in __rhtab_map_lookup_and_delete_batch()Ömer Mete Kaya
bpf_mem_cache_free_rcu() uses this_cpu_ptr() which requires migration to be disabled. All callers of rhtab_delete_elem() disable migration except __rhtab_map_lookup_and_delete_batch(), which calls it under rcu_read_lock() only. On CONFIG_PREEMPT_RCU, rcu_read_lock() does not disable preemption or migration, so the task can migrate between CPUs during the delete loop, causing this_cpu_ptr() to trigger: BUG: using smp_processor_id() in preemptible [00000000] code Fix by wrapping the delete loop in migrate_disable()/migrate_enable() in __rhtab_map_lookup_and_delete_batch(), matching the migration protection that the other callers already provide. Fixes: 818e00848227 ("bpf: Implement iteration ops for resizable hashtab") Reported-by: syzbot+fd7e415d891073b83e1f@syzkaller.appspotmail.com Signed-off-by: Ömer Mete Kaya <omermetekaya0@gmail.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Acked-by: Mykyta Yatsenko <yatsenko@meta.com> Link: https://patch.msgid.link/20260929081609.557899-1-omermetekaya0@gmail.com Closes: https://syzkaller.appspot.com/bug?extid=fd7e415d891073b83e1f
2026-10-01bpf: Fix objects stuck in free_by_rcu_ttraceAlexei Starovoitov
do_call_rcu_ttrace() returns early when call_rcu_ttrace_in_progress is set and leaves the objects in free_by_rcu_ttrace. __free_rcu() frees waiting_for_gp_ttrace only and clears the flag. Hence the objects that free_bulk() or __free_by_rcu() added while RCU tasks trace GP was in flight stay in free_by_rcu_ttrace until free_bulk() or alloc_bulk() is called for the same bpf_mem_cache again, which may never happen. The number of such objects is not bounded. Turn call_rcu_ttrace_in_progress into three states: 0 - idle 1 - __free_rcu() is queued 2 - __free_rcu() is queued and free_by_rcu_ttrace got more objects since do_call_rcu_ttrace() sets 2. __free_rcu() does cmpxchg(1 -> 0) and starts the next GP when it fails. It cannot clear the flag first and check free_by_rcu_ttrace later, since bpf_mem_alloc_destroy() frees bpf_mem_cache without waiting for RCU callbacks when the flag is zero. Now __free_rcu() queues itself, so the one that didn't see 'draining' may do call_rcu_tasks_trace() after rcu_barrier_tasks_trace() in free_mem_alloc(). Queue it under rcu_read_lock() and do synchronize_rcu() before the barriers. Calling rcu_barrier_tasks_trace() twice works too, but creating and destroying hash maps in a loop on many cpus slows down to one free_mem_alloc() per GP and kworkers pile up. Fixes: 8d5a8011b35d ("bpf: Batch call_rcu callbacks instead of SLAB_TYPESAFE_BY_RCU.") Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://lore.kernel.org/bpf/20260930095920.601738-3-alexei.starovoitov@gmail.com Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-10-01bpf: Factor out __do_call_rcu_ttrace()Alexei Starovoitov
Move the part of do_call_rcu_ttrace() that runs after call_rcu_ttrace_in_progress is set into __do_call_rcu_ttrace(). The next patch will call it from __free_rcu(). No functional change. Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://lore.kernel.org/bpf/20260930095920.601738-2-alexei.starovoitov@gmail.com Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-10-01bpf: Fix packet range of pointers sharing an idAlexei Starovoitov
Since commit 022ac0750883 ("bpf: use reg->var_off instead of reg->off for pointers"), find_good_pkt_pointers() sets the range of all packet pointers sharing an id from the umax of the compared pointer, and check_packet_access() requires umax + off + size <= range. That assumes the umax of two such pointers differ by exactly their constant distance. reg_bounds_sync() breaks it when var_off tightens one umax and not the other: r4 &= 0x38 if r4 > 50 goto exit ; umax 50, var_off (0x0; 0x38) r5 = pkt + r4 ; umax 50 r6 = r5 r6 += 8 ; umax 56, not 58 Comparing r6 with pkt_end sets the range to 56, and the valid 8-byte load at r5 is rejected (50 + 8 > 56). Comparing r5 sets it to 50, and the out-of-bounds 1-byte load at r6 - 7, i.e. r5 + 1, is accepted (56 - 7 + 1 <= 50). Don't call reg_bounds_sync() on a packet pointer that keeps its id (a constant was added or subtracted) or its range (an unknown non-negative value was subtracted), so that var_off cannot tighten its umax. Only update the 32-bit bounds from var_off: reg_bounds_sanity_check() wants them constant when the lower half of var_off is, e.g. for pkt + 8. This relies on nothing else changing the 64-bit bounds of a packet pointer, which holds today. var_off of such a pointer is no longer narrowed by its bounds. Adjust three verifier_align expectations; the low bits, which the alignment checks use, don't change. veristat on the selftests shows no verdict changes and +0.8% insns in test_cls_redirect_subprogs. Fixes: 022ac0750883 ("bpf: use reg->var_off instead of reg->off for pointers") Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://lore.kernel.org/bpf/20261001145255.855630-1-alexei.starovoitov@gmail.com Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-09-29bpf: Skip detached progs in trampoline images that are still in useFlorent Revest (Anthropic)
bpf_tramp_image_put() makes sure a trampoline image is not freed while a task may still be running in it, but nothing similar is done for the progs called by that image. Detach drops the last prog reference right away and the prog is freed after grace periods, on the basis that a task still in the traced function skips the fexit progs once the nop at ip_after_call is patched to a jump. That leaves out a task sleeping in a sleepable prog that runs before the detached one, which no grace period waits for: CPU 0 CPU 1 in image I, sleeping in prog S detach P from I's trampoline -> new image, bpf_tramp_image_put(I) bpf_prog_put(P), last ref grace periods, P freed back from S __bpf_prog_enter(P) call P->bpf_func If S and P are fexit progs the task is already past the patched jump, and fentry only images don't have one. On x86 this is an int3 in poisoned bpf_prog_pack memory: Oops: int3: 0000 [#1] SMP NOPTI CPU: 18 UID: 0 PID: 94573 Comm: x169 Not tainted 6.18.44 #1 PREEMPT(lazy) RIP: 0010:0xffffffffc0601d8d Call Trace: <TASK> ? bpf_trampoline_6442515411+0x1a4/0x21b bpf_lsm_bprm_committed_creds+0x5/0x10 security_bprm_committed_creds+0x5f/0x70 begin_new_exec+0x2d6/0x410 ... We hit this in production when progs attached through trampolines got detached while their hooks were busy, and it was independently found with a fuzzer and KASAN. Have the JITs emit a patchable nop in front of each prog call sequence and record it in the image. When a prog is detached, patch its nop to a jump over the call sequence. Progs that stay attached keep running for the tasks that are in the image, and ip_after_call isn't needed anymore. The task can be in any image that isn't freed yet, not only in the current one. It sleeps in image I1 that calls S, P and Q, then P is detached and the trampoline moves to image I2, then Q is detached and its call is still in I1. So the trampoline keeps a list of its images until they are freed, and detaching a prog patches its nop in all of them. Images hold a reference on the trampoline for that long. On riscv and loongarch a jump of any range takes several instructions, and a task preempted in the middle of them could resume into half of the new sequence. The nop is a single instruction there, patched to a near branch through arch_bpf_trampoline_skip(). With the extra nops, BPF_MAX_TRAMP_LINKS progs no longer fit in a page on x86 and arm64 (and already didn't on powerpc), so lower the limit there like s390 does. Fixes: e21aa341785c ("bpf: Fix fexit trampoline.") Reported-by: Sechang Lim <rhkrqnwk98@gmail.com> Suggested-by: Alexei Starovoitov <ast@kernel.org> Signed-off-by: Florent Revest (Anthropic) <florent.revest@linux.dev> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20260926135605.1217928-3-florent.revest@linux.dev Closes: https://lore.kernel.org/bpf/20260815071927.147049-1-zirajs7@gmail.com/
2026-09-29bpf: Wait for an RCU tasks grace period before freeing trampoline progsFlorent Revest (Anthropic)
When a prog is detached from a trampoline, it is freed after an RCU grace period, or an RCU tasks trace one if it is sleepable. This covers the tasks that are running the prog, since the prog's enter helper takes the matching read lock before the prog is called. On a preemptible kernel, it doesn't cover a task that was preempted in the trampoline just before the enter helper. That task holds no lock yet, and it calls the prog after it was freed: BUG: KASAN: vmalloc-out-of-bounds in __bpf_prog_enter_recur+0x3a5/0x3f0 Read of size 8 at addr ffffc90000055040 by task candidate/110 CPU: 1 UID: 0 PID: 110 Comm: candidate Not tainted 7.3.0-rc2-00014-g15071f2a1263-dirty #2 PREEMPT(full) Call Trace: <TASK> __bpf_prog_enter_recur+0x3a5/0x3f0 bpf_trampoline_6442509193+0x37/0xf1 __x64_sys_futex+0x9/0x410 do_syscall_64+0xb0/0x530 ... Wait for an RCU tasks grace period before the existing one when freeing a prog that was linked to a trampoline. An RCU tasks grace period only ends once the tasks that were preempted have run again, and bpf_tramp_image_put() already relies on it to free the image. It doesn't wait for tasks that sleep in the trampoline, the next commit takes care of those. Only progs that were linked to a trampoline can be called this way, so bpf_trampoline_add_prog() marks them and other progs are still freed as before. Fixes: e21aa341785c ("bpf: Fix fexit trampoline.") Reported-by: Junseo Lim <zirajs7@gmail.com> Signed-off-by: Florent Revest (Anthropic) <florent.revest@linux.dev> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20260926135605.1217928-2-florent.revest@linux.dev Closes: https://lore.kernel.org/bpf/aqdrwVpanH3WGurX@omen-arch/
2026-09-25bpf: Hold map BTF for the memory allocator destructor recordKumar Kartikeya Dwivedi
bpf_ma_set_dtor() duplicates map->record so that the bpf_mem_alloc destructor can release the special fields of hash and rhash map elements once the allocator frees them for good. btf_record_dup() only acquires references on kernel and module BTF. Fields whose types live in the map BTF keep pointing into it: a kptr to a local type refers to map->btf, and a list_head or rb_root field carries a value_rec owned by the struct meta table of map->btf. That borrowed state can outlive the map. When RCU callbacks are still in flight, bpf_mem_alloc_destroy() copies the allocator and defers the final drain, together with the destructor context, to a workqueue. bpf_map_free() then frees map->record and drops the map's BTF reference as soon as map_free() returns. Once the program that loaded the BTF is gone as well, the BTF is freed while the worker still uses the duplicate. The worker dereferences it through btf_is_kernel() and btf_find_struct_meta() in bpf_obj_free_fields() when destroying elements that were freed under RCU, and through btf_is_kernel() in btf_record_free() when releasing the context itself: BUG: KASAN: slab-use-after-free in btf_is_kernel+0x19/0x30 Read of size 1 at addr ffff88811505dce8 by task kworker/u16:2/50 Workqueue: events_unbound free_mem_alloc_deferred Call Trace: btf_is_kernel+0x19/0x30 btf_record_free+0xc7/0xf0 htab_dtor_ctx_free+0x1a/0x30 free_mem_alloc_no_barrier+0x39/0x230 free_mem_alloc_deferred+0x2a/0x40 process_scheduled_works+0x5ce/0x9d0 worker_thread+0x42a/0x5e0 kthread+0x1f5/0x230 ret_from_fork+0x1dd/0x390 ret_from_fork_asm+0x1a/0x30 Allocated by task 358: __kmalloc_cache_noprof+0x287/0x510 btf_new_fd+0xf7/0x3d0 __sys_bpf+0x487/0x840 Freed by task 0: kfree+0x186/0x560 rcu_core+0x6f6/0xd40 Freeing a map that still holds more elements than the allocator's high watermark is enough to get there, because the bulk free during teardown queues the RCU callback that makes bpf_mem_alloc_destroy() defer. Non-preallocated hash maps and resizable hash maps both register the destructor and are affected alike. Keep one reference on the map BTF in the destructor context. Every non-kernel pointer in the duplicated record is either map->btf or memory owned by it, so a single reference covers kptrs, list heads, and rb roots, and mirrors what bpf_map_meta_alloc() does for the duplicated record of an inner map. Release it only after the duplicated record has been freed: the worker is preemptible and not in an RCU read-side critical section, so dropping the last reference first would let the BTF be freed while btf_record_free() still reads its fields. Fixes: 1df97a7453ee ("bpf: Register dtor for freeing special fields") Reported-by: Yuan Chen <chenyuan@kylinos.cn> Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://lore.kernel.org/bpf/20260901062845.1379760-3-chenyuan_fl@163.com Link: https://patch.msgid.link/20260925064652.2261885-1-memxor@gmail.com
2026-09-25bpf: Fix overflow of jump offset in constant blindingAlexei Starovoitov
bpf_jit_blind_insn() rewrites 'if rX op imm goto off' into: AX = imm ^ rnd AX ^= rnd if rX op AX goto off and subtracts 2 from off when the jump is backward. off is s16, so -32767 and -32768 wrap and the jump goes 32767 insns forward, past the end of the prog. The jump is inside the patch, hence bpf_adj_branches() doesn't check it. The kernel crashes when such prog runs with bpf_jit_harden=2. Return -ERANGE in such case. bpf_jit_blind_constants() fails the same way when blinding of other insns moves a jump out of range. Fixes: 4f3446bb809f ("bpf: add generic constant blinding for use in jits") Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://lore.kernel.org/bpf/20260924182353.277516-1-alexei.starovoitov@gmail.com Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-09-24bpf: Zero-fill other CPUs when BPF_F_CPU creates a per-cpu hash elementDonggeun Yoo
pcpu_init_value() initializes the per-cpu area of a newly created [lru_]percpu_hash element. The area is recycled, so when the value comes from a BPF program (onallcpus == false) it writes the running CPU's slot and zeroes the rest. bpf_percpu_hash_update() passes onallcpus == true, which delegates to pcpu_copy_value(). pcpu_copy_value() writes only the CPU named in map_flags when BPF_F_CPU is set, so on the create path the other slots keep the recycled element's values: update(k1, 0xdeadc0de, BPF_F_ALL_CPUS) every CPU holds 0xdeadc0de delete(k1) element back on the freelist update(k2, 0xc0ffee, BPF_F_CPU | 0) creates, writes CPU 0 only lookup(k2) CPU 0 0xc0ffee, rest 0xdeadc0de Zero-fill the other CPUs on that arm too. Fixes: c6936161fd55 ("bpf: Add BPF_F_CPU and BPF_F_ALL_CPUS flags support for percpu_hash and lru_percpu_hash maps") Signed-off-by: Donggeun Yoo <donggeunyoo.kernel@gmail.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20260924102321.2120434-2-donggeunyoo.kernel@gmail.com
2026-09-23bpf: Reject dev-bound-only programs on other devicesWeiming Shi
__bpf_offload_dev_match() falls back to comparing offdev pointers after an exact netdev mismatch. Bound-only programs normally have NULL offdevs, so unrelated netdevs compare equal. A bound-only program on an offload-registered netdev can instead inherit a real offdev and match a sibling port. With CAP_BPF and CAP_NET_ADMIN, a caller can use bpf(BPF_LINK_CREATE) with a different target ifindex to run metadata kfuncs specialized for the bound driver on the target driver's xdp_buff. Running a veth-bound program on tun reads beyond tun's bare stack xdp_buff as a veth_xdp_buff. Oops: general protection fault, probably for non-canonical address KASAN: null-ptr-deref in range [0x0000000000000010-0x0000000000000017] RIP: 0010:veth_xdp_rx_timestamp (drivers/net/veth.c:1673) Call Trace: ... tun_build_skb (drivers/net/tun.c:1739) tun_get_user (drivers/net/tun.c:1856) tun_chr_write_iter (drivers/net/tun.c:2091) vfs_write (fs/read_write.c:595 fs/read_write.c:687) ksys_write (fs/read_write.c:739) do_syscall_64 (arch/x86/entry/syscall_64.c:84) entry_SYSCALL_64_after_hwframe (arch/x86/entry/entry_64.S:121) Kernel panic - not syncing: Fatal exception in interrupt Restrict non-offloaded programs to exact netdev matches and retain the shared-offdev fallback only for genuinely offloaded multi-port programs. Fixes: 2b3486bc2d23 ("bpf: Introduce device-bound XDP programs") Reported-by: <co+ac0a8c41de69121d@bugs.sh> Signed-off-by: Weiming Shi <bestswngs@gmail.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://lore.kernel.org/bpf/20260917161335.1020405-2-bestswngs@gmail.com/ Link: https://patch.msgid.link/20260920132303.4109240-3-bestswngs@gmail.com
2026-09-22bpf: Prevent variable arena/non-arena register contentsEmil Tsalapatis
The verifier marks ALU instructions that include at least one arena operand with needs_zext: These instructions are fixed up after verification to be ALU32 instructions to ensure that the result is a valid offset into an arena. However, different code paths may provide two non-arena 64-bit arguments to the same instruction. The result of the operation in that code path is wrong, since it is now unexpectedly truncated to 32 bits and zero-extended. Add logic to the verifier to ensure every instruction either always has at least one PTR_TO_ARENA argument, or never does. Since needs_zext already tracks the first scenario, add a prevent_zext field in bpf_insn_aux to track the latter. Reject instructions that use arena arguments and have prevent_zext set, or do not have arena arguments and have needs_zext set. Fixes: 6082b6c328b5 ("bpf: Recognize addr_space_cast instruction in the verifier.") Reported-by: Nicholas Carlini <nicholas@carlini.com> Suggested-by: Nicholas Carlini <nicholas@carlini.com> Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20260922172028.6269-8-emil@etsalapatis.com
2026-09-22bpf: Reject pkt arguments in mutating subprogsEmil Tsalapatis
The verifier tracks changes in how PTR_TO_PACKET registers' bounds are modified across subprog boundaries. PTR_TO_PACKET registers are actually passed as PTR_TO_MEM, which is assumed valid for the entire call. This is not the case with packet memory, where a pskb_* call may invalidate its memory region. Reject BPF code that passes PTR_TO_PACKET pointers to subprogs that may mutate a packet. We cannot pass the pointer as a true PTR_TO_PACKET because we would also need to somehow pass the PTR_TO_PACKET_META or PTR_TO_PACKET_END to the subprog. Since we cannot avoid representing the pointer in the subprog as PTR_TO_MEM, only permit it if the subprog is guaranteed not to mutate the packet. Fixes: 80f281664f5a ("bpf: Support pointers in global func args") Reported-by: Nicholas Carlini <nicholas@carlini.com> Suggested-by: Nicholas Carlini <nicholas@carlini.com> Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20260922172028.6269-6-emil@etsalapatis.com
2026-09-21bpf: Reject non-negative offsets in stack_slot_obj_get_spi()Xu Yunxiang
bpf_get_spi() computes (-off - 1) / BPF_REG_SIZE using C division, which truncates toward zero. For off == 0, this produces spi 0, the same index used by the valid stack slot at fp-8. stack_slot_obj_get_spi() currently checks alignment and the resulting spi bounds, but does not reject the non-negative offset itself. It can therefore validate a PTR_TO_STACK register holding fp+0 against an iterator stored at fp-8 even though the runtime receives the actual fp+0 pointer. An effectful iterator kfunc can then interpret memory outside the BPF stack as iterator state. Reject non-negative offsets before converting the offset to an spi. All valid stack objects begin at a negative offset from the frame pointer. Fixes: 06accc8779c1 ("bpf: add support for open-coded iterator loops") Signed-off-by: Xu Yunxiang <xyx2021@mail.ustc.edu.cn> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Reviewed-by: Sun Jian <sun.jian.kdev@gmail.com> Link: https://lore.kernel.org/bpf/20260920210423.345636-2-xyx2021@mail.ustc.edu.cn
2026-09-19bpf: Check params size before reading reserved fieldsYuqi Xu
bpf_crypto_ctx_create() is a kfunc whose second argument is declared with the __sz annotation, so the verifier only guarantees that params__sz bytes of params are valid. The function nevertheless reads params->reserved[0] and params->reserved[1] (offsets 14 and 15) before comparing params__sz against the size of struct bpf_crypto_params, so a BPF program can pass a shorter buffer and have the kernel read past the region that was validated for it. Move the size check in front of the reserved field reads. Fixes: 3e1c6f35409f ("bpf: make common crypto API for TC/XDP programs") Reported-by: Vega <vega@nebusec.ai> Signed-off-by: Yuqi Xu <xuyuqiabc@gmail.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Reviewed-by: Ren Wei <weir@nebusec.ai> Link: https://patch.msgid.link/4f3ab4b03e79017e215521743996555439bf0bb3.1789802413.git.xuyuqiabc@gmail.com
2026-09-19bpf: Bound ownership depth through local kptrs and graph rootsKumar Kartikeya Dwivedi
Program-allocated objects can own other local objects through referenced kptrs. bpf_obj_free_fields() follows those pointers through __bpf_obj_drop_impl() synchronously, before the object storage is freed through RCU. A self-referential local kptr type therefore permits arbitrarily deep object chains, and dropping the head can exhaust the kernel stack. Long acyclic type chains have the same problem. btf_check_and_fixup_fields() still assumes referenced kptrs only point to kernel types and checks ownership through list and rbtree roots only. Its existing rule is sufficient for graph-only cycles: the target of each graph edge must contain a node, so every type in a cycle has both a root and a node. The rule rejects such a type owning another root, breaking every cycle. It also limits graph-only chains to three types, or two if the first type contains a node, and conservatively rejects longer acyclic chains. The missing local-kptr edges, rather than a missed graph-only cycle, are the bug introduced by support for bpf_kptr_xchg() into local kptrs. Replace that restriction with one bounded ownership walk covering graph roots and local referenced kptrs. Run it after all BTF records have been fixed up, reject cycles and paths deeper than eight record-bearing types, and cache each type's suffix depth while checking it against the remaining budget. This also permits the longer acyclic graph-only layouts rejected by the old rule; update their existing BTF tests accordingly. Keep the bound independent of MAX_CALL_FRAMES because recursive destruction can run below a BPF call chain. A plain local pointee without special-field metadata adds only a final non-recursing drop. Non-owning kptrs and kernel-BTF kptrs do not recurse through local records and remain outside the walk. Include local percpu-kptr edges too, although allocation of percpu objects with special fields is currently forbidden, so that relaxing that restriction cannot bypass the ownership bound. btf_check_and_fixup_fields() continues to initialize graph_root.value_rec, including for separately allocated map records. The ownership relationships belong to immutable program BTF and only need validation at BTF load time. Fixes: b0966c724584 ("bpf: Support bpf_kptr_xchg into local kptr") Reported-by: Nicholas Carlini <npc@anthropic.com> Suggested-by: Nicholas Carlini <npc@anthropic.com> Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://patch.msgid.link/20260914132444.2564218-2-memxor@gmail.com
2026-09-19bpf: Compare stack frames in regs_exact()Kumar Kartikeya Dwivedi
regs_exact() compares the register state up to id, followed by the ID mappings, but does not compare frameno. The PTR_TO_STACK case in regsafe() checks frameno separately, which is bypassed when exact comparison is requested. Consequently, infinite-loop detection can treat pointers to different stack frames as the same pointer and reject a finite loop. For example, initialize fp-8 to zero in the caller and to one in the callee, then pass the caller's fp-8 to the callee as r1: loop: r0 = *(u64 *)(r1 + 0); if r0 != 0 goto done; r1 = r10; r1 += -8; goto loop; done: exit; The loop terminates after reading the callee's slot on its second iteration. At the loop header, however, the only relevant difference is r1's frameno, so exact comparison incorrectly reports an infinite loop. The same problem occurs when the pointer is spilled to the stack. Move frameno into the type-specific metadata union, ahead of id, so the existing prefix comparison in regs_exact() covers it. Ordinary stack pointers do not use another union member. Iterator and IRQ stack-slot states use their dedicated union views and do not need a frame lookup. This also keeps bpf_reg_state at 80 bytes. Since frameno now shares storage with other pointer metadata, it is only meaningful for PTR_TO_STACK registers. Return NULL from bpf_func() for other register types. process_iter_arg(), get_constant_map_key() and is_dynptr_reg_valid_init() look up the frame before checking the register type and would otherwise index frame[] with a byte of the register's map or BTF pointer. They dereference the frame only after their type check. Move the states_maybe_looping() boundary from frameno to precise after the field relocation. Its prefix comparison continues to cover the complete value state and now includes frameno. Continue to ignore precise. Precision marks control whether pruning may ignore scalar ranges; they do not change the represented values, and exact comparison already compares those ranges unconditionally. Marks can also change through backtracking while an ancestor state is still being explored. Fixes: d5b892fd607a ("bpf: make infinite loop detection in is_state_visited() exact") Reported-by: Eduard Zingerman <eddyz87@gmail.com> Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Acked-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://patch.msgid.link/20260919014213.1840880-2-memxor@gmail.com
2026-09-17bpf: Assign lock identity to callback map valuesKumar Kartikeya Dwivedi
A nested bpf_for_each_map_elem() callback can unlock a different element of the same map: static long inner(void *map, int *key, struct value *v, struct value **outer_value) { bpf_spin_lock(&v->lock); bpf_spin_unlock(&(*outer_value)->lock); return 0; } static long outer(void *map, int *key, struct value *v, void *ctx) { bpf_for_each_map_elem(map, inner, &v, 0); return 0; } Both callback values currently have ID zero and the same map_ptr. process_spin_lock() compares those two fields, so it accepts the unlock even though the two callbacks can receive different map elements. Assign a fresh ID to every callback map value in the for-each, timer/workqueue, and task-work constructors. Copies of one callback argument retain its ID, so locking and unlocking through that argument continues to work. Distinct callbacks also get distinct IDs for single-element arrays, including inner arrays sharing inner_map_meta. Preserve map_uid for every inner-map lookup and compare it through check_ids() during state pruning. This preserves relationships between maps, keys, and values while allowing equivalent states with different lookup IDs to match. It avoids field-specific rules for when an inner map needs an identity. Move map_uid out of the metadata union and next to the other IDs, so register comparisons can use the existing memcmp() ranges and remap the IDs separately. Clear it when resetting a register or converting a map lookup result to a socket pointer. Shrink frameno to u8, which is enough for MAX_CALL_FRAMES, to make room without growing bpf_reg_state. Fixes: d0d78c1df9b1 ("bpf: Allow locking bpf_spin_lock global variables") Reported-by: Nicholas Carlini <npc@anthropic.com> Suggested-by: Nicholas Carlini <npc@anthropic.com> Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Acked-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://patch.msgid.link/20260917233222.2542500-9-memxor@gmail.com Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
2026-09-17bpf: Apply CO-RE relocations before subprogram validationKumar Kartikeya Dwivedi
check_subprogs() verifies that each subprogram ends in an exit or an unconditional jump before in-kernel CO-RE relocations are applied. An unresolved relocation can then replace that terminal instruction with an invalid helper call. The resulting fall-through into another subprogram breaks the CFG invariant used by postorder and stack liveness analysis, which can write past their per-subprogram arrays. Apply CO-RE relocations immediately after preparing the program BTF, before subprogram discovery and validation. Keep func_info and line_info validation after subprogram discovery because those records depend on the complete subprogram layout. Reject an ldimm64 first slot at the end of the instruction stream before CO-RE can inspect its missing second slot. check_subprogs() previously rejected this form before relocation processing because it is not a valid subprogram terminator. Moving CO-RE ahead of check_subprogs() removes that implicit protection, so perform an explicit check before applying relocations. Include core_relo_cnt when deciding whether to prepare program BTF. A load that supplied only CO-RE relocation metadata previously skipped both BTF setup and relocation processing. Fixes: fbd94c7afcf9 ("bpf: Pass a set of bpf_core_relo-s to prog_load command.") Suggested-by: Andrii Nakryiko <andrii@kernel.org> Suggested-by: Alexei Starovoitov <ast@kernel.org> Suggested-by: Eduard Zingerman <eddyz87@gmail.com> Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Acked-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://patch.msgid.link/20260917233222.2542500-5-memxor@gmail.com Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
2026-09-17bpf: Preserve packet pointer class displacement in regsafe()Kumar Kartikeya Dwivedi
regsafe() maps packet pointer IDs between states and checks that each current register range is a subset of the corresponding explored register range. It does not, however, preserve the displacement between registers that share a packet pointer ID. This is unsound because packet range is shared by ID. A bounds check on one class member updates every member, and a later access can consume the range through another member. Commit 022ac0750883 ("bpf: use reg->var_off instead of reg->off for pointers") folded the fixed pointer offset into r64 and removed the old off equality check, so two individually narrower registers can prune even when their displacement has changed. The explored path can then license an out-of-bounds packet access on the pruned path. Require matching range bases for packet pointers with an ID. Together with the existing ID mapping, this preserves the displacement between members of each packet-pointer class without adding per-ID state. Packet pointers without an ID remain unaffected. Fixes: 022ac0750883 ("bpf: use reg->var_off instead of reg->off for pointers") Reported-by: Nicholas Carlini <npc@anthropic.com> Suggested-by: Nicholas Carlini <npc@anthropic.com> Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Acked-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://patch.msgid.link/20260917233222.2542500-3-memxor@gmail.com Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
2026-09-17bpf: Make post-verification instruction rewrites killableKumar Kartikeya Dwivedi
After do_check() returns, the verifier runs several instruction rewrite passes. Some of them patch or remove one instruction at a time. Each operation moves the remaining instruction and auxiliary-data arrays and adjusts all branch offsets, making the overall work quadratic in the program length. A privileged loader can submit 131072 unconditional jumps by zero followed by a valid return. Verification finishes quickly, but bpf_opt_remove_nops() then spends a long time removing each jump separately. Since this post-verification work neither checks for signals nor reschedules, a pending SIGKILL cannot terminate the task until the rewrite finishes. Make bpf_patch_insn_data() and verifier_remove_insns() common cancellation and rescheduling points. These helpers run from BPF_PROG_LOAD process context, and bpf_patch_insn_data() can already sleep while reallocating auxiliary data. Report interrupted constant blinding as -EINTR and propagate it through both JIT paths, including kernels that permit interpreter fallback. Other blinding failures retain the existing fallback behavior. This does not reduce the quadratic cost of the rewrite passes, but it makes the work preemptible and allows a killed loader to be torn down promptly. Fixes: 52875a04f4b2 ("bpf: verifier: remove dead code") Reported-by: Nicholas Carlini <npc@anthropic.com> Suggested-by: Nicholas Carlini <npc@anthropic.com> Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Acked-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://patch.msgid.link/20260917233222.2542500-2-memxor@gmail.com Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
2026-09-17bpf: Skip unsettled links in link iteratorWeiming Shi
bpf_link_prime() inserts a link into link_idr before anon_inode_getfile() succeeds and before bpf_link_settle() publishes the ID in link->id. bpf_link_by_id() treats such an ID-zero link as unsettled, but the link iterator takes a reference without this check. If anon_inode_getfile() then fails, the creator removes the ID and frees its still-private link directly. The iterator is left with a dangling reference and its next bpf_link_put() accesses freed memory. Treat ID-zero entries as transient in bpf_link_get_curr_or_next(), just as bpf_link_by_id() does. BUG: KASAN: slab-use-after-free in bpf_link_put Write of size 8 by task exp/384 Call Trace: bpf_link_put kernel/bpf/syscall.c:3372 bpf_link_seq_next kernel/bpf/link_iter.c:33 bpf_seq_read kernel/bpf/bpf_iter.c:158 vfs_read fs/read_write.c:572 ksys_read fs/read_write.c:716 do_syscall_64 arch/x86/entry/syscall_64.c:84 entry_SYSCALL_64_after_hwframe arch/x86/entry/entry_64.S:121 Kernel panic - not syncing: KASAN: panic_on_warn set ... Fixes: 9f8836127308 ("bpf: Add bpf_link iterator") Reported-by: Xiang Mei <xmei5@asu.edu> Signed-off-by: Weiming Shi <bestswngs@gmail.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Link: https://lore.kernel.org/bpf/20260914170206.170723-2-bestswngs@gmail.com
2026-09-17bpf: Verify global subprogs in each sleepability contextKumar Kartikeya Dwivedi
Global subprograms are verified independently with a fresh verifier root. do_check_common() currently seeds that root's in_sleepable state from the program, even though a global subprogram can also run from callbacks whose execution context differs from the program's main entry point. In particular, workqueue and task-work callbacks are sleepable even when the containing program is not. A global subprogram of that program is therefore verified as non-sleepable, making in_rcu_cs() true and allowing loads of RCU-protected kptrs to produce trusted MEM_RCU pointers. The same subprogram can then be called from a sleepable callback without a classic RCU reader. It can retain such a pointer while the object is freed and use it after free. The verifier's execution-context predicates are complementary. A state is sleepable only when in_sleepable is set and no RCU, preemption, IRQ, or lock region is active. Each condition which prevents sleeping also provides RCU protection, while in_rcu_cs() treats a non-sleepable state as implicitly protected. Use this relationship to represent a global subprogram caller with only the result of in_sleepable_context(). A protected sleepable caller is normalized to in_sleepable=false at the independent verification root. This both prevents sleepable operations and makes in_rcu_cs() true without copying caller-owned lock state. Track only the contexts in which each global subprogram is actually reached. Verify it once if all reachable calls use the same context, and twice only if both sleepable and non-sleepable calls reach it. Calls found while verifying globals or asynchronous callbacks mark further contexts for checking. Repeat the existing subprogram walk until all called contexts have been verified; unreachable global calls remain unchecked. Accumulate instruction counts over those verification passes. Preserve the total recorded before each pass, since path accounting has already added this pass's synchronous instructions and its root total must also include asynchronous subprograms. This makes an unprotected callback verify the global subprogram as sleepable, turning its RCU-protected kptr load into an untrusted pointer. Protected callers and global subprograms which do not depend on implicit RCU protection remain valid. Fixes: 81f1d7a583fa ("bpf: wq: add bpf_wq_set_callback_impl") Fixes: 38aa7003e369 ("bpf: task work scheduling kfuncs") Reported-by: Nicholas Carlini <npc@anthropic.com> Suggested-by: Nicholas Carlini <npc@anthropic.com> Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Acked-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://patch.msgid.link/20260914131923.2544250-2-memxor@gmail.com Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
2026-09-13bpf: Fix u32 overflow issue in map batch operationsMasoud Aghasi
Several map batch operation implementations such as generic_map_lookup_batch() use calculations in the form of "values + cp * map->value_size" to compute the desired userspace memory address for reading or writing. This can overflow the u32 type (the result of "cp * map->value_size") when the map size exceeds 4GB. generic_map_lookup_batch() may corrupt values for some keys in userspace memory, and in some cases it mismatches values for some keys while still reporting success. Other batch operations may fail to delete or update some keys, or the syscall may return unexpected errors. Add size_t casts to prevent the affected offset and size calculations from overflowing. Fixes: cb4d03ab499d ("bpf: Add generic support for lookup batch op") Fixes: aa2e93b8e58e ("bpf: Add generic support for update and delete batch ops") Fixes: 057996380a42 ("bpf: Add batch ops to all htab bpf map") Signed-off-by: Masoud Aghasi <maghasi@disroot.org> Link: https://lore.kernel.org/r/20260903082734.623904-1-maghasi@disroot.org Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2026-09-13bpf: Fix UAF due to concurrent consumption of ttrace lists in alloc_bulkPu Lehui
Syzkaller repeatedly triggered UAF splats related to nodes in waiting_for_gp_ttrace within the bpf memalloc: BUG: KASAN: slab-use-after-free in llist_del_first+0x85/0x110 lib/llist.c:61 Read of size 8 at addr ffff8881572cd080 by task syz.4.470/5112 ... llist_del_first+0x85/0x110 lib/llist.c:61 alloc_bulk+0x193/0x460 kernel/bpf/memalloc.c:229 bpf_mem_refill+0x386/0x560 kernel/bpf/memalloc.c:436 Freed by task 14: ... __free_rcu kernel/bpf/memalloc.c:281 [inline] __free_rcu_tasks_trace+0x48/0xd0 kernel/bpf/memalloc.c:291 rcu_tasks_invoke_cbs+0x1ec/0x3e0 kernel/rcu/tasks.h:571 rcu_tasks_one_gp+0x13d/0x220 kernel/rcu/tasks.h:621 rcu_tasks_kthread+0xf3/0x120 kernel/rcu/tasks.h:651 The reason is that the UAF occurs after the RCU Tasks Trace GP expires: when the __free_rcu() callback runs, there is no synchronization protecting llist_del_all() against concurrent alloc_bulk() operating on waiting_for_gp_ttrace, leading to the race condition below: CPU0 CPU1 __free_rcu (RCU Tasks Trace callback) alloc_bulk llist_del_first(&c->waiting_for_gp_ttrace) entry = smp_load_acquire(&head->first); do { if (entry == NULL) return NULL; free_all(llist_del_all(&c->waiting_for_gp_ttrace)) llist_for_each_safe(pos, t, llnode) free_one(pos); next = READ_ONCE(entry->next); <-- trigger UAF } while (!try_cmpxchg(&head->first, &entry, next)); In addition, there is also a theoretical race condition on the free_by_rcu_ttrace list. This race requires two preconditions: an in-flight Tasks Trace GP keeping c->call_rcu_ttrace_in_progress == 1, and concurrent cross-CPU frees repopulating c->free_by_rcu_ttrace with new nodes. Under these conditions, the following scenario triggers UAF: // CPU0 // irq work is still busy (on PREEMPT_RT) alloc_bulk() llist_del_first(&c->free_by_rcu_ttrace) entry = smp_load_acquire(&head->first); do { if (entry == NULL) return NULL; // CPU1 bpf_mem_alloc_destroy() WRITE_ONCE(c->draining, true) // wait for CPU0 irq_work_sync() // CPU2 do_call_rcu_ttrace(tgt(CPU0)) if (c->draining) { llist_del_all(&c->free_by_rcu_ttrace) free_all() } // CPU0 continue next = READ_ONCE(entry->next); <-- trigger UAF while (!try_cmpxchg(&head->first, &entry, next)); Fix this by introducing a raw spinlock to synchronize the concurrent consumption on waiting_for_gp_ttrace and free_by_rcu_ttrace. Fixes: 04fabf00b4d3 ("bpf: Allow reuse from waiting_for_gp_ttrace list.") Suggested-by: Alexei Starovoitov <ast@kernel.org> Suggested-by: Hou Tao <houtao1@huawei.com> Signed-off-by: Pu Lehui <pulehui@huawei.com> Acked-by: Hou Tao <houtao1@huawei.com> Link: https://lore.kernel.org/r/20260905021139.4116529-1-pulehui@huaweicloud.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2026-09-12bpf: Avoid soft lockup in __htab_map_lookup_and_delete_batch()Jose Fernandez (Anthropic)
__htab_map_lookup_and_delete_batch() has no rescheduling point. The batch count bounds how many entries are copied out, not how many buckets are visited, so one BPF_MAP_LOOKUP_BATCH call can walk the map end to end. The empty-bucket fast path is worse: it stays inside a single rcu_read_lock() / bpf_disable_instrumentation() section for any run of consecutive empty buckets. That holds up on small maps, but it falls apart at scale. On a 144-CPU arm64 host running a CONFIG_PREEMPT_NONE kernel, periodic BPF_MAP_LOOKUP_BATCH calls against an LRU hash map with 16,777,216 buckets held a CPU inside the batch op for 77+ seconds and triggered the soft lockup watchdog. Commit 75134f16e7dd ("bpf: Add schedule points in batch ops") fixed this same problem in the generic batch ops, but not in this htab-native path, which every htab-based hash map variant uses for its lookup[_and_delete] batch ops. Complete that fix here. Leave the critical section after 64 consecutive empty buckets, call cond_resched_tasks_rcu_qs(), and resume at the saved bucket cursor. No locks are held at that point, and resuming from the cursor is already the function's behavior for non-empty buckets. Add the same call to the per-bucket loop after copy_to_user(), where every lock has been dropped. cond_resched_rcu() is not enough here: sleeping with bpf_prog_active elevated makes tracing programs on that CPU silently skip their invocations. Plain cond_resched() is not enough either. It is a no-op under PREEMPT and PREEMPT_LAZY, the only models arm64 and x86 have offered since commit 7dadeaa6e851 ("sched: Further restrict the preemption modes"). It is also never a Tasks RCU quiescent state, in any model: the reschedule counts as a preemption. The walking task stays a holdout and stalls every synchronize_rcu_tasks() caller, ftrace and BPF trampoline teardown included, until the syscall returns [1]. cond_resched_tasks_rcu_qs() is the usual tool for that [2]. It reports the quiescent state at each yield and still reschedules as cond_resched() does on PREEMPT_NONE and PREEMPT_VOLUNTARY kernels. Fixes: 057996380a42 ("bpf: Add batch ops to all htab bpf map") Cc: "Paul E. McKenney" <paulmck@kernel.org> Cc: Rik van Riel <riel@surriel.com> Link: https://lore.kernel.org/bpf/20260715215314.44423f47@fangorn/ [1] Link: https://lore.kernel.org/bpf/9d444098-7c03-4163-af12-bd0a79a51443@paulmck-laptop/ [2] Assisted-by: LLM Signed-off-by: Jose Fernandez (Anthropic) <jose.fernandez@linux.dev> Signed-off-by: Josef Bacik <josef@toxicpanda.com> Reviewed-by: Rik van Riel <riel@surriel.com> Link: https://lore.kernel.org/r/20260909-b4-htab-batch-resched-v2-1-0cb529d8f95a@toxicpanda.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2026-09-12bpf: Allow terminal gotox instructionsSiddharth Chintamaneni
check_subprogs() treats gotox as a direct jump and validates its reserved zero offset. When gotox is the final instruction, this produces a synthetic successor one instruction past the end of the subprogram and rejects an otherwise valid program. Skip direct-offset validation for gotox and accept it as a non-fallthrough terminal instruction. Its actual targets remain validated from the instruction-array jump table during CFG construction. Fixes: 493d9e0d6083 ("bpf, x86: add support for indirect jumps") Signed-off-by: Siddharth Chintamaneni <sidchintamaneni@gmail.com> Reviewed-by: Anton Protopopov <a.s.protopopov@gmail.com> Link: https://lore.kernel.org/r/20260902171414.96165-1-sidchintamaneni@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2026-09-10bpf: Fix divide-by-zero in btf_struct_walk()Jiayuan Chen
When an access goes past the struct and the last member is a flexible array, btf_struct_walk() folds the offset back into a single element with (off - moff) % t->size, but never checks that the element type has a size. BTF takes an empty struct, so this in program BTF /* event could be empty */ struct event { #ifdef HAVE_TIMESTAMP __u64 ts; #endif }; struct batch { int nr; struct event events[]; }; divides by zero at prog load time. Getting there needs a PTR_TO_BTF_ID that is not MEM_ALLOC, e.g. a plain read of a local kptr stashed in a map from a sleepable program. Oops: divide error: 0000 [#1] SMP KASAN PTI RIP: 0010:btf_struct_walk+0x53f/0x1570 Call Trace: <TASK> btf_struct_access+0x42a/0xcd0 check_ptr_to_btf_access+0x4dc/0x1160 check_mem_access+0x3a45/0x8740 check_load_mem+0x36a/0xd10 do_check_common+0x3ef0/0xb210 bpf_check+0x6d3b/0x8580 bpf_prog_load+0xf7c/0x2720 __sys_bpf+0xa83/0x3690 __x64_sys_bpf+0xc7/0x150 x64_sys_call+0x1f3f/0x27e0 do_syscall_64+0xe5/0x610 entry_SYSCALL_64_after_hwframe+0x76/0x7e </TASK> Reject a zero-sized element type. The fixed array path in the same function already bails out on the same thing: btf_struct_walk() ... /* skip empty array */ if (moff == mtrue_end) continue; msize /= total_nelems; Fixes: 9c5f8a1008a1 ("bpf: Support variable length array in tracing programs") Signed-off-by: Jiayuan Chen <jiayuan.chen@linux.dev> Acked-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/r/20260910122316.186384-1-jiayuan.chen@linux.dev Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2026-09-10bpf: Clear scalar delta on narrowing stack spillDaniel Borkmann
check_stack_write_fixed_off() breaks the scalar link on a narrowing spill by zeroing the id of the destination stack slot, but leaves the delta in place. save_register_state() has just copied the source register verbatim, so the slot keeps the BPF_ADD_CONST delta of a register it is no longer linked to. This is the same shape as the case addressed in 1b327732c846 ("bpf: Clear delta when clearing reg id for non-{add,sub} ops"). Unlike the latter, no miscomputation seems reachable, so mainly consistency. Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/r/20260910185604.44949-1-daniel@iogearbox.net Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2026-09-10bpf: Require CAP_PERFMON for untrusted read-only memory readsDaniel Borkmann
Marking bpf_rdonly_cast() KF_PERFMON CAP-limits one producer of PTR_TO_MEM | MEM_RDONLY | PTR_UNTRUSTED, but not the type itself. A global subprogram argument tagged __arg_untrusted results in the same register with no kfunc call. Reported-by: STAR Labs SG <info@starlabs.sg> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/r/20260910213510.49358-3-daniel@iogearbox.net Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2026-09-10bpf: Require CAP_PERFMON for kfuncs reading memoryDaniel Borkmann
Mark fault-safe probe reading kfuncs as KF_PERFMON, similarly as we do for the old-style BPF helper equivalents. bpf_rdonly_cast() is included in this list as well as it returns PTR_TO_MEM | MEM_RDONLY | PTR_UNTRUSTED for an unchecked object and is using fault-safe BPF_PROBE_MEM. Note that only the void form of bpf_rdonly_cast() produced a register that was readable without CAP_PERFMON. For a struct type id the kfunc returns PTR_TO_BTF_ID | PTR_UNTRUSTED, whose dereference has always been gated in check_ptr_to_btf_access(). The flag is not conditional on the type id, so for the latter it only moves the rejection from the dereference to the call itself, which is the better place to report it anyway. The bpf_stream_vprintk() and bpf_stream_print_stack() kfuncs are marked as well. The former ends up in the same bpf_bprintf_prepare() as the bpf_snprintf() helper, where %pks, %pus and %pI4 read through a program- supplied address and %pB resolves one into a symbol. The latter walks the stack and prints each instruction pointer via %pS. Lastly, bpf_get_kmem_cache() takes a raw scalar address that the verifier does not constrain and dereferences the page/slab metadata for it. Field reads on the returned pointer are still blocked (PTR_TO_BTF_ID | PTR_UNTRUSTED -> check_ptr_to_btf_access() results in -EPERM without the CAP), but the NULL/non-NULL result is observable. Reported-by: STAR Labs SG <info@starlabs.sg> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/r/20260910213510.49358-2-daniel@iogearbox.net Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2026-09-10bpf: Add KF_PERFMON kfunc flagDaniel Borkmann
Tracing related BPF helpers e.g. under bpf_base_func_proto() are gated behind CAP_PERFMON. However, the same is currently not true for kfuncs and they are accessible via plain CAP_BPF. Add a new KF_PERFMON flag which can be used such that check_kfunc_call() ensures env->allow_ptr_leaks is permitted. This follows similar pattern to existing KF_DESTRUCTIVE flag. The rejection returns -EPERM to match the other CAP_PERFMON gates in the verifier, that is, check_ptr_to_btf_access() and check_ptr_to_map_access(), which report the very same policy to user space. Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/r/20260910213510.49358-1-daniel@iogearbox.net Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2026-09-06Merge tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpfLinus Torvalds
Pull bpf fixes from Alexei Starovoitov: "This mainly contains verifier fixes that address bugs reported by Nicholas Carlini. - Fix incorrect non-NULL inference in pointer comparisons: pointer types that may be NULL at runtime, pointers with unbounded offsets, JMP32 comparisons with zero, and imprecise zero registers (Eduard Zingerman) - Fix precision tracking for half-dead zero spills, ld_abs/ld_ind implicit subprog exit, bpf_loop() callbacks, linked scalar ids and NULL call arguments (Eduard Zingerman) - Reject BPF_PSEUDO_FUNC reference to the main program, fix zero extension of arena 32-bit cmpxchg, don't rewrite bpf_fastcall patterns entered by a jump (Eduard Zingerman) - Fix percpu map update and BPF_F_CPU validation with sparse CPU IDs (Hui Su) - Fix NULL-ptr-derefs in bpf_snprintf_btf() for void and VAR types, and reject key-less BTF for hash maps (Jiayuan Chen) - Various fixes (Kumar Kartikeya Dwivedi): - Fix out-of-bounds access in disassembler on invalid LDSX instruction - mark siginfo of signal tracepoints as scalar and sched_process_wait argument as nullable - mark faultable stack helpers as sleepable - reject tail calls and legacy packet loads from callbacks - enforce rbtree callback lock restrictions for resilient locks - require MEM_PERCPU for percpu kptr stores - clear NON_OWN_REF after RCU protection ends - mark NULL kptr stores precise - preserve inner map identity in callback frames - reject non-scalar bpf_loop() iteration counts - Fix trampoline allocation slowdown on x86 by using EXECMEM_MODULE_DATA (Mike Rapoport) - Keep bpf_refcount_acquire() nullable for borrowed RCU kptrs and reject untrusted allocated-object pointers (Ning Ding) - Fix special fields handling in recycled rhtab elements (Nuoqi Gui, Yuan Chen)" * tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf: (86 commits) bpf, riscv: Make arena support depend on ZACAS selftests/bpf: Test pointer bpf_loop iteration count rejection bpf: Reject non-scalar bpf_loop iteration counts bpf: use mark_arg_precision() in check_mem_size_reg() bpf: propagate mark_chain_precision() errors out of loop_flag_is_zero() selftests/bpf: precision of a NULL global subprogram BTF_ID argument bpf: mark a NULL BTF_ID argument of a global subprogram precise selftests/bpf: precision of a NULL kfunc argument bpf: mark a NULL kfunc argument precise selftests/bpf: precision of a NULL global subprogram memory argument bpf: mark a NULL memory argument of a call precise selftests/bpf: precision of a NULL helper argument bpf: mark a NULL call argument precise selftests/bpf: Test inner map identities in callbacks bpf: Preserve inner map identity in callback frames selftests/bpf: Test imprecise scalar kptr stores bpf: Mark NULL kptr stores precise selftests/bpf: Test rhtab kptr cancellation semantics bpf: Cancel special fields when recycling rhtab elements selftests/bpf: Test timer field on recycled rhtab element ...
2026-09-05bpf: Reject non-scalar bpf_loop iteration countsKumar Kartikeya Dwivedi
bpf_loop() declares its nr_loops argument as ARG_ANYTHING. Privileged programs may pass pointer values to such arguments, so check_func_arg() lets a pointer-valued R1 reach the helper-specific checks. Since commit bb124da69c47 ("bpf: keep track of max number of bpf_loop callback iterations"), the verifier marks R1 precise and reads its upper bound to limit callback simulation. Precision backtracking only accepts scalar registers, so passing a pointer instead triggers the "backtracking misuse" verifier warning. Kernels with panic_on_warn enabled subsequently panic. Introduce ARG_SCALAR for helper arguments that only accept scalar values and use it for bpf_loop() nr_loops. Generic helper argument validation then rejects pointers before loop inlining and precision processing. Fixes: bb124da69c47 ("bpf: keep track of max number of bpf_loop callback iterations") Reported-by: syzbot+7b47f87674e9a1569110@syzkaller.appspotmail.com Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://patch.msgid.link/20260905014735.1452988-2-memxor@gmail.com Closes: https://lore.kernel.org/bpf/6a9ad24c.b5d4176b.238c3e.0001.GAE@google.com/ Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
2026-09-04treewide: refresh kmalloc_obj() conversionsKees Cook
This is another run of the Coccinelle script for converting kmalloc() family of allocations to kmalloc_obj() via the existing rules in scripts/coccinelle/api/kmalloc_objs.cocci This catches both the set of kmalloc() uses added since the first kmalloc_obj() conversions in v7.0 and adds a large group missed in the first pass due to Coccinelle not interacting well with the cleanup.h scoped_...() family of macros[1]. I worked around this with spatch's "--macro-file" argument to a file with all the scoped_...() macros mapped to Coccinelle's YACFE_ITERATOR[2] as that was the closest viable control flow indicator I could find. Build tested allmodconfig on x86, arm64, arm, loongarch, mips, powerpc, riscv, and s390 with no new warnings. Link: https://lore.kernel.org/lkml/202609021314.8A9C0B8@keescook/ [1] Link: https://github.com/coccinelle/coccinelle/blob/master/standard.h [2] Signed-off-by: Kees Cook <kees+treewide@kernel.org>
2026-09-04bpf: use mark_arg_precision() in check_mem_size_reg()Eduard Zingerman
Use newly added mark_arg_precision() helper in check_mem_size_reg(). Signed-off-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/r/20260904-register-is-null-precise-fixes-v1-10-0f5a360ff15d@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2026-09-04bpf: propagate mark_chain_precision() errors out of loop_flag_is_zero()Eduard Zingerman
Stop verification if mark_chain_precision() fails when called from loop_flag_is_zero(). No functional change intended for the paths where backtracking succeeds. Fixes: 1ade23711971 ("bpf: Inline calls to bpf_loop when callback is known") Signed-off-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/r/20260904-register-is-null-precise-fixes-v1-9-0f5a360ff15d@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2026-09-04bpf: mark a NULL BTF_ID argument of a global subprogram preciseEduard Zingerman
btf_check_func_arg_match() accepts a NULL register for an ARG_PTR_TO_BTF_ID argument tagged __arg_nullable and skips check_reg_type() and check_func_arg_reg_off() without marking the register precise. Hence a checkpoint created on such a path would prune against arbitrary scalar value. Fixes: e2b3c4ff5d18 ("bpf: add __arg_trusted global func arg tag") Signed-off-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/r/20260904-register-is-null-precise-fixes-v1-7-0f5a360ff15d@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2026-09-04bpf: mark a NULL kfunc argument preciseEduard Zingerman
check_kfunc_arg() allows bpf_register_is_null() for nullable arguments w/o marking the underlying scalar register precise. Hence a checkpoint created on such a path would prune against arbitrary scalar value. Fixes: 3bda08b63670 ("bpf: Allow NULL buffers in bpf_dynptr_slice(_rw)") Signed-off-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/r/20260904-register-is-null-precise-fixes-v1-5-0f5a360ff15d@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2026-09-04bpf: mark a NULL memory argument of a call preciseEduard Zingerman
check_mem_reg() allows bpf_register_is_null() for nullable arguments w/o marking the underlying scalar register precise. Hence a checkpoint created on such a path would prune against arbitrary scalar value. The argument may live on the stack rather than in a register when a call has more than MAX_BPF_FUNC_REG_ARGS arguments, hence the new mark_arg_precision() helper. Fixes: e5069b9c23b3 ("bpf: Support pointers in global func args") Signed-off-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/r/20260904-register-is-null-precise-fixes-v1-3-0f5a360ff15d@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2026-09-04bpf: mark a NULL call argument preciseEduard Zingerman
check_func_arg() allows bpf_register_is_null() for nullable arguments w/o marking the underlying scalar register precise. Hence a checkpoint created on such a path would prune against arbitrary scalar value. check_helper_call() enforces second parameter of the bpf_get_local_storage() to be zero, w/o marking the underlying scalar register precise. Hence a checkpoint created on such a path would prune against arbitrary scalar value. Grouping these two into one patch, as they share the same fixes tag. Fixes: b5dc0163d8fd ("bpf: precise scalar_value tracking") Signed-off-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/r/20260904-register-is-null-precise-fixes-v1-1-0f5a360ff15d@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2026-09-04bpf: Preserve inner map identity in callback framesKumar Kartikeya Dwivedi
Callback frame constructors initialize map-typed argument registers with __mark_reg_known_zero() and then restore map_ptr. This clears map_uid, which is the only field distinguishing inner maps that share an inner_map_meta template. When a timer callback invokes bpf_for_each_map_elem() on a second inner map, both the saved first map and the second map value can reach the nested callback as the same template with map_uid zero. bpf_timer_init() then accepts pairing the timer from the second map with the first map. The runtime records the first map in the timer without taking a reference. Freeing that map does not find the timer stored in the second map, so a later timer callback dereferences the freed map. Copy map_uid from the same caller register as map_ptr when constructing for-each, timer/workqueue, and task-work callback arguments. The existing identity check can then reject mismatched inner maps while allowing a callback value to be paired with its actual map. Fixes: 3e8ce29850f1 ("bpf: Prevent pointer mismatch in bpf_timer_init.") Fixes: 69c087ba6225 ("bpf: Add bpf_for_each_map_elem() helper") Fixes: 5c8fd7e2b5b0 ("bpf: bpf task work plumbing") Reported-by: Nicholas Carlini <npc@anthropic.com> Suggested-by: Nicholas Carlini <npc@anthropic.com> Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://lore.kernel.org/r/20260904104203.345917-8-memxor@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2026-09-04bpf: Mark NULL kptr stores preciseKumar Kartikeya Dwivedi
check_map_kptr_access() permits a scalar store into an untrusted kptr field only when the register is known to contain zero. Unlike other verifier checks whose outcome depends on a scalar value, it does not mark that register precise. A state checkpoint reached with an imprecise zero can therefore prune a second path that reaches the store with an arbitrary nonzero scalar. The program can write attacker-controlled bits into the kptr field and load them back as a PTR_TO_BTF_ID. Call mark_chain_precision() before accepting a known-zero register. This forces state equivalence to compare its scalar range and makes the verifier visit and reject a path carrying a nonzero value. Fixes: 61df10c7799e ("bpf: Allow storing unreferenced kptr in map") Reported-by: Nicholas Carlini <npc@anthropic.com> Suggested-by: Nicholas Carlini <npc@anthropic.com> Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Acked-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/r/20260904104203.345917-6-memxor@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2026-09-04bpf: Cancel special fields when recycling rhtab elementsNuoqi Gui
rhtab_map_update_existing() and rhtab_delete_elem() call bpf_obj_free_fields() when replacing or deleting a value. These map operations can run from BPF programs in NMI context, where releasing a referenced kptr or another complex field is not generally safe. Array and hash maps avoid that problem by cancelling only the asynchronous fields which can be stopped safely in the caller context. Other ownership state remains attached to the allocation until its memory allocator destructor performs the final cleanup. Use bpf_obj_cancel_fields() for the corresponding rhtab paths as well. This cancels timers, workqueues, and task work while allowing rhtab_mem_dtor() to release referenced kptrs when the allocation is eventually destroyed. Fixes: 6905f8601298 ("bpf: Allow special fields in resizable hashtab") Signed-off-by: Nuoqi Gui <gnq25@mails.tsinghua.edu.cn> Acked-by: Mykyta Yatsenko <yatsenko@meta.com> [ kkd: Rebased, used direct helper calls, and rewrote the commit log ] Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://lore.kernel.org/r/20260904104203.345917-4-memxor@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2026-09-04bpf: Preserve special fields in recycled rhtab elementsYuan Chen
rhtab_map_update_elem() initializes special fields after obtaining an element from bpf_mem_cache_alloc(). The allocator can return a fresh, zeroed unit, or recycle one from its RCU-pending lists before the registered destructor has run. A BPF program can retain a map-value pointer after deleting its element and initialize and arm a timer through that pointer. If the deleted unit is recycled, check_and_init_map_value() clears the only pointer to the timer. Neither a later deletion nor rhtab_mem_dtor() can then cancel it, and the callback can run with its key and value pointing into freed memory. Do not reinitialize special fields on insertion. Fresh allocator units are already zeroed. For recycled units, the special fields are ownership state that must remain visible to the eventual destructor. copy_map_value() already skips those fields, matching the non-preallocated hash-map path and the lifecycle established by commit 275c30bcee66 ("bpf: Don't reinit map value in prealloc_lru_pop"). Fixes: 6905f8601298 ("bpf: Allow special fields in resizable hashtab") Reported-by: Nicholas Carlini <npc@anthropic.com> Suggested-by: Nicholas Carlini <npc@anthropic.com> Signed-off-by: Yuan Chen <chenyuan@kylinos.cn> [ kkd: Split out the fix and rewrote the commit log ] Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://lore.kernel.org/r/20260904104203.345917-2-memxor@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2026-09-04bpf: Clear NON_OWN_REF after RCU protection endsKumar Kartikeya Dwivedi
A local kptr load of an object containing a graph node is marked MEM_RCU and NON_OWN_REF while protected by RCU. When the last RCU read-side critical section ends, invalidate_rcu_protected_refs() removes MEM_RCU and marks the pointer PTR_UNTRUSTED, but leaves NON_OWN_REF set. The stale flag lets graph kfunc argument checks continue treating the pointer as a live borrowed reference. In particular, bpf_rbtree_remove() can accept a pointer after its protection ended and return it as a new owning reference, even though the object may already have been freed. Clear NON_OWN_REF when an RCU-protected pointer is demoted. A spin lock also provides implicit RCU protection, so invalidate non-owning references before demoting RCU-protected pointers when releasing the lock. Otherwise the demotion would clear the flag before invalidate_non_owning_refs() can find and invalidate those aliases. The demoted pointer remains available for fault-protected reads. Exempt such reads from the allocated-object reference-state assertion; writes through a fault-prone pointer are already rejected, and bpf_may_fault_on_deref() makes the surviving loads use BPF_PROBE_MEM. Fixes: 1b12171533a9 ("bpf: Mark direct ld of stashed bpf_{rb,list}_node as non-owning ref") Reported-by: Nicholas Carlini <npc@anthropic.com> Suggested-by: Nicholas Carlini <npc@anthropic.com> Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://lore.kernel.org/r/20260904084325.52250-6-memxor@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2026-09-04bpf: Keep refcount_acquire nullable for borrowed RCU kptrsNing Ding
bpf_refcount_acquire() is fallible for a borrowed reference because the object may have reached a zero refcount. The verifier therefore keeps KF_RET_NULL on the return value unless the argument is an owning reference. An RCU-protected load of a local kptr is marked MEM_ALLOC, but it only receives NON_OWN_REF when the pointee contains a graph node. A refcounted object without a graph node consequently looks like an owning reference even though the loaded register has no acquired reference state. If the program drops the last real reference while remaining in the RCU critical section, refcount_inc_not_zero() returns NULL while the verifier treats the result as non-NULL. Only classify the argument as owning when it is backed by a verifier-tracked reference. This retains the non-NULL return for pointers from bpf_obj_new(), bpf_kptr_xchg(), or an earlier successful acquisition, while requiring a NULL check for borrowed RCU kptrs. Fixes: 1b12171533a9 ("bpf: Mark direct ld of stashed bpf_{rb,list}_node as non-owning ref") Reported-by: Nicholas Carlini <npc@anthropic.com> Suggested-by: Nicholas Carlini <npc@anthropic.com> Signed-off-by: Ning Ding <dingning04@gmail.com> [ kkd: Rewrote commit log ] Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://lore.kernel.org/r/20260904084325.52250-4-memxor@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2026-09-04bpf: Require MEM_PERCPU for percpu kptr storesKumar Kartikeya Dwivedi
map_kptr_match_type() treats perm_flags as the set of register type flags that a kptr field permits. Adding MEM_PERCPU to that set for BPF_KPTR_PERCPU does not require the source register to carry it, however. The subset test consequently accepts both a plain bpf_obj_new() allocation and a referenced kernel pointer into a __percpu_kptr map field. Loads from the field are always marked MEM_PERCPU. Consumers then treat the stored value as the cookie returned by bpf_percpu_obj_new(): per-CPU pointer helpers relocate it, and map teardown selects the per-CPU free path. A plain allocation can therefore provide an arbitrary kernel read/write, while a kernel pointer can be relocated into an invalid address or sent through a missing destructor. Require the source MEM_PERCPU flag to match the destination field kind. This preserves valid bpf_percpu_obj_new() stores and rejects both the program-BTF and kernel-BTF variants. Fixes: 36d8bdf75a93 ("bpf: Add alloc/xchg/direct_access support for local percpu kptr") Reported-by: Nicholas Carlini <npc@anthropic.com> Suggested-by: Nicholas Carlini <npc@anthropic.com> Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Link: https://lore.kernel.org/r/20260904084325.52250-2-memxor@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2026-09-04bpf: Mark the zero register precise for a register-form NULL checkEduard Zingerman
check_cond_jmp_op() accepts "if rA <op> rB" as a NULL check for a nullable pointer rA when rB is a scalar known to be zero, lifts PTR_MAYBE_NULL from rA in the corresponding branch and does not mark rB precise. Consider the following program: r0 = bpf_get_prandom_u32(); r6 = 1; /* the r6 == 0 path is explored first */ if (r0 == 0) goto 1f; r6 = 0; 1: r0 = bpf_map_lookup_elem(map, &0); /* absent, NULL at runtime */ if (r0 == r6) goto 2f; /* taken as a NULL check for r0 */ *(u8 *)(r0 + 0); /* verifier: map value; runtime: zero */ 2: return 0; The r6 == 0 path is explored first and the dereference is accepted. The r6 == 1 path is pruned at the checkpoint recorded for (1), so the comparison is never verified with a non-zero r6. At runtime a failed lookup returns NULL, NULL != 1 takes the non-NULL edge and the program dereferences a pointer that is zero. Fixes: 2f4cb53eed44 ("bpf: detect non null pointer with register operand in JEQ/JNE.") Reported-by: Nicholas Carlini <npc@anthropic.com> Suggested-by: Nicholas Carlini <npc@anthropic.com> Signed-off-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/bpf/20260904083325.2083493-7-eddyz87@gmail.com Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-09-04bpf: Don't predict JMP32 pointer vs zero comparisonsEduard Zingerman
Consider the following program: r1 = map_value; /* low 32 bits are zero at runtime */ r6 = 0xdead000000000000; if w1 != 0 goto l1; l0: r1 += r6; r2 = *(u64 *)(r1 + 0); exit; l1: r6 = 0; goto l0; At the moment is_branch_taken() reports the jump as always taken, because it does not distinguish between BPF_JMP and BPF_JMP32 comparisons when processing 'if w1 != 0 ...'. Fixes: cac616db39c2 ("bpf: Verifier track null pointer branch_taken with JNE and JEQ") Reported-by: Nicholas Carlini <npc@anthropic.com> Suggested-by: Nicholas Carlini <npc@anthropic.com> Signed-off-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/bpf/20260904083325.2083493-5-eddyz87@gmail.com Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
2026-09-04bpf: Don't resurrect a scalar id dropped by collect_linked_regs()Eduard Zingerman
check_cond_jmp_op() copies the compared registers into env->{false,true}_reg{1,2} before collect_linked_regs() runs and copies those snapshots back into both branch states afterwards. collect_linked_regs() records at most LINKED_REGS_MAX members of a linked registers group in the jump history and calls clear_scalar_id() for every member that does not fit. The compared register is not exempt from that. As a consequence, sync_linked_regs() might adjust ranges for more registers than bpf_bt_sync_linked_regs() can propagate precision to. Collect the linked registers before the snapshots are taken instead. This might lead to some unnecessary clear_scalar_id's, but from previous testing situations with many linked registers are extremely rare. Fixes: ec1d77cb0ee9 ("bpf: Use bpf_verifier_env buffers for reg_set_min_max") Reported-by: Nicholas Carlini <npc@anthropic.com> Suggested-by: Nicholas Carlini <npc@anthropic.com> Signed-off-by: Eduard Zingerman <eddyz87@gmail.com> Link: https://lore.kernel.org/bpf/20260904083325.2083493-3-eddyz87@gmail.com Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>