linux-next.git/kernel/bpf/fixups.c, branch master

bpf: Reject programs with inlined helpers if JIT is not available

2026-07-09T06:25:26+00:00

When an architecture (such as LoongArch, ARM64, and RISC-V) implements bpf_jit_inlines_helper_call(), the verifier skips rewriting the helper call offset (insn->imm) in bpf_do_misc_fixups(). This is because the helper is expected to be inlined by the JIT compiler later. Therefore, insn->imm remains as the raw helper enum ID. However, if JIT is disabled at runtime (net.core.bpf_jit_enable=0) or if JIT compilation fails dynamically (e.g., due to OOM), the program falls back to the BPF interpreter. When the interpreter executes (__bpf_call_base + insn->imm) with the unpatched raw ID, it jumps into an invalid address space, triggering an instruction alignment fault or a kernel panic. Although these helpers have valid C implementations in the kernel, the omission of offset rewriting makes runtime interpreter fallback fatal. Fix this by setting 'prog->jit_required = 1' when helper call rewriting is skipped for JIT inlining. This ensures that such programs are safely rejected if JIT is not available, preventing the runtime kernel panic. Fixes: 2ddec2c80b44 ("riscv, bpf: inline bpf_get_smp_processor_id()") Suggested-by: Alexei Starovoitov Suggested-by: KaFai Wan Acked-by: Leon Hwang Signed-off-by: Tiezhu Yang Signed-off-by: Eduard Zingerman

bpf: Introduce jit_required flag and remove bpf_prog_has_kfunc_call()

2026-07-09T06:24:41+00:00

Introduce a 'jit_required' bitfield flag in struct bpf_prog to track whether a BPF program strictly requires the JIT compiler to run. This prevents a dangerous runtime fallback to the interpreter for features that are only implemented in the JIT compiler. Currently, bpf_prog_has_kfunc_call() is used only for kernel function calls, replace the kfunc-specific helper with the new 'jit_required' flag. This makes it easy to support other JIT-only BPF features, such as inlined helpers. Suggested-by: Alexei Starovoitov Suggested-by: KaFai Wan Suggested-by: Leon Hwang Acked-by: Leon Hwang Signed-off-by: Tiezhu Yang Signed-off-by: Eduard Zingerman

bpf: Use BPF_CALL_IMM macro consistently in bpf_do_misc_fixups

2026-07-01T19:59:49+00:00

In bpf_do_misc_fixups(), the conversion from a function address to a BPF immediate value is handled using the BPF_CALL_IMM macro inside the 'patch_map_ops_generic' label block. However, immediately following it in the 'patch_call_imm' label block, the immediate value is calculated manually by subtracting __bpf_call_base from fn->func. Inspired by KaFai Wan's review comments on fixing helper call offsets, use the BPF_CALL_IMM macro in 'patch_call_imm' as well to clean this up. This removes the redundant manual pointer arithmetic and ensures coding style consistency across adjacent label blocks within the same function. Signed-off-by: Tiezhu Yang Signed-off-by: Andrii Nakryiko Link: https://lore.kernel.org/bpf/20260701035107.8069-1-yangtiezhu@loongson.cn

bpf: Add support for tracing_multi link session

2026-06-07T17:03:01+00:00

Adding support to use session attachment with tracing_multi link. Adding new BPF_TRACE_FSESSION_MULTI program attach type, that follows the BPF_TRACE_FSESSION behaviour but on the tracing_multi link. Such program is called on entry and exit of the attached function and allows to pass cookie value from entry to exit execution. Signed-off-by: Jiri Olsa Link: https://lore.kernel.org/r/20260606123955.345967-16-jolsa@kernel.org Signed-off-by: Alexei Starovoitov

bpf: Add multi tracing attach types

2026-06-07T17:03:01+00:00

Adding new program attach types multi tracing attachment: BPF_TRACE_FENTRY_MULTI BPF_TRACE_FEXIT_MULTI and their base support in verifier code. Programs with such attach type will use specific link attachment interface coming in following changes. This was suggested by Andrii some (long) time ago and turned out to be easier than having special program flag for that. Bpf programs with such types have 'bpf_multi_func' function set as their attach_btf_id and keep module reference when it's specified by attach_prog_fd. They are also accepted as sleepable programs during verification, and the real validation for specific BTF_IDs/functions will happen during the multi link attachment in following changes. Suggested-by: Andrii Nakryiko Signed-off-by: Jiri Olsa Link: https://lore.kernel.org/r/20260606123955.345967-11-jolsa@kernel.org Signed-off-by: Alexei Starovoitov

bpf: Refactor object relationship tracking and fix dynptr UAF bug

2026-06-02T01:31:41+00:00

Refactor object relationship tracking in the verifier and fix a dynptr use-after-free bug where file/skb dynptrs are not invalidated when the parent referenced object is freed. Add parent_id to bpf_reg_state to precisely track child-parent relationships. A child object's parent_id points to the parent object's id. This replaces the PTR_TO_MEM-specific dynptr_id. Remove ref_obj_id from bpf_reg_state by folding its role into the existing id field. Previously, id tracked pointer identity for null checking while ref_obj_id tracked the owning reference for lifetime management. These are now unified: acquire helpers and kfuncs set id to the acquired reference id, and release paths use id directly. Add reg_is_referenced() which checks if a register is referenced by looking up its id in the reference array. This replaces all former ref_obj_id checks. For release_reference(), invalidating an object now also invalidates all descendants by traversing the object tree. This is done using stack-based DFS to avoid recursive call chains of release_reference() -> unmark_stack_slots_dynptr() -> release_reference(). Referenced objects encountered during tree traversal are reported as leaked references. Add parent_id to bpf_reference_state to enable hierarchical reference tracking. When acquiring a reference, a parent_id can be specified to link the new reference to an existing one (e.g., referenced dynptrs acquire a reference with parent_id linking to the parent object's reference). Pointer casting: For pointer casting helpers (bpf_sk_fullsock, bpf_tcp_sock), instead of propagating ref_obj_id, the cast result reuses the same reference id as the source pointer. Since the cast may return NULL for a non-NULL input, the NULL case is explored as a separate verifier branch. This allows releasing any of the original or cast pointers to invalidate all others. Referenced dynptrs: When constructing a referenced dynptr, acquire a intermediate reference with parent_id linking to the parent referenced object. The dynptr and all clones share the same parent_id (pointing to the intermediate ref) but get unique ids for independent slice tracking. Releasing a referenced dynptr releases the parent reference, which in turn invalidates all clones and their derived slices. Owning to non-owning reference conversion: After converting owning to non-owning by clearing id (e.g., object(id=1) -> object(id=0)), the verifier releases the reference state via release_reference_nomark(). Note that the error message "reference has not been acquired before" in the helper and kfunc release paths is removed. This message was already unreachable. The verifier only calls release_reference() after confirming the reference is valid, so the condition could never trigger in practice. Fixes: 870c28588afa ("bpf: net_sched: Add basic bpf qdisc kfuncs") Signed-off-by: Amery Hung Acked-by: Eduard Zingerman Link: https://lore.kernel.org/r/20260529014936.2811085-6-ameryhung@gmail.com Signed-off-by: Alexei Starovoitov

Merge git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf 7.1-rc5

2026-05-25T13:33:30+00:00

Cross-merge BPF and other fixes after downstream PR. Signed-off-by: Alexei Starovoitov

bpf,x86: Fix exception unwinding with outgoing stack arguments

2026-05-17T20:53:24+00:00

When a main program with exception_boundary has outgoing stack arguments (e.g. from calling subprogs with >5 args), bpf_throw() fails to correctly restore callee-saved registers, causing a kernel crash. The x86 JIT allocates the outgoing stack arg area below the callee-saved registers via 'sub rsp, outgoing_rsp' in the prologue. When bpf_throw() unwinds, it captures the main program's sp (which includes this outgoing area) and passes it to the exception callback. The callback gets rsp and rbp, followed by pop_callee_regs, but rsp points into the outgoing arg area rather than the callee-saved registers, so the pops restore garbage values. Returning to the kernel with corrupted callee-saved registers causes a crash. Fix this by adjusting the sp (adding stack_arg_sp_adjust) passed to the exception callback, so it points to the bottom of the callee-saved registers instead of the outgoing arg area. When stack_arg_sp_adjust is 0 (the common case), this is a no-op. Fixes: 324c3ca6eed6 ("bpf,x86: Implement JIT support for stack arguments") Acked-by: Kumar Kartikeya Dwivedi Signed-off-by: Yonghong Song Link: https://lore.kernel.org/r/20260517150702.288031-1-yonghong.song@linux.dev Signed-off-by: Alexei Starovoitov

bpf: Clean up redundant stack arg checks for non-JITed programs

2026-05-17T00:46:16+00:00

Remove a redundant stack_arg_cnt check in __bpf_prog_select_runtime() and start the stack arg loop from index 0 in bpf_fixup_call_args(). Both changes are no-ops that simplify the code: In __bpf_prog_select_runtime(), the subprog_info[0].stack_arg_cnt check is unreachable: - when there is only a main program (no bpf-to-bpf calls), subprog_info[0].stack_arg_cnt is always 0 because the main program's arg_cnt is forced to 1 - when bpf-to-bpf calls use stack args and JIT succeeds, fp->bpf_func is set and this code is skipped - when JIT fails, bpf_fixup_call_args() rejects the program before we get to __bpf_prog_select_runtime(). In bpf_fixup_call_args(), starting the loop at i=1 skipped subprog 0, which is safe since the main program always has arg_cnt=1 and thus bpf_in_stack_arg_cnt() returns 0. Starting at i=0 removes the need to reason about this invariant. Signed-off-by: Yonghong Song Link: https://lore.kernel.org/r/20260515225101.824054-1-yonghong.song@linux.dev Signed-off-by: Alexei Starovoitov

bpf: Reject stack arguments in non-JITed programs

2026-05-13T16:27:31+00:00

The interpreter does not understand the bpf register r11 (BPF_REG_PARAMS) used for stack arguments. So reject interpreter usage if stack arguments are used either in the main program or any subprogram. Signed-off-by: Yonghong Song Link: https://lore.kernel.org/r/20260513045049.2390444-1-yonghong.song@linux.dev Signed-off-by: Alexei Starovoitov