| Age | Commit message (Collapse) | Author |
|---|
|
When add_taint_module() was added, the documentation for it was copied
directly from add_taint(). But that documentation was a bit wrong
(wrong variable name, and misleading boolean reference.) Fix up both
functions to have the correct documentation for how to use the
lockdep_ok parameter to these functions.
Reported-by: Petr Pavlu <petr.pavlu@suse.com>
Reviewed-by: Bradley Morgan <brads@mainlining.org>
Reviewed-by: Aaron Tomlin <atomlin@atomlin.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260914-bind_taint-v4-2-eadf8a090903@linuxfoundation.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
|
|
Other kernel code might want to call add_taint_module() so pull it out
and make it global. If modules are not enabled, this defaults to a call
to add_taint(), so all is fine.
Reviewed-by: Johan Hovold <johan@kernel.org>
Tested-by: Johan Hovold <johan@kernel.org>
Reviewed-by: Bradley Morgan <brads@mainlining.org>
Reviewed-by: Petr Pavlu <petr.pavlu@suse.com>
Acked-by: Danilo Krummrich <dakr@kernel.org>
Link: https://patch.msgid.link/20260914-bind_taint-v4-1-eadf8a090903@linuxfoundation.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
|
|
Mark linux_binprm->mm as a trusted-or-null nested pointer so BPF programs
can pass it to kfuncs after a NULL check.
The field is either NULL or points to a live mm_struct whenever BPF can
access a linux_binprm. On successful exec, exec_mmap() installs the new
address space before begin_new_exec() clears bprm->mm. The bprm_mm_init()
error path clears the field before mmdrop(), and free_bprm() clears it
before mmput(), as ensured by an earlier patch in this series.
Update the existing LSM selftest to check bprm->mm for NULL before
dereferencing it, as required for trusted-or-null pointers.
Signed-off-by: Anastasios Papagiannis <tasos.papagiannnis@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Reviewed-by: Sun Jian <sun.jian.kdev@gmail.com>
Reviewed-by: Matt Bobrowski <matt@bobrowski.net>
Link: https://patch.msgid.link/20260918131757.42802-5-tasos.papagiannnis@gmail.com
|
|
On CONFIG_MMU kernels, when security_bprm_check() runs, the argument and
environment strings for the exec have been copied into bprm->mm. The new
address space is not associated with a task_struct until exec_mmap(), so
existing BPF user memory helpers cannot access it.
Add bpf_copy_from_user_mm() and bpf_copy_from_user_mm_str() kfuncs. Both
take a struct mm_struct pointer directly, allowing callers to access
trusted address spaces that are not associated with a task_struct.
bpf_copy_from_user_mm() has similar semantics to
bpf_copy_from_user_task(). bpf_copy_from_user_mm_str() copies one
NUL-terminated string and returns its size including the NUL terminator.
It accepts BPF_F_PAD_ZEROS to clear unused destination bytes on success.
Refactor the task-based helpers and the new mm-based kfuncs to share
static internal implementations. The task-based interfaces validate their
arguments before acquiring and holding a reference to the task's mm for
the copy. No behavior change is intended for the existing task-based
interfaces.
Register both new kfuncs and mark them KF_SLEEPABLE because accessing a
remote address space can fault.
Signed-off-by: Anastasios Papagiannis <tasos.papagiannnis@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Reviewed-by: Matt Bobrowski <matt@bobrowski.net>
Link: https://patch.msgid.link/20260918131757.42802-4-tasos.papagiannnis@gmail.com
|
|
Proxy execution can move a blocked donor's scheduling context to the
lock owner's CPU even when the donor is migration-disabled. The donor
does not execute there, and its original execution CPU remains recorded
in wake_cpu.
set_task_cpu() warns unconditionally for migration-disabled tasks, so a
subsequent proxy migration or the wakeup path returning the donor home
triggers a false positive: moving a blocked scheduling context does not
violate the migration-disabled execution context.
For example, creating a mutex owner on CPU1 and a migration-disabled
waiter on CPU0 can trigger the following warning:
proxy_migrate_repro: donor blocking on CPU0 with migration disabled
proxy_migrate_repro: donor moved from CPU0 to CPU1
WARNING: kernel/sched/core.c:3389 at set_task_cpu+0x1d3/0x280
...
Call Trace:
try_to_wake_up+0x43f/0x780
__mutex_unlock_slowpath+0x330/0x540
owner_fn+0x9f/0xc0 [proxy_migrate_repro]
...
proxy_migrate_repro: donor woke on CPU0, task_cpu=0
proxy_migrate_repro: completed
Exclude blocked proxy donors from the warning. The proxy wakeup path
restores an executable placement before clearing the blocked state.
Fixes: b049b81bdff6 ("sched: Handle blocked-waiter migration (and return migration)")
Signed-off-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: John Stultz <jstultz@google.com>
Link: https://patch.msgid.link/20260915184101.2621252-1-arighi@nvidia.com
|
|
A typical module unload occurring event when there is an active perf
connection leads to freeing of the pmu pointer. The call log is something
like:
..
__pmu_detach_event
pmu_detach_event
pmu_detach_events
perf_pmu_unregister
..
__pmu_detach_event() sets event->pmu to null. When the perf connection
finally is closed, the following stack trace is observed:
Oops: general protection fault, kernel NULL pointer dereference
...
RIP: 0010:_free_event+0x3e/0x370
...
Call Trace:
...
perf_event_release_kernel+0x260/0x2d0
perf_release+0x12/0x20
A call to mediated_pmu_unaccount_event() inside _free_event() is the root
cause of this crash. Adding a check inside is_include_guest_event() ensures
we don't accidentally access a null pmu ptr. In addition to this, we will
now call mediated_pmu_unaccount_event() before clearing the pmu ptr so that
nr_include_guest_events counts are maintained correctly.
Fixes: eff95e170275 ("perf: Add APIs to create/release mediated guest vPMUs")
Assisted-by: Claude:Claude-Sonnet-5
Signed-off-by: Vinay Belgaumkar <vinay.belgaumkar@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Dapeng Mi <dapeng1.mi@linux.intel.com>
Link: https://patch.msgid.link/20260904181625.1394082-1-vinay.belgaumkar@intel.com
|
|
A nested bpf_for_each_map_elem() callback can unlock a different element
of the same map:
static long inner(void *map, int *key, struct value *v,
struct value **outer_value)
{
bpf_spin_lock(&v->lock);
bpf_spin_unlock(&(*outer_value)->lock);
return 0;
}
static long outer(void *map, int *key, struct value *v, void *ctx)
{
bpf_for_each_map_elem(map, inner, &v, 0);
return 0;
}
Both callback values currently have ID zero and the same map_ptr.
process_spin_lock() compares those two fields, so it accepts the unlock
even though the two callbacks can receive different map elements.
Assign a fresh ID to every callback map value in the for-each,
timer/workqueue, and task-work constructors. Copies of one callback
argument retain its ID, so locking and unlocking through that argument
continues to work. Distinct callbacks also get distinct IDs for
single-element arrays, including inner arrays sharing inner_map_meta.
Preserve map_uid for every inner-map lookup and compare it through
check_ids() during state pruning. This preserves relationships between
maps, keys, and values while allowing equivalent states with different
lookup IDs to match. It avoids field-specific rules for when an inner map
needs an identity.
Move map_uid out of the metadata union and next to the other IDs, so
register comparisons can use the existing memcmp() ranges and remap the
IDs separately. Clear it when resetting a register or converting a map
lookup result to a socket pointer. Shrink frameno to u8, which is enough
for MAX_CALL_FRAMES, to make room without growing bpf_reg_state.
Fixes: d0d78c1df9b1 ("bpf: Allow locking bpf_spin_lock global variables")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260917233222.2542500-9-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
check_subprogs() verifies that each subprogram ends in an exit or an
unconditional jump before in-kernel CO-RE relocations are applied. An
unresolved relocation can then replace that terminal instruction with an
invalid helper call. The resulting fall-through into another subprogram
breaks the CFG invariant used by postorder and stack liveness analysis,
which can write past their per-subprogram arrays.
Apply CO-RE relocations immediately after preparing the program BTF, before
subprogram discovery and validation. Keep func_info and line_info validation
after subprogram discovery because those records depend on the complete
subprogram layout.
Reject an ldimm64 first slot at the end of the instruction stream before
CO-RE can inspect its missing second slot. check_subprogs() previously
rejected this form before relocation processing because it is not a valid
subprogram terminator. Moving CO-RE ahead of check_subprogs() removes that
implicit protection, so perform an explicit check before applying
relocations.
Include core_relo_cnt when deciding whether to prepare program BTF. A load
that supplied only CO-RE relocation metadata previously skipped both BTF
setup and relocation processing.
Fixes: fbd94c7afcf9 ("bpf: Pass a set of bpf_core_relo-s to prog_load command.")
Suggested-by: Andrii Nakryiko <andrii@kernel.org>
Suggested-by: Alexei Starovoitov <ast@kernel.org>
Suggested-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260917233222.2542500-5-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
regsafe() maps packet pointer IDs between states and checks that each
current register range is a subset of the corresponding explored
register range. It does not, however, preserve the displacement between
registers that share a packet pointer ID.
This is unsound because packet range is shared by ID. A bounds check on
one class member updates every member, and a later access can consume the
range through another member. Commit 022ac0750883 ("bpf: use reg->var_off
instead of reg->off for pointers") folded the fixed pointer offset into
r64 and removed the old off equality check, so two individually narrower
registers can prune even when their displacement has changed. The
explored path can then license an out-of-bounds packet access on the
pruned path.
Require matching range bases for packet pointers with an ID. Together
with the existing ID mapping, this preserves the displacement between
members of each packet-pointer class without adding per-ID state.
Packet pointers without an ID remain unaffected.
Fixes: 022ac0750883 ("bpf: use reg->var_off instead of reg->off for pointers")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260917233222.2542500-3-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
After do_check() returns, the verifier runs several instruction rewrite
passes. Some of them patch or remove one instruction at a time. Each
operation moves the remaining instruction and auxiliary-data arrays and
adjusts all branch offsets, making the overall work quadratic in the
program length.
A privileged loader can submit 131072 unconditional jumps by zero followed
by a valid return. Verification finishes quickly, but bpf_opt_remove_nops()
then spends a long time removing each jump separately. Since this
post-verification work neither checks for signals nor reschedules, a pending
SIGKILL cannot terminate the task until the rewrite finishes.
Make bpf_patch_insn_data() and verifier_remove_insns() common cancellation
and rescheduling points. These helpers run from BPF_PROG_LOAD process
context, and bpf_patch_insn_data() can already sleep while reallocating
auxiliary data.
Report interrupted constant blinding as -EINTR and propagate it through
both JIT paths, including kernels that permit interpreter fallback.
Other blinding failures retain the existing fallback behavior.
This does not reduce the quadratic cost of the rewrite passes, but it makes
the work preemptible and allows a killed loader to be torn down promptly.
Fixes: 52875a04f4b2 ("bpf: verifier: remove dead code")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260917233222.2542500-2-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
fprobe_fgraph_entry() reserves shadow stack space for every fprobe with
an exit handler, but only fills it for those whose entry handler returns
0. fgraph_reserve_data() does not clear the area, so fprobe_return()
parses the unused tail as headers left over from an earlier call, and an
exit handler can run twice or despite its entry handler asking to skip
it.
Write a zero word after the last entry to terminate the walk. A zeroed
slot does not decode to a NULL fprobe on the arches that encode the
header into one unsigned long, since arch_decode_fprobe_header_fp() ORs
in FPROBE_HEADER_MSB_PATTERN, so make read_fprobe_header() return NULL
for a zeroed slot.
Link: https://lore.kernel.org/all/20260917212407.384468-1-devnexen@gmail.com/
Fixes: e0a384434ae1 ("tracing: fprobe: do not zero out unused fgraph_data")
Cc: stable@vger.kernel.org
Suggested-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: David Carlier <devnexen@gmail.com>
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
The lazy RCU shrinker decremented sc->nr_to_scan (unsigned long)
and then tested the result with <= 0. When a single CPU's lazy
callback count exceeds the remaining budget, the subtraction wraps
to a large positive value and the <= 0 comparison, which is
equivalent to == 0 for an unsigned type, never fires again. The
scan loop then iterates through every nocb CPU instead of honouring
the reclaim budget.
Accumulate into count and stop once count >= nr_to_scan.
Fixes: c945b4da7a448 ("rcu: Shrinker for lazy rcu")
Assisted-by: Zcode:GLM-5.2
Signed-off-by: Longlong Xia <xialonglong@kylinos.cn>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
bpf_link_prime() inserts a link into link_idr before anon_inode_getfile()
succeeds and before bpf_link_settle() publishes the ID in link->id.
bpf_link_by_id() treats such an ID-zero link as unsettled, but the link
iterator takes a reference without this check.
If anon_inode_getfile() then fails, the creator removes the ID and frees
its still-private link directly. The iterator is left with a dangling
reference and its next bpf_link_put() accesses freed memory.
Treat ID-zero entries as transient in bpf_link_get_curr_or_next(), just as
bpf_link_by_id() does.
BUG: KASAN: slab-use-after-free in bpf_link_put
Write of size 8 by task exp/384
Call Trace:
bpf_link_put kernel/bpf/syscall.c:3372
bpf_link_seq_next kernel/bpf/link_iter.c:33
bpf_seq_read kernel/bpf/bpf_iter.c:158
vfs_read fs/read_write.c:572
ksys_read fs/read_write.c:716
do_syscall_64 arch/x86/entry/syscall_64.c:84
entry_SYSCALL_64_after_hwframe arch/x86/entry/entry_64.S:121
Kernel panic - not syncing: KASAN: panic_on_warn set ...
Fixes: 9f8836127308 ("bpf: Add bpf_link iterator")
Reported-by: Xiang Mei <xmei5@asu.edu>
Signed-off-by: Weiming Shi <bestswngs@gmail.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260914170206.170723-2-bestswngs@gmail.com
|
|
Global subprograms are verified independently with a fresh verifier root.
do_check_common() currently seeds that root's in_sleepable state from the
program, even though a global subprogram can also run from callbacks whose
execution context differs from the program's main entry point.
In particular, workqueue and task-work callbacks are sleepable even when
the containing program is not. A global subprogram of that program is
therefore verified as non-sleepable, making in_rcu_cs() true and allowing
loads of RCU-protected kptrs to produce trusted MEM_RCU pointers. The same
subprogram can then be called from a sleepable callback without a classic
RCU reader. It can retain such a pointer while the object is freed and use
it after free.
The verifier's execution-context predicates are complementary. A state is
sleepable only when in_sleepable is set and no RCU, preemption, IRQ, or lock
region is active. Each condition which prevents sleeping also provides RCU
protection, while in_rcu_cs() treats a non-sleepable state as implicitly
protected.
Use this relationship to represent a global subprogram caller with only the
result of in_sleepable_context(). A protected sleepable caller is normalized
to in_sleepable=false at the independent verification root. This both
prevents sleepable operations and makes in_rcu_cs() true without copying
caller-owned lock state.
Track only the contexts in which each global subprogram is actually
reached. Verify it once if all reachable calls use the same context, and
twice only if both sleepable and non-sleepable calls reach it. Calls found
while verifying globals or asynchronous callbacks mark further contexts
for checking. Repeat the existing subprogram walk until all called
contexts have been verified; unreachable global calls remain unchecked.
Accumulate instruction counts over those verification passes. Preserve
the total recorded before each pass, since path accounting has already
added this pass's synchronous instructions and its root total must also
include asynchronous subprograms.
This makes an unprotected callback verify the global subprogram as
sleepable, turning its RCU-protected kptr load into an untrusted pointer.
Protected callers and global subprograms which do not depend on implicit RCU
protection remain valid.
Fixes: 81f1d7a583fa ("bpf: wq: add bpf_wq_set_callback_impl")
Fixes: 38aa7003e369 ("bpf: task work scheduling kfuncs")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260914131923.2544250-2-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
ops.dequeue() is invoked with the source user DSQ's lock still held on
the consume and move paths (scx_consume_dispatch_q(),
move_task_between_dsqs()). A BPF scheduler which locks the source user
DSQ from ops.dequeue() - e.g. by iterating it with bpf_iter_scx_dsq -
self-deadlocks.
ops.dequeue() can only call the "any" kfuncs and none of them can lock a
builtin DSQ, so the global and bypass paths can't deadlock; however,
all DSQ locks share one lockdep class, so iterating any user DSQ from
ops.dequeue() on those paths trips the recursion check.
Move the invocation after the DSQ unlock on all three paths.
SCX_TASK_IN_CUSTODY is cleared under the lock serializing the transfer
so that the callback is invoked exactly once.
Fixes: ebf1ccff79c4 ("sched_ext: Fix ops.dequeue() semantics")
Cc: stable@vger.kernel.org # v7.1+
Acked-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: fangqiurong <fangqiurong@kylinos.cn>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
schedule_deferred_locked() skips scheduling a deferred action while
SCX_RQ_IN_WAKEUP is set and relies on the task_woken_scx() call that follows
a wakeup enqueue to run it. enqueue_task_scx() sets the flag from the merged
enqueue flags, which include the flags stashed for a remote activation.
move_remote_task_to_local_dsq() thus sets SCX_RQ_IN_WAKEUP on the
destination rq when the moved task was woken up, although no
task_woken_scx() follows that activation.
An IMMED insert into a busy destination requests a local reenqueue during
that enqueue. The request gets linked but not scheduled and stays pending
until an unrelated wakeup or preemption on that CPU runs the deferred
actions. The IMMED task sits behind the running task in the meantime. If
nothing runs them before the scheduler is disabled, the request outlives the
scheduler and points into its freed per-cpu area, which the next scheduler
dereferences from run_deferred().
Test the core enqueue flags for the wakeup bit. Only the core's wakeup path
is followed by task_woken_scx().
Fixes: 57ccf5ccdc56 ("sched_ext: Fix enqueue_task_scx() truncation of upper enqueue flags")
Cc: stable@vger.kernel.org # v7.1+
Reported-by: Andrea Righi <arighi@nvidia.com>
Link: https://lore.kernel.org/all/20260916145807.3250167-1-arighi@nvidia.com/
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/mszyprowski/linux
Pull dma-mapping fixes from Marek Szyprowski:
"A few fixes for the DMA-mapping code:
- resolved regression in accessing encrypted memory by IOMMU-backed
devices (Aneesh Kumar K.V)
- improved failure handling and removed rare bug in swiotlb/highmem
(Donggeun Yoo)"
* tag 'dma-mapping-7.3-2026-09-17' of git://git.kernel.org/pub/scm/linux/kernel/git/mszyprowski/linux:
x86/mm: Don't force unencrypted DMA for IOMMU-backed devices
dma-mapping: don't trace the DMA address when the allocation fails
swiotlb: use the adjusted address for the highmem page lookup
dma-coherent: report a failed reserved memory assignment
|
|
bpf_local_storage_update() takes the map bucket lock while holding
local_storage->lock. bpf_selem_unlink_map() does the same; its only
caller holds local_storage->lock. The outer lock is acquired with
raw_res_spin_lock_irqsave(), so interrupts are already disabled at both
sites.
Using raw_res_spin_lock_irqsave() for the nested lock saves the already
disabled IRQ state and issues another IRQ disable. The matching unlock
tests that saved state before leaving interrupts disabled. On x86-64,
this adds a pushfq/popq/cli sequence and a test/branch around an
unreachable sti to each acquisition.
Use raw_res_spin_lock() and raw_res_spin_unlock() instead. They retain
preemption nesting, memory ordering and resilient-lock bookkeeping. The
outer unlock remains responsible for restoring the caller's IRQ state.
In the tested clang x86-64 build, this removes five executed instructions
from each uncontended nested acquisition. It also shrinks
bpf_local_storage_update() from 1732 to 1702 bytes and bpf_selem_unlink()
from 1030 to 992 bytes. The affected paths are updates that add or replace
an element in existing owner storage and successful unlinks.
Document the owner-lock requirement of bpf_selem_unlink_map() and assert
that interrupts are disabled.
Signed-off-by: Usama Arif <usama.arif@linux.dev>
Reviewed-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/bpf/20260916181001.201787-1-usama.arif@linux.dev
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Now that POSIX CPU timers cannot be enqueued on a task after PF_EXITING is
set and process wide timers cannot be enqueued when PF_EXITING is set and
the last thread in the group is exiting, it is possible to mop up POSIX
timers in do_exit() completely.
This requires to cancel an eventually pending POSIX CPU timer task work
right there because do_exit() invokes exit_task_work() later, which would
be acting on torn down data.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Kijo Park <red993688@gmail.com>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Link: https://patch.msgid.link/20260911090541.947206186@kernel.org
|
|
To prepare for cleaning up POSIX CPU timers in do_exit(), prevent
enqueueing POSIX CPU timers on a task which has PF_EXITING set. Queueing a
timer on such a task is pointless because the task won't expire the timer
anymore.
The same applies to process wide timers when tsk::signal::flags has
SIGNAL_GROUP_EXIT set.
Pretending that the timer is armed allows to keep the POSIX timer mechanism
"working" so that the timer stays accessible up to the point where a task
is unhashed.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Kijo Park <red993688@gmail.com>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Link: https://patch.msgid.link/20260911090541.892445443@kernel.org
|
|
Right now POSIX CPU timers use task::exit_state to check whether a task is
exiting. That works correctly, but exit_state is set later in do_exit() and
too late for allowing to cleanup POSIX timers earlier.
It does not matter in case of exit whether the cutoff is a bit earlier.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Kijo Park <red993688@gmail.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Link: https://patch.msgid.link/20260911090541.837831386@kernel.org
|
|
They are only used in the POSIX CPU timer code. No point in exposing them
globally and parsing them for nothing.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Kijo Park <red993688@gmail.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Link: https://patch.msgid.link/20260911090541.785749315@kernel.org
|
|
Hyunwoo debugged the following KASAN UAF splat:
BUG: KASAN: slab-use-after-free in __send_signal_locked+0xb27/0xba0
Write of size 8 at addr ffff888007ed80c8 by task poc/79
...
Call Trace:
__send_signal_locked+0xb27/0xba0
do_send_sig_info+0xa7/0x160
do_send_specific+0x76/0xa0
__x64_sys_tgkill+0x193/0x270
...
Allocated by task 80:
do_timer_create+0x1a4/0x1030
__x64_sys_timer_create+0x145/0x190
...
Freed by task 12:
kmem_cache_free_bulk+0x1f8/0x4a0
kvfree_rcu_bulk+0x14f/0x1c0
kfree_rcu_work+0x128/0x1a0
...
Last potentially related work creation:
kvfree_call_rcu+0x39/0x390
__flush_itimer_signals+0x211/0x320
flush_itimer_signals+0x47/0x90
begin_new_exec+0xa6b/0x28c0
It turned out that this happens with a non-leader exec() as Hyunwoo
explained:
de_thread() calls exchange_tids() before release_task(leader), so the
struct pid held by a SIGEV_THREAD_ID timer created against the leader's tid
now points to the thread which called execve(). pid_task() returns that
thread and lock_task_sighand() on it succeeds.
If the timer signal is blocked, its sigqueue stays queued on the leader's
task::pending. The next expiry of that timer can then run while
release_task() flushes the queue.
posixtimer_send_sigqueue() checks whether the sigqueue is already queued
with a plain list_empty(), which only reads list_head::next.
list_del_init() is not atomic and INIT_LIST_HEAD() stores list_head::next
before list_head::prev, so the check can pass in between. list_add_tail()
queues the entry on the task::pending of the live thread, and the
list_head::prev store from the flush then overwrites the list_head::prev
link that list_add_tail() has just set.
__flush_itimer_signals() does not undo that either. With list_head::prev
pointing at the entry itself, its list_del_init() only stores the same
values again, so the entry is not removed from the list. It is still there
after the last reference is dropped and the timer is freed by RCU, and the
list_add_tail() of a later tgkill() follows that list_head::prev into the
freed timer.
This problem surfaced with the recent commit which moved the sigqueue flush
out of the sighand lock held region.
Hyonwoo proposed to fix this by using list_del_init_careful(), but that
just papers over the problem. After some disucssions and various attempts
to solve it, Eric pointed out that there is no reason to flush
task::pending late in release_task() and it should be done in
exit_signals() already.
As nothing can collect and deliver signals which are queued in a dying
task's pending queue, there is no reason to delay it further.
But it has to be ensured that no signals can be queued into it after that
point. exit_signals() sets PF_EXITING in task::flags, which can be used as
an indicator for this.
Cure it by:
- Preventing signal queueing for task private signals (PIDTYPE_PID) when
the task has PF_EXITING set in __send_signal_locked() and in
posixtimer_send_sigqueue().
- Protecting the unlocked setting of PF_EXITING in exit_signals() for the
task group empty and the group exit case with sighand lock
- Flushing task::pending signals right there.
Optimize that by moving the whole pending list to an on-stack list head
under sighand lock and free the signals without the lock held.
There has been quite some discussion about the lockless flush and the
non-leader exec case on weakly ordered systems. The problem is that a third
party which tries to send a posix timer signal relies on the PID lookup to
find the target task and that lookup might result in the new leader when
the signal was originaly directed to the old leader. In case that the
signal was queued on the old leader then the lockless flush raised a
concern over the following situation:
old_leader new_leader third party
A: flush_list() // list_del_init() stores to sigqueue
LOCK (tasklist)
old_leader->exit_state = EXIT_ZOMBIE;
B: UNLOCK (tasklist)
C: LOCK (tasklist)
if (old_leader->exit_state)
transfer_tids()
D: store PID
posix_timer_send_sigqueue()
// Observes #D so t = new_leader
E: t = get_target()
F: LOCK (sighand)
G: if (list_empty(sigqueue))
list_add(sigqueue)
The concern was that the third party might observe #D but not observe #A
and therefore would proceed to #G while the list_del() stores (#A) in
flush_list() are not visible yet, which could result in list corruption.
That would be possible if looking at it solely from a RELEASE+ACQUIRE
ordering point of view, but B-C is a UNLOCK+LOCK hand-over, which is not
the same as RELEASE+ACQUIRE:
RELEASE+ACQUIRE: RCpc, only the CPUs involved agree on the ordering
UNLOCK+LOCK: RCtso, the hand-over is store-ordering
As B-C is UNLOCK+LOCK, which is RCtso and that does impose store order,
A stores must happen before the D store.
Combine with E-F, which has a data dependency from the LOAD to the LOCK and
thereby constraints later LOADs, those sigqueue loads in G that come after
F must in fact observe the A stores.
Fixes: fb3bbcfe344e ("exit: change the release_task() paths to call flush_sigqueue() lockless")
Reported-by: Hyunwoo Kim <imv4bel@gmail.com>
Debugged-by: Hyunwoo Kim <imv4bel@gmail.com>
Suggested-by: "Eric W. Biederman" <ebiederm@xmission.com>
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Kijo Park <red993688@gmail.com>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260911090541.572536604@kernel.org
Closes: https://patch.msgid.link/aok1rdkBgZsynHZB@v4bel
|
|
Move the POSIX timer group exit handling into the posix timer code and
provide a proper stub when POSIX timers are disabled in Kconfig.
No functional change.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Kijo Park <red993688@gmail.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Link: https://patch.msgid.link/20260911090541.733966325@kernel.org
|
|
Move it to the POSIX timer code and provide a proper stub when POSIX timers
are disabled in Kconfig.
No functional change.
Signed-off-by: Thomas Gleixner <tglx@kernel.org>
Tested-by: Kijo Park <red993688@gmail.com>
Reviewed-by: Frederic Weisbecker <frederic@kernel.org>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Link: https://patch.msgid.link/20260911090541.680668094@kernel.org
|
|
Correct "dependend" to "dependent", reported by scripts/checkpatch.pl using
the misspelling list in scripts/spelling.txt. Only touches comments, no
code changes.
Assisted-by: Cursor:claude-opus-5
Signed-off-by: Hemanth Selam <hemanth.selam@gmail.com>
Reviewed-by: Bradley Morgan <brads@mainlining.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
|
|
As Jann demonstrated, it is entirely feasible to access the mm through vfork().
Therefore we need to allocate a private hash on vfork() as well as any other
CLONE_VM user.
Specifically, it must be avoided to have (private) futex waiters before
allocating the private hash.
Fixes: ee9dce44362b ("futex: Drop CLONE_THREAD requirement for private default hash alloc")
Reported-by: Jann Horn <jannh@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260911090447.GT788244@noisy.programming.kicks-ass.net
|
|
ebf1ccff79c4 ("sched_ext: Fix ops.dequeue() semantics") moved the final
ops_state store in scx_dispatch_enqueue() after the DSQ unlock so that the
custody update and ops.dequeue() precede it. A task can thus be found on a
DSQ while still SCX_OPSS_DISPATCHING.
The dequeue and core-sched pick paths wait for the state to clear in
ops_dequeue() but the reenqueue paths don't. A reenqueue in that window runs
ops.enqueue() and sets SCX_OPSS_QUEUED before the dispatch has completed.
The dispatcher's final store then overwrites it with SCX_OPSS_NONE and
finish_dispatch() drops every later dispatch of the task.
Wait for SCX_OPSS_DISPATCHING to clear before dequeueing a task for
reenqueue, the same way ops_dequeue() does.
Fixes: ebf1ccff79c4 ("sched_ext: Fix ops.dequeue() semantics")
Cc: stable@vger.kernel.org # v7.1+
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext
Pull sched_ext fixes from Tejun Heo:
- An error raised by a BPF program before the scheduler finished
enabling was consumed by the disable path's pre-enable shortcut,
leaving a running scheduler that couldn't be disabled and was later
freed while in use.
- Two compat kfuncs dereferenced a NULL scheduler when handed an exited
or idle task, oopsing the kernel.
- Keep-running decisions in the dispatch path used the root scheduler's
flags for tasks belonging to a sub-scheduler, causing warnings and
stalls.
- Schedulers with their own CPU ID mapping had no way to learn which
IDs are online. Add a kernel-maintained online mask to plug the hole.
- Cgroup idle state: the initial cpu.idle state wasn't passed on cgroup
init and same-value rewrites delivered spurious callbacks.
- Example scheduler fixes for a reenqueue loop on attach, placements on
CPUs without effective grants, stalled partition work and stale idle
tracking.
* tag 'sched_ext-for-7.3-rc3-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext:
sched_ext: Maintain an online cid mask in the scheduler arena
sched_ext: scx_qmap: Restore unused idle claims from ops.dispatch()
sched_ext: Close the pre-enable ops error claim window
sched_ext: scx_qmap: Fix pending partition work handoff
sched_ext: scx_qmap: Place only on cids whose caps are in effect
sched_ext: scx_qmap: Do not add IMMED to rescue inserts
sched_ext: Use @prev's scheduler for the keep decisions in dispatch_one()
sched_ext: Rename sch to root_sch in dispatch_one()
sched_ext: Fix NULL sched deref in kfunc sub-sched error paths
sched_ext: Don't deliver duplicate ops.cgroup_set_idle() for same value
sched_ext: Pass the initial cpu.idle state in scx_cgroup_init_args
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup
Pull cgroup fix from Tejun Heo:
- The task iterator could pick up a dying task whose refcount had
already dropped to zero and resurrect it, leading to a use-after-free
when reading cgroup.procs. Skip such tasks.
* tag 'cgroup-for-7.3-rc3-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup:
cgroup: Avoid iteration of dying tasks with zero refcount
|
|
Schedulers on the default cid mapping treat [0, nr_online_cids) as the
online set and restart on hotplug. Schedulers that install their own mapping
with scx_bpf_cid_override() have no way to learn which cids are online: the
count no longer identifies members and the CPU-form cpumask is unusable from
cid programs. This is an obvious hole in the cid API.
Add scx_bpf_online_cmask(), a kernel-maintained cmask in the scheduler's
arena, allocated alongside the per-CPU scratch masks and populated after the
cid mapping is finalized and before ops.init(), for child schedulers too.
The pointer stays valid through ops.exit() with no reference to take. It is
the arena offset as a void pointer, the same form struct_ops arena arguments
arrive in. The verifier types the void return as a scalar for the program's
arena cast.
The mask follows the SCX hotplug notifications: seeded from cpu_active_mask
and updated before ops.cid_online/offline() runs, so it lags cpu_online_mask
only inside a hotplug transition. Updates walk the scheduler list under the
lock that also serializes unlinking. Reads are live, not atomic snapshots.
Root initialization excludes hotplug.
v2: Reworded the getter kerneldoc (Andrea Righi).
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
scx_qmap tracks idle cids itself. pick_direct_dispatch_cid() claims a cid by
clearing its bit and the task is inserted into that cid's local DSQ, which
kicks the CPU. When the task does not arrive, for example because the insert
fell back to the global DSQ after an affinity change, the CPU wakes, finds
nothing and picks idle again. That is not an idle transition, so
ops.update_idle() is not called and the cid stays marked busy until an
unrelated task runs on it.
Restore the claim from ops.dispatch(). The kick guarantees a dispatch on the
kicked CPU, and when it finds nothing to run with a NULL @prev, the CPU is
going back to idle. Document the pattern in ops.update_idle(), which reports
only actual transitions.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Cc: Andrea Righi <arighi@nvidia.com>
|
|
Return MAX_JIFFY_OFFSET for all the values truncated when val (u64) is
passed to msecs_to_jiffies (u32). This aligns with how very large
millisecond values get translated into MAX_JIFFY_OFFSET.
Fixes: b96b5c6708ea ("sysctl: Replace do_proc_do{int,ulong,uint}vec with do_proc_vec")
Suggested-by: Kuniyuki Iwashima <kuniyu@google.com>
Signed-off-by: Joel Granados <joel.granados@kernel.org>
|
|
Add the range check back to do_proc_ulong_conv_ms_jiffies that commit
b96b5c6708ea ("sysctl: Replace do_proc_do{int,ulong,uint}vec with
do_proc_vec") incorrectly removed. Append "_minmax" to the end of
do_proc_ulong_conv_ms_jiffies so it is clear that there should be a
range check.
Fixes: b96b5c6708ea ("sysctl: Replace do_proc_do{int,ulong,uint}vec with do_proc_vec")
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Signed-off-by: Joel Granados <joel.granados@kernel.org>
|
|
Add the range check to do_proc_int_conv_ms_jiffies_minmax that commit
d174174c6776 ("sysctl: replace SYSCTL_INT_CONV_CUSTOM macro with
functions") incorrectly removed.
Fixes: d174174c6776 ("sysctl: replace SYSCTL_INT_CONV_CUSTOM macro with functions")
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Signed-off-by: Joel Granados <joel.granados@kernel.org>
|
|
The commit 260fbcb92bbea ("cgroup: Move dying_tasks cleanup from
cgroup_task_release() to cgroup_task_free()") extended the lifetime of
tasks on the dying_tasks list.
The iterators have provision to go through dying_tasks because of
dying threadgroup leaders or explicit CSS_TASK_ITER_WITH_DEAD, however,
it was expected that such tasks can obtain a new reference (that is
possible before cgroup_task_release()/put_task_struct_rcu_user()).
The tasks after cgroup_task_release() and before cgroup_task_free()
are subject to race when they may or may not have ->usage count > 0.
The race window is between css_task_iter_next() invocations
when css_set_lock is released and we may arrive at a new ->task_pos.
The iterator should not attempt to resurrect tasks whose ->usage count
dropped to zero. (When that happens, __put_task_struct_rcu_cb() is
already imminent and the returned task_struct would could be used
after free.)
As for the fix, we cannot simply check the signal->live count of a task
on the dying list because that won't distinguish regular zombies waiting
to be reaped from RCU remnant tasks that are going to be free'd.
Therefore add an extra check to rule out ->usage==0 tasks from any
iteration.
The repeat: loop in css_task_iter_advance() doesn't consider ->usage
count, so add a new loop to css_task_iter_next() to skip de-used tasks
on the dying_list.
Rough illustration of the possible race
R (reader of cgroup.procs) T (thread) L (group leader)
--------------------------------- -------------------------------- --------------------------------
L exits, signal->live > 0
cgroup_task_dead(L)
css_set_skip_task_iters() // skips only cset->tasks
list_add_tail(&L->cg_list, &cset->dying_tasks)
css_task_iter_next()
take css_set_lock
css_task_iter_advance()
leader && signal->live != 0
=> it->task_pos = &L->cg_list
release css_set_lock
T exits
--signal->live == 0
cgroup_task_dead(T) // css_set_lock
release_task(T)
cgroup_task_release(T)
release_task(L) // zap_leader
cgroup_task_release(L)
put_task_struct_rcu_user(L)
...RCU...
put_task_struct(L)
L->usage = 0
/* L still on dying_tasks */
...RCU...
__put_task_struct(L)
css_task_iter_next() // another iteration
take css_set_lock
it->task_pos = &L->cg_list
get_task_struct(L)
=> addition on 0
drop css_set_lock
cgroup_task_free(L)
css_set_skip_task_iters() // dying skip comes too late
free_task(L)
cgroup_procs_show()
task_pid_vnr(L)
Fixes: 260fbcb92bbea ("cgroup: Move dying_tasks cleanup from cgroup_task_release() to cgroup_task_free()")
Cc: stable@vger.kernel.org # v6.19+
Link: https://lists.debian.org/debian-kernel/2026/08/msg00220.html
Reported-by: Noah Elias Feldt <N.Feldt@mittwald.de>
Reported-by: Salvatore Bonaccorso <carnil@debian.org>
Tested-by: Salvatore Bonaccorso <carnil@debian.org>
Signed-off-by: Michal Koutný <mkoutny@suse.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
After the previous cleanup there are two CONFIG_SYSCTL blocks. Move
the second block up into the first one and eliminate the redundant
Also add a proc_do_static_key() stub to the #else block. Not strictly
necessary today, but consistent with the other stubs declared in
include/linux/sysctl.h.
Signed-off-by: Oleg Nesterov <oleg@redhat.com>
Signed-off-by: Joel Granados <joel.granados@kernel.org>
|
|
After the removal of CONFIG_PROC_SYSCTL, several #ifdef CONFIG_SYSCTL
blocks ended up nested inside the outer (now the same) CONFIG_SYSCTL
guards.
Collapse them and remove a now-stale "/proc/sys support" comment which
referred to CONFIG_PROC_SYSCTL.
Signed-off-by: Oleg Nesterov <oleg@redhat.com>
Reviewed-by: Bradley Morgan <include@grrlz.net>
Signed-off-by: Joel Granados <joel.granados@kernel.org>
|
|
Several map batch operation implementations such as
generic_map_lookup_batch() use calculations in the form of
"values + cp * map->value_size" to compute the desired userspace memory
address for reading or writing. This can overflow the u32 type
(the result of "cp * map->value_size") when the map size exceeds 4GB.
generic_map_lookup_batch() may corrupt values for some keys in
userspace memory, and in some cases it mismatches values for some keys
while still reporting success.
Other batch operations may fail to delete or update some keys,
or the syscall may return unexpected errors.
Add size_t casts to prevent the affected offset and size calculations
from overflowing.
Fixes: cb4d03ab499d ("bpf: Add generic support for lookup batch op")
Fixes: aa2e93b8e58e ("bpf: Add generic support for update and delete batch ops")
Fixes: 057996380a42 ("bpf: Add batch ops to all htab bpf map")
Signed-off-by: Masoud Aghasi <maghasi@disroot.org>
Link: https://lore.kernel.org/r/20260903082734.623904-1-maghasi@disroot.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
|
|
Syzkaller repeatedly triggered UAF splats related to nodes in
waiting_for_gp_ttrace within the bpf memalloc:
BUG: KASAN: slab-use-after-free in llist_del_first+0x85/0x110 lib/llist.c:61
Read of size 8 at addr ffff8881572cd080 by task syz.4.470/5112
...
llist_del_first+0x85/0x110 lib/llist.c:61
alloc_bulk+0x193/0x460 kernel/bpf/memalloc.c:229
bpf_mem_refill+0x386/0x560 kernel/bpf/memalloc.c:436
Freed by task 14:
...
__free_rcu kernel/bpf/memalloc.c:281 [inline]
__free_rcu_tasks_trace+0x48/0xd0 kernel/bpf/memalloc.c:291
rcu_tasks_invoke_cbs+0x1ec/0x3e0 kernel/rcu/tasks.h:571
rcu_tasks_one_gp+0x13d/0x220 kernel/rcu/tasks.h:621
rcu_tasks_kthread+0xf3/0x120 kernel/rcu/tasks.h:651
The reason is that the UAF occurs after the RCU Tasks Trace GP expires:
when the __free_rcu() callback runs, there is no synchronization
protecting llist_del_all() against concurrent alloc_bulk() operating on
waiting_for_gp_ttrace, leading to the race condition below:
CPU0 CPU1
__free_rcu (RCU Tasks Trace callback)
alloc_bulk
llist_del_first(&c->waiting_for_gp_ttrace)
entry = smp_load_acquire(&head->first);
do {
if (entry == NULL)
return NULL;
free_all(llist_del_all(&c->waiting_for_gp_ttrace))
llist_for_each_safe(pos, t, llnode)
free_one(pos);
next = READ_ONCE(entry->next); <-- trigger UAF
} while (!try_cmpxchg(&head->first, &entry, next));
In addition, there is also a theoretical race condition on the
free_by_rcu_ttrace list. This race requires two preconditions: an
in-flight Tasks Trace GP keeping c->call_rcu_ttrace_in_progress == 1,
and concurrent cross-CPU frees repopulating c->free_by_rcu_ttrace with
new nodes. Under these conditions, the following scenario triggers UAF:
// CPU0
// irq work is still busy (on PREEMPT_RT)
alloc_bulk()
llist_del_first(&c->free_by_rcu_ttrace)
entry = smp_load_acquire(&head->first);
do {
if (entry == NULL)
return NULL;
// CPU1
bpf_mem_alloc_destroy()
WRITE_ONCE(c->draining, true)
// wait for CPU0
irq_work_sync()
// CPU2
do_call_rcu_ttrace(tgt(CPU0))
if (c->draining) {
llist_del_all(&c->free_by_rcu_ttrace)
free_all()
}
// CPU0 continue
next = READ_ONCE(entry->next); <-- trigger UAF
while (!try_cmpxchg(&head->first, &entry, next));
Fix this by introducing a raw spinlock to synchronize the concurrent
consumption on waiting_for_gp_ttrace and free_by_rcu_ttrace.
Fixes: 04fabf00b4d3 ("bpf: Allow reuse from waiting_for_gp_ttrace list.")
Suggested-by: Alexei Starovoitov <ast@kernel.org>
Suggested-by: Hou Tao <houtao1@huawei.com>
Signed-off-by: Pu Lehui <pulehui@huawei.com>
Acked-by: Hou Tao <houtao1@huawei.com>
Link: https://lore.kernel.org/r/20260905021139.4116529-1-pulehui@huaweicloud.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
Pull tracing fixes from Steven Rostedt:
- Don't destroy user event fields when removal fails
User event fields are destroyed before the event is removed from
visibility. But that can fail leaving the still visible event with no
fields. Move the destroying of the fields to after the event is
successfully removed from visibility.
- Initialize function graph state is fork before calling
copy_exec_state()
For non-CLONE_VM forks, copy_exec_state() allocates a new
task_exec_state. If that allocation fails, ftrace_graph_exit_task()
will free the tasks ret_stack pointer. Since that pointer is still
using the parent's ret_stack, it mistakenly frees the parent's
pointer too.
Call ftrace_graph_init() on the task first which will NULL out the
new tasks's ret_stack and if the copy fails, it will not free
anything.
- Remove FGRAPH_MAX_INDEX
The macro FGRAPH_MAX_INDEX was added but never used. Remove it.
- Save ent_size in function graph printing of nested functions
The function graph tracer needs to look at the next event to see if
the next event is the return of the current function entry. If it is,
it prints a single line:
ktime_get();
Otherwise it prints it like a nested function:
tick_nohz_irq_exit() {
ktime_get();
kcpustat_irq_exit();
}
In order to look at the next event, it must save the current event so
that it has the information to print from it. It saves the event in
the iterator descriptor called "ent". What it doesn't save is the
ent_size of the event which is now used to know if the function graph
arguments are to be printed. The peek doesn't save the size so the
size used happens to be that of the size of the last event that was
seen.
Save the entry event size in the iterator descriptor so that the
correct size is used.
- Fix several errors with freeing data in the histogram code
The histogram code had a lot of leaked or or incorrect accounting
when failures happen. Correct them.
- Fix histogram regression of .percent and .graph modifiers
Up until 6.3 histogram values could have "percent" or "graph"
modifiers that changed how they were printed. But a change that added
restricting histograms values from being strings, stack traces and
other modifiers inadvertently prevented them from using the percent
and graph modifiers, which were legal use cases for values.
Put back the percent and graph modifiers.
- Fix various typos in the comments
- Set the trace_clock before initializing a histogram with clock
argument
The histogram API allows the user to specific which trace clock to
use via a "clock=" string. The histogram is set up first before the
clock is checked. If the passed in clock is not valid, it exits
without fully fixing up the histogram leaving it on the list and a
use-after-free can trigger.
Update the clock argument first and if it fails then exit gracefully
before the histogram trigger is placed on any lists.
- Restore :mod: trailer after parsing in ftrace_set_clr_event
The function ftrace_set_clr_event() modifies the parse string and
needs to put it back to what was passed in. It searches for ":mod:"
via a strsep() but fails to put back the first ':' in the string.
Add back the ':' in the passed in string.
- Take trace_array reference when opening a tracer options file
The options files are dynamically created and some tracers add their
own options. When a tracer adds their own list of options, the
trace_array holding them has an array to hold the list of options for
each tracer. This array increases in size via a krealloc(), and the
new entry gets a newly allocated array to hold the options of the new
tracer being added.
The element in each entry of the tracer's option array holds a
pointer back to the trace_array, a pointer to the tracer it is
associated to, a pointer to the flags of the option.
The issue is that these arrays are freed when the trace_array is
freed when its instance it represents is removed from the instances
directory. There's a race that an open of one of these options files
can happen when the instance is being removed.
Add a new helper function to be called by the open function of the
options file to iterate all existing trace_arrays under a lock and
find the one that has the given option element in one of it's tracer
arrays. If found, then update the associated trace_array's reference
counter to keep it from being freed. If not found, have the open call
return -ENODEV.
- Disable interrupts when acquiring the lock in rb_wake_up_waiters()
The function rb_wake_up_waiters() assumes it will be called in
interrupt context and does not disable irqs when taking
cpu_buffer->reader_lock, which can be called in hard interrupt
context. The issue is in PREEMPT_RT, this function is called in
thread context leaving this lock open to a deadlock.
Take the lock with interrupts disabled.
- Use rcu_assign_pointer() for tmp_ops filter hash
The tmp_ops used in update_ftrace_direct_mod() assigns its
filter_hash field directly, but that field is annotated as __rcu and
sparse complains. Assign it with rcu_assign_pointer()
- Fix use-after-free in enable_trigger_private_data_free()
The trace_event_call is accessed through the event_trigger_data's
trace_event_file pointer to put the trace_event_call on freeing. The
issue is that the trace_event_file data may have been freed already
causing a use-after-free. Add a field to the event_trigger_data that
points directly to the trace_event_call so that it can decrement its
reference directly without needing to go through the
trace_event_file.
- Fix accounting of buffer data remote headers
trace_buffer_desc_size() and trace_remote_alloc_buffer() undercount
the number of pages is needed for the asked for size as it doesn't
take into account the meta data on each page. Add a helper function
to do the calculation properly and use that in these functions.
- Catch nr_page_va overflow in ring_buffer_desc sizing
The number of pages per remote ring buffer is capped by
ring_buffer_desc::nr_page_va (32 bits). A buffer_size large enough to
overflow that field would silently allocate a descriptor smaller than
what was asked for.
- Do not resize the subbuf order if any per_cpu buffer is disabled
The mmapping of ring buffers disables resizing the subbuffers, but it
is done per-cpu whereas the subbuf size change is done for all the
per_cpu buffers under the buffer->mutex. It could change the size of
some while the mapping is happening on others. Have the resize of the
subbuf order check all the per_cpu buffers under the lock to see if
any of them is disabled before starting and causing an inconsistency
between buffers that are being mapped.
* tag 'trace-v7.3-rc2' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace: (25 commits)
ring-buffer: Check resize_disabled before publishing the new subbuf order
tracing/remotes: Catch nr_page_va overflow in ring_buffer_desc sizing
tracing/remotes: Account for ring buffer page header in size calculation
tracing: Don't dereference trace_event_file in deferred trigger free
ftrace: Use rcu_assign_pointer() for tmp_ops filter hash
ring-buffer: Acquire the lock with irqsave in rb_wake_up_waiters()
tracing: Take trace_array reference when opening a tracer options file
tracing: Fix ring_buffer_read_page_size() kernel-doc
tracing: Restore :mod: trailer after parsing in ftrace_set_clr_event()
tracing: Fix memory corruption from a "STACKTRACE" histogram key
tracing: Fix memory corruption from the histogram stacktrace modifier
tracing: Undo the registration when enabling the histogram trigger fails
tracing: Take the reference before publishing the named histogram trigger
tracing: Set the trace clock before registering the histogram trigger
tracing: Fix typo "preceeded" in comment
tracing: Fix typo "availabe" in comment
tracing: Let histogram values keep the percent and graph modifiers
tracing: Keep the entry count when the histogram stats allocation fails
tracing: Free histogram the field rejected for a bad modifier
tracing: Free histogram the var ref when its initialization fails
...
|
|
ring_buffer_subbuf_order_set() stores the new order and only then walks
the CPUs, returning -EBUSY if any of them has resizing disabled. A user
mapped buffer has resizing disabled, and __rb_map_vma() reads
buffer->subbuf_order without buffer->mutex, so an mmap of an already
mapped CPU racing the failing order change sizes the mapping with the
new order and inserts pages past the sub-buffer into the VMA.
Check the CPUs before storing the new order.
Cc: stable@vger.kernel.org
Fixes: 117c39200d9d ("ring-buffer: Introducing ring-buffer mapping functions")
Link: https://patch.msgid.link/20260912103938.1127021-1-devnexen@gmail.com
Signed-off-by: David Carlier <devnexen@gmail.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
The number of pages per remote ring buffer is capped by
ring_buffer_desc::nr_page_va (32 bits). A buffer_size large enough to
overflow that field would silently allocate a descriptor smaller than
what was asked for.
Return SIZE_MAX from trace_buffer_desc_size() on nr_page_va overflow.
Link: https://patch.msgid.link/20260911193937.602202-3-vdonnefort@google.com
Fixes: 2e67fabd8b77 ("ring-buffer: Introduce ring-buffer remotes")
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
trace_buffer_desc_size() and trace_remote_alloc_buffer() undercount the
required pages because every ring buffer page contains a header
(BUF_PAGE_HDR_SIZE). Account for that header to ensure allocated remote
ring buffers aren't smaller than requested by the user.
The newly introduced helper __calc_nr_pages_ring_buffer_desc() can
return a value that overflows the descriptor nr_pages field (32 bits).
Link: https://patch.msgid.link/20260911193937.602202-2-vdonnefort@google.com
Fixes: 2e67fabd8b77 ("ring-buffer: Introduce ring-buffer remotes")
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull timer fixes from Ingo Molnar:
- Fix clockevents replacement race when a broadcast
device is replaced which may trigger a BUG() crash
(朱恺乾 - Zhu Kaiqian)
- Fix potential timerqueue ordering bug when rearming
a queued timer with nonzero slack (Andrea Parri)
* tag 'timers-urgent-2026-09-13' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
hrtimer: Use hard expiry when updating timers on the same base
tick/broadcast: Plug clockevents replacement race
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull scheduler fixes from Ingo Molnar:
- Fix EEVDF se->max_slice value on enqueueing (Vincent Guittot)
- Fix EEVDF augmented rb-trees re-balancing with multiple
fields (Vincent Guittot)
- In proxy scheduling, account cgroup CPU time to the execution
context, not the scheduling context (Hui Su)
- Likewise, call wq_worker_tick() for the execution context,
not the scheduling context (Hui Su)
* tag 'sched-urgent-2026-09-13' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
sched/core: Call wq_worker_tick() for the execution context
sched: Account cgroup CPU time to the execution context
sched/eevdf: Fix rb augmented with multi fields
sched/eevdf: Fix augmented max_slice
|
|
scx_alloc_and_add_sched() publishes ops->priv before
scx_root_enable_workfn() switches the state to SCX_ENABLING. An error
claimed via scx_bpf_error_bstr() from an associated BPF program in that
window is consumed by scx_disable_workfn(), which takes the pre-enable
shortcut in scx_root_disable(). The shortcut returns without any teardown
and restores SCX_DISABLED with an unconditional scx_set_enable_state() xchg
racing the enable workfn's own transition. The enable then completes with
the claim consumed: the scheduler stays up but can never be disabled again,
and bpf_scx_unreg() frees it while still in use, resulting in a
use-after-free. Both WARN_ON_ONCE()s fire back to back:
WARNING: kernel/sched/ext/ext.c:7522 at
scx_root_enable_workfn+0xeec/0x1be0, CPU#3: scx_enable_help/276
WARNING: kernel/sched/ext/ext.c:6398 at scx_root_disable+0xb50/0xdb8,
CPU#0: sched_ext_helpe/664
scx_root_enable_workfn() switches to SCX_ENABLING before the scheduler
allocation, so ops->priv is never visible while SCX_DISABLED. The allocation
failure path restores SCX_DISABLED.
Fixes: 105dcd005be2 ("sched_ext: Introduce scx_prog_sched()")
Cc: stable@vger.kernel.org
Signed-off-by: fangqiurong <fangqiurong@kylinos.cn>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull perf events fixes from Ingo Molnar
- Fix sched_cb_list corruption on PMU callbacks that
invoke list_del() during perf_event_overflow()
calls (Thomas Richter)
- Fix PEBS pt_regs->flags snapshot data that
regressed with the introduction of adaptive
PEBS v4 support (Dapeng Mi)
- Fix possible drain_pebs() re-entry bug when
intel_pmu_drain_pebs_buffer() is called from
process context (Dapeng Mi)
* tag 'perf-urgent-2026-09-13' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
perf/x86/intel: Prevent drain_pebs() reentry
perf/x86/intel: Correct pt_regs->flags update for PEBS path
perf/core: Allow list_del during perf_event_overflow()
|
|
The kernel-doc comment of dmem_cgroup_try_charge() refers to
dmem_cgroup_evict_valuable, which does not exist. The name is
dmem_cgroup_state_evict_valuable. Say dmem_cgroup_state_evict_valuable.
Fixes: b168ed458dde ("kernel/cgroup: Add "dmem" memory accounting cgroup")
Assisted-by: LLM
Signed-off-by: Karl Mehltretter <kmehltretter@gmail.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
The previous patch refuses a kfunc argument of more than one eightbyte.
This patch allows up to 16 byte kfunc arguments.
But different architectures have different ways to map the BPF calling
convention (no gap, no backfill) to the native one. Rather than have each
arch open-code where it wants an argument, describe the convention with a
register count and four booleans, and let each arch set what applies to
it:
struct bpf_jit_arg_abi {
u8 nr_arg_regs;
bool even_reg_align;
bool even_stack_align;
bool split_at_boundary;
bool backfill_after_stack;
};
The four booleans are meant to cover x86-64, arm64, RISC-V LP64 and
PowerPC64 ELFv2, although only x86-64 and arm64 fill the struct in here.
bpf_jit_place_args() and bpf_jit_plan_arg_moves() use that description to
work out where each argument belongs and which slots the JIT then has to
move, to be used in the JIT later on.
Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
Link: https://lore.kernel.org/r/20260912195242.989416-1-yonghong.song@linux.dev
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
|
