| Age | Commit message (Collapse) | Author |
|---|
|
Patch series "Introduce arch_do_panic", v6.
Replace architecture-specific ifdef sections in vpanic() with a clean
arch_do_panic() hook. Currently s390 and sparc embed their panic handlers
directly in vpanic() using preprocessor conditionals, making the common
code path harder to maintain.
Introduce arch_do_panic() as an architecture extension point called at the
end of vpanic(). Architectures can use this hook to implement their
specific panic handling without polluting the generic panic code.
Remove s390s ifdef block in vpanic() and move the corresponding code block
to s390s own arch_do_panic() implementation in architecture specific code.
Move sparc panic handling from ifdef blocks to arch_do_panic(). Remove
the preprocessor conditionals from vpanic() and place the Stop-A
enablement code in architecture-specific files where it belongs. Stop-A
enablement markers are now printed after "end Kernel panic" line. To me,
there are no better alternatives other than setup.c to put sparc's
arch_do_panic() implementation. The other files under arch/sparc/kernel
are either divided to *_32.c and *_64.c variants, which mean code
duplication, or unrelated.
The cleanup reduces vpanic() complexity and establishes a pattern for
other architectures needing custom panic behavior. No functional changes,
only minor print order changes.
This patch (of 3):
Introduce a hook for architectures to put their specific panic handlers.
s390 and sparc already have ifdef preprocessor checks to execute
architecture specific code. Pave the way for vpanic() cleanup.
Link: https://lore.kernel.org/20260831-arch_do_panic-v6-1-a1e170a9e7fd@linux.ibm.com
Link: https://lore.kernel.org/all/20260730-arch_do_panic-v3-0-d5401e683cdb@linux.ibm.com/ [1]
Signed-off-by: Mete Durlu <meted@linux.ibm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: Bradley Morgan <include@grrlz.net>
Suggested-by: Sven Schnelle <svens@linux.ibm.com>
Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Andreas Larsson <andreas@gaisler.com>
Cc: Christian Borntraeger <borntraeger@linux.ibm.com>
Cc: David S. Miller <davem@davemloft.net>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Petr Mladek <pmladek@suse.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
|
|
node->name is a flexible array member sized to exactly strlen(name) + 1
bytes at allocation time in new_node(), so this copy can never actually
overflow. Still, prefer the bounded strscpy() over strcpy() on general
principle; pass the same strlen(name) + 1 bound the allocation used, since
sizeof() cannot be applied to a flexible array member.
No functional change.
Link: https://lore.kernel.org/20260831144953.324441-1-hrushirajg23@gmail.com
Signed-off-by: Hrushiraj Gandhi <hrushirajg23@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: Bradley Morgan <brads@mainlining.org>
Cc: Peter Oberparleiter <oberpar@linux.ibm.com>
|
|
Once the warning budget is exhausted, hung_task_info() normally stops
printing per-task details. When panic is triggered, full details are
still printed so diagnostics remain available before panic.
To retain visibility without restoring per-task output after budget
exhaustion, emit a single aggregate summary line at the end of each
watchdog scan that detects hung tasks with an exhausted budget. This
keeps non-panic per-task reports budgeted during system-wide hangs.
Link: https://lore.kernel.org/20261004163610.413506-3-atomlin@atomlin.com
Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Suggested-by: Petr Mladek <pmladek@suse.com>
Suggested-by: Lance Yang <lance.yang@linux.dev>
Reviewed-by: Petr Mladek <pmladek@suse.com>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Reviewed-by: Bradley Morgan <brads@mainlining.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: David Laight <david.laight.linux@gmail.com>
Cc: Daniel Vacek <neelx@suse.com>
Cc: Sean Ashe <sean@ashe.io>
Cc: <chjohnst@gmail.com>
Cc: <steve@abita.co>
Cc: <mproche@gmail.com>
Cc: <nick.lange@gmail.com>
|
|
Patch series "hung_task: Improve warning budget handling and task
reporting", v11.
The hung_task watchdog detects tasks stuck in TASK_UNINTERRUPTIBLE (D)
state for longer than CONFIG_DEFAULT_HUNG_TASK_TIMEOUT seconds. To
prevent log spam during system spikes, sysctl_hung_task_warnings enforces
a budget on the number of logged warnings.
However, the current implementation has two major limitations:
1. Permanent exhaustion of warning budget
sysctl_hung_task_warnings is decremented directly when printing
warnings. Once this budget hits zero, no further warnings are
reported until an administrator manually updates the sysctl value or
reboots the system. Consequently, a single temporary hang episode
permanently blinds the kernel watchdog to any subsequent hung tasks
after system recovery.
2. Total log suppression when budget is exhausted
Once the warning budget reaches zero, hung_task_info() completely
suppresses all output, including the basic single-line alert. While
suppressing verbose stack dumps and lock debugging is desirable to
prevent dmesg flooding, hiding basic task alerts leaves
administrators entirely unaware that tasks are hanging.
This patch series resolves both limitations by decoupling the configured
warning limit from the active runtime budget, automatically resetting the
budget upon system recovery or sysctl updates, and emitting a single
aggregate summary line when hung tasks are detected under an exhausted
warning budget.
Patch 1 separates the configured sysctl hung_task_warnings from the
runtime budget, making khungtaskd the sole owner of runtime budget
updates. The budget is reloaded directly when a scan finds zero hung
tasks, or via an atomic reset request published on sysctl write.
Patch 2 prevents dmesg flooding during system-wide hangs by keeping
non-panic per-task stack dumps budgeted, while providing ongoing
visibility by logging a single aggregate summary line at the end of each
scan iteration when the warning budget is exhausted.
This patch (of 2):
The sysctl hung_task_warnings currently holds both the configured warning
limit and the remaining budget. Each detailed report decrements the
sysctl, so once it reaches zero, the configured limit is lost and cannot
be restored automatically.
Keep sysctl_hung_task_warnings as the configured warning limit and make
khungtaskd the sole owner of the remaining budget. A check that finds no
hung tasks reloads the budget directly from the configured limit. A
successful sysctl write publishes an atomic reset request, which
khungtaskd consumes at the start of the next check.
Link: https://lore.kernel.org/20261004163610.413506-1-atomlin@atomlin.com
Link: https://lore.kernel.org/20261004163610.413506-2-atomlin@atomlin.com
Signed-off-by: Aaron Tomlin <atomlin@atomlin.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Suggested-by: Petr Mladek <pmladek@suse.com>
Suggested-by: Lance Yang <lance.yang@linux.dev>
Tested-by: Lance Yang <lance.yang@linux.dev>
Reviewed-by: Lance Yang <lance.yang@linux.dev>
Reviewed-by: Bradley Morgan <brads@mainlining.org>
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: David Laight <david.laight.linux@gmail.com>
Cc: Daniel Vacek <neelx@suse.com>
Cc: Sean Ashe <sean@ashe.io>
Cc: <chjohnst@gmail.com>
Cc: <steve@abita.co>
Cc: <mproche@gmail.com>
Cc: <nick.lange@gmail.com>
|
|
A task waiting for a muxed region can sleep forever in
TASK_UNINTERRUPTIBLE even though the region it waits for is already free.
__request_region_locked() queues itself on muxed_resource_wait and drops
resource_lock before setting TASK_UNINTERRUPTIBLE, while
__release_region() wakes the queue after dropping the same lock. A wakeup
landing in between finds TASK_RUNNING, does not match TASK_NORMAL and is
discarded; callers hold a muxed region only across a bounded transaction,
so no further release is coming. The task is unkillable and its caller
never returns.
The window is one store wide, but an interrupt is enough to hold the
waiter in it, and the machine this was seen on runs PREEMPT_DYNAMIC in its
voluntary default. Since v6.11 spd5118 exports the DDR5 sensors of AMD
boards through i2c-piix4, which takes a muxed region per SMBus
transaction; a third of the in-tree users of request_muxed_region() are
hwmon drivers, so reading a world-readable attribute is all an
unprivileged user needs to drive the contention. The blocked task sleeps
holding the i2c adapter bus lock, and 27 more piled up behind it.
Reproduced by building a kernel with the two orderings selectable at
runtime and a 2ms delay inside the window. Switching only that knob, a
two-thread barriered reproducer loses the wakeup 200 times out of 200
before the fix and 0 out of 200 after it; without the delay it goes 20000
times through the wait path and loses none.
Fix it by setting the task state before dropping resource_lock, as
prepare_to_wait() does: the releasing side needs resource_lock to unlink
the resource, so it cannot reach the wakeup before the state is published.
Link: https://lore.kernel.org/20260821-b4-resource-muxed-lost-wakeup-v1-1-37eb6473a76c@gmail.com
Fixes: 8b6d043b7ee2 ("resource: shared I/O region support")
Signed-off-by: DAI RENJIE <drj19981414013@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: Bradley Morgan <brads@mainlining.org>
Reviewed-by: Andrew Morton <akpm@linux-foundation.org>
Assisted-by: Claude:claude-opus-5
Cc: Mark Brown <broonie@kernel.org>
Cc: Kees Cook <kees@kernel.org>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: <stable@vger.kernel.org>
|
|
Patch series "mm: Unconditional per-VMA locks and cleanups", v7.
tl;dr: Make per-VMA locks available in all configs. Simplify some of the
per-VMA lock users now that they can rely on them being always available.
Binder and networking folks: Your code is the target of the cleanups. I'm
cc'ing you now on v2 because there's emerging consensus on the mm side
that the approach here is sane. I'm not quite sure how this pile would
get merged, but ack/review tags would be appreciated if this looks good to
you.
Longer version:
When working on some x86 shadow stack code, it was a real pain to avoid
causing recursive locking problems with mmap_lock. One way to avoid those
was to avoid mmap_lock and use per-VMA locks instead. They are great, but
they are not available in all configs which makes them unusable in generic
code, or if you want to completely avoid mmap_lock.
Make per-VMA locks available in all configs. Right now, they are only
available on select architectures when SMP and MMU are enabled. But all
of the primitives that per-VMA locks are built on (RCU, maple trees,
refcounts) work just fine without SMP or MMU.
The only real downside is that making VMAs a wee bit bigger on !MMU and
!SMP builds.
The upside is much cleaner code, lower complexity and less #ifdeffery.
Clean up a binder VMA locking site now that it can rely on per-VMA locks.
Building on top of universally-available per-VMA locks, introduce a new
helper. Since the new API does not require callers to have a fallback to
mmap_lock, it's much easier to use. Callers can potentially replace this
very common kernel idiom:
mmap_read_lock(mm);
vma = vma_lookup()
// fiddle with vma
mmap_read_unlock(mm);
with:
vma = vma_start_read_unlocked(mm, address);
// fiddle with vma
vma_end_read(vma);
Which avoids mmap_lock entirely in the fast path.
Use that new API for another binder site and one in the TCP code.
This patch (of 7):
The per-VMA locks have been around for several years. They've had some
bugs worked out of them and have seen quite wide use. However, they are
still only available when architectures explicitly enable them. Remove
the conditional compilation around the per-VMA locks, making them
available on all architectures and configs.
The approach up to now seemed to be to add ARCH_SUPPORTS_PER_VMA_LOCK when
the architecture started using per-VMA locks in the fault handler. But,
contrary to the naming, the Kconfig option does not really indicate
whether the architecture supports per-VMA locks or not. It is more of a
marker for whether the architecture is likely to benefit from per-VMA
locks.
To me, the most important thing side-effect of universal availability is
letting per-VMA locks be used in SMP=n configs. This lets us use
per-VMA locking in all x86 code without fallbacks.
Overall, this just generally makes the kernel simpler. Just look at the
diffstat. It also opens the door to users that want to use the per-VMA
locks in common code. Doing *that* brings additional simplifications.
The downside of this is adding some fields to vm_area_struct and
mm_struct. There are likely ways to optimize this, especially for things
like SMP=n configs. For now, do the simplest thing: use the same
implementation everywhere.
== Considerations for NOMMU config ==
NOMMU systems do not write-lock VMAs, therefore read-locking a VMA would
always succeed unless VMA is detached. Therefore for NOMMU config we make
vma_mark_attached() a NOOP, which keeps VMAs always in detached state.
This causes VMA read-locking to always fail and the caller falls back to
locking mmap_lock.
The following functions will have a different implementation in NOMMU
config:
- vma_mark_attached(), vma_mark_detached() are made NOOPs, keeping VMAs
always in a detached state and preventing assertions and refcount
underflows;
- vma_start_write(), vma_start_write_killable() are made NOOPs to avoid
warnings in __vma_start_write() due to VMAs being detached. These
functions are not used in NOMMU code but __vma_start_write() is an
exported function, therefore might be used by drivers.
- vma_assert_attached() is made NOOP because it's reachable from NOMMU
code via split_vma()->vma_iter_store_new()->vma_iter_store_overwrite();
- vma_assert_write_locked() is asserting vma->vm_mm is write-locked, as
was done before this change;
- vma_assert_locked() is asserting vma->vm_mm is locked, as was done
before this change;
The following functions work for both MMU and NOMMU configs:
- vma_lock_init() performs the same initialization as for MMU config;
- mm_lock_seqcount_init(), mm_lock_seqcount_begin(),
mm_lock_seqcount_end() are called from mmap_write_{lock|unlock} and
update mm_lock_seq correctly.
- mmap_lock_speculate_try_begin(), mmap_lock_speculate_retry() work as
is because mm_lock_seq is updated correctly;
- vma_start_read(), vma_start_read_locked() will always fail because
VMAs are always detached;
- vma_end_read() will never be called because vma_start_read() never
succeeds;
- vma_is_attached() always return false because VMAs are always
detached;
- vma_assert_detached() will never trigger because VMAs are never
attached;
- vma_start_read_locked() always return false because VMAs are always
detached;
- lock_vma_under_rcu() will be safe as the attempted read lock will bail;
Changes in the following files are not affecting NOMMU config:
task_mmu.c - not compiled when CONFIG_MMU=n;
pagewalk.c - not compiled when CONFIG_MMU=n;
userfaultfd.c - not compiled when CONFIG_MMU=n (CONFIG_USERFAULTFD depends
on CONFIG_MMU);
The following changes in the BPF code are made to keep NOMMU config
working like before:
stack_map_lock_vma() - keeps mmap_lock in NOMMU config;
bpf_iter_task_vma_new() - bails out in NOMMU config;
Link: https://lore.kernel.org/20260831203056.838265-1-surenb@google.com
Link: https://lore.kernel.org/20260831203056.838265-2-surenb@google.com
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: Suren Baghdasaryan <surenb@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Todd Kjos <tkjos@android.com>
Cc: Christian Brauner <christian@brauner.io>
Cc: Carlos Llamas <cmllamas@google.com>
Cc: Alice Ryhl <aliceryhl@google.com>
Cc: David S. Miller <davem@davemloft.net>
Cc: David Ahern <dsahern@kernel.org>
Cc: Arve Hjønnevåg <arve@android.com>
|
|
Patch series "arch, mm/execmem: resolve confusion about
set_direct_map_valid_noflush()", v3.
Recent discussion about implementation of execmem's ROX caches on arm64
revealed a confusion about how set_direct_map_valid_noflush() implemented
on different architectures.
On arm64 it sets or clears the PTE_VALID bit marking a PTE as present or
not present.
On other architectures it's a range version of
set_direct_map_invalid_noflush() and set_direct_map_default_noflush()
Unlike arm64::set_direct_map_valid_noflush(),
set_direct_map_default_noflush() not only marks PTE as present, but also
sets its default protection mode.
Other than that, initial design of execmem ROX caches didn't rely on
restoration of large mappings that's now available on x86, but completely
removed the memory allocated for the ROX cache from the direct map to
ensure that large mappings are not split. This precluded usage of
VM_FLUSH_RESET_PERMS for the ROX cache allocations and required execmem to
implement manipulation of the direct map alias.
Current implementation of ROX caches does not remove the direct map alias
but simply calls set_memory_rox() that updates the permissions in both
vmalloc address space and the direct map and relies on
collapse_large_pages() in x86 CPA to keep large mappings.
This allow using VM_FLUSH_RESET_PERMS for execmem ROX cache allocations
with small adjustments to set_direct_map APIs and vmalloc::reset_perms()
behaviour: adding number of pages parameter to set_direct_map APIs and
making resetting of the direct map permissions in vmalloc VMAP_HUGE
friendly.
Implement these adjustments, make execmem always use VM_FLUSH_RESET_PERMS
and revert set_direct_map_valid_noflush() changes.
This patch (of 6):
When set_direct_map APIs were introduced by the commit d253ca0c3865
("x86/mm/cpa: Add set_direct_map_*() functions") the single page parameter
made sense because the initial callers (vmalloc and hibernation) had sets
of unsorted struct pages that required changes of their mappings in the
direct map.
Since there is an increasing demand for direct map manipulation and it is
also desirable to be able to update larger physically contiguous mappings,
for example an entire large folio, extend set_direct_map APIs to receive
number of pages parameter.
As there is still only a handful of callers, change the existing functions
directly and update all the call sites rather than adding wrappers for
single page case.
Link: https://lore.kernel.org/20260903-execmem-set-vm-perms-v0-2-v3-0-949b64a9f755@kernel.org
Link: https://lore.kernel.org/20260903-execmem-set-vm-perms-v0-2-v3-1-949b64a9f755@kernel.org
Link: https://lore.kernel.org/all/20260611130144.1385343-4-abarnas@google.com [1]
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Kevin Brodsky <kevin.brodsky@arm.com>
Cc: Albert Ou <aou@eecs.berkeley.edu>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Alexandre Ghiti <alex@ghiti.fr>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: "Borislav Petkov (AMD)" <bp@alien8.de>
Cc: Brendan Jackman <brendan.jackman@linux.dev>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Christian Borntraeger <borntraeger@linux.ibm.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Gerald Schaefer <gerald.schaefer@linux.ibm.com>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: Huacai Chen <chenhuacai@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Len Brown <lenb@kernel.org>
Cc: Palmer Dabbelt <palmer@dabbelt.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: "Rafael J. Wysocki" <rafael@kernel.org>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Sven Schnelle <svens@linux.ibm.com>
Cc: "Uladzislau Rezki (Sony)" <urezki@gmail.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: WANG Xuerui <kernel@xen0n.name>
Cc: Will Deacon <will@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
|
|
A parent sched that delegates cpus to sub-scheds has no way of telling how
much each delegatee is using them. A parent that sizes its delegations by
demand cannot measure it, and one that wants to hand a cid from one sub to
another cannot see which sub it would take the cid from.
Add ops.sub_cid_sched_updated(), invoked on a sched when what it sees
running on a cid changes: NONE when no task of its subtree runs there, SELF
for its own task, or a direct child's cgroup id. Changes inside a child's
subtree are not reported.
The op costs one pointer compare per context switch when the sched does not
change, a notification reaches only the scheds that see a change, and the
whole mechanism sits behind a static key that is on only while a loaded
sched implements the op. After a sched's tasks are gone, its disable closes
every rq session still pointing at it, so nothing refers to it once it is
freed.
The op is cid-form only like the other sub ops and runs with the rq lock
held on the context switch path.
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq
Pull workqueue fixes from Tejun Heo:
- Fix a NULL dereference in the chained work check when a kworker
queues work on a draining or destroying workqueue outside work item
execution.
* tag 'wq-for-7.3-rc6-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq:
workqueue: Fix NULL current_pwq deref in chained work check
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup
Pull cgroup fixes from Tejun Heo:
- During CPU offline, the active mask drops the CPU before cpuset
updates the effective CPUs, so a task placement in that window could
find no active CPU in the top cpuset and dereference NULL. Restore
the NULL check.
- The cpuset v2-mode test read the subsystem's root pointer, which is
stale during a cgroup filesystem rebind, and the hotplug handler
evaluated it before taking the cpuset mutex. Record the mode in a
flag and test it under the mutex.
* tag 'cgroup-for-7.3-rc6-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup:
cgroup/cpuset: Call is_in_v2_mode() after acquiring cpuset_mutex in cpuset_handle_hotplug()
cgroup/cpuset: Handle cpu hotplug race in guarantee_active_cpus()
cgroup/cpuset: Don't access cpuset_cgrp_subsys.root in is_in_v2_mode()
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext
Pull sched_ext fixes from Tejun Heo:
- Taking a CPU offline could hang, or stall until the watchdog ejected
the BPF scheduler, when tasks on the dying CPU were still held by the
scheduler or sitting on a user dispatch queue. Re-enqueue them onto
the local queue when the runqueue goes offline so that the CPU pushes
them off like the other sched classes.
- The sequence number guarding against stale dispatches was per
runqueue, so a task re-enqueued on another CPU could get the same
number and a dispatch meant for its earlier instance was applied to
the new one. Use a per-task counter.
- A task dispatched to another CPU's local queue got its ops.dequeue()
only when picked to run and flagged as a core-sched pick. Call it at
insertion like for same-CPU dispatches.
* tag 'sched_ext-for-7.3-rc6-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext:
sched_ext: Generate qseq from a per-task counter
selftests/sched_ext: Add a test for ops.dequeue() on remote local DSQ moves
sched_ext: Call ops.dequeue() when a task arrives on a remote local DSQ
sched_ext: Fix CPU hotplug hang when a dying CPU's tasks sit in the BPF scheduler
|
|
A sub-scheduler can leave standing state behind on the cids delegated to it,
cpuperf targets being the obvious case, and the kernel does not track who
wrote it. Restoring it is the parent's job and ops.sub_detach() is the place
for it. Today the parent can't rely on that: the dying sub-scheduler keeps
every cap until it is marked dead after ops.exit(), so its ops.exit() or a
late timer can write after the parent has cleaned up.
Clear every cap of the sub-scheduler before ops.sub_detach(). The pshard
caps go first as a pending sync recomputes ecaps from them, and ecaps are
then zeroed directly. A grant racing the clear is refused under the child's
pshard lock, so nothing hands the caps back.
Fixes: 86094b95efcf ("sched_ext: Add per-shard cap delegation for sub-schedulers")
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
New workloads can do large bursts of call_rcu_tasks() invocations in a
short time period, followed by a quiet time period long enough to drain
all of the callbacks, followed by another burst of call_rcu_tasks()
invocations. This can cause RCU Tasks to switch back and forth between
queuing callbacks only on CPU 0 (during quiet periods) and on all CPUs
(during bursts).
Which is fine. Except for the fact that each cycle from CPU-0-only to
all-CPUs queuing and back generates three console messages, one announcing
the shift to all-CPUs queuing, another announcing the start of the shift
back to CPU-0-only queuing, and the third announcing completion of this
shift after an RCU grace period. And these console messages can overrun
console-log communications channels and obscure other console-message-based
debugging information. And the only known use for these console messages
is debugging RCU Tasks itself.
This commit therefore adds a rcupdate.rcu_task_collapse_debug module
parameter that defaults to false (suppressing these console messages).
Those debugging or otherwise playing with RCU Tasks callback queuing
auto-adjustment can set this parameter to the value true.
[ paulmck: Apply Breno Leitao feedback. ]
Reported-by: Breno Leitao <leitao@debian.org>
Reported-by: David Dai <david.dai@linux.dev>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Reviewed-by: Breno Leitao <leitao@debian.org>
|
|
The bcachefs ktest allocation-leak check writes rcutree.do_rcu_barrier
before reading /proc/allocinfo. While testing bcachefs performance
changes, small objects released with kfree_rcu() remained visible after
repeated writes to the hook and 20 seconds of waiting, causing otherwise
clean tests to fail their leak check.
The test assumes a stronger contract than the hook currently documents:
rcu_barrier() waits for ordinary callbacks, but does not flush objects
still held in kfree_rcu() batching or per-CPU SLUB sheaves. The retained
population eventually fell as a sheaf filled; there is no evidence here
of unbounded growth or OOM.
Changing the hook to drain kvfree_rcu() work let the same unmodified
bcachefs workload pass its allocation check. All eight checkpoints in
one VM, after 50 through 400 option changes, reported zero retained
reconcile_scan objects. This motivated the separate private-cache test
used to isolate the incomplete drain from bcachefs.
Calling kvfree_rcu_barrier() from rcu_barrier_throttled() was proposed
when kvfree_rcu_barrier() was added in 2024, to restore a clean baseline
between userspace benchmark runs. The discussion concluded that keeping
the existing hook name, adding the second operation and documenting both
was the safest compatibility choice, but the follow-up was not added.
Add that drain and document the stronger test interface. Always retain
the existing start-rate limit and perform the kvfree_rcu() drain: an
unrelated ordinary barrier does not establish that this work completed.
Retain the entry ordinary-barrier sequence snapshot. After draining,
skip the final ordinary barrier only if that snapshot is complete,
preserving the memory barrier on the completion path. Otherwise, invoke
rcu_barrier() explicitly. This keeps the ordinary-callback guarantee
independent of whether kvfree_rcu_barrier() embeds an ordinary barrier.
Clarify that the documented completion guarantee covers work queued
before the request, without preventing new work from being queued.
Earlier validation of the unconditional-drain version used four fresh
VM pairs with a private-cache fixture: controls retained the queued
object (60 to 60 active objects), and treatments drained it (60 to 59).
An ordinary-callback test passed on both kernels. Those runs predated
the guarded skip and do not validate that change. No elapsed-time
improvement is claimed.
Link: https://lore.kernel.org/all/20240820155935.1167988-1-urezki@gmail.com/
Signed-off-by: Matthias Goergens <matthias.goergens@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
Nothing in the tree.c translation unit uses anything from the
private timekeeping header ../time/tick-internal.h, included
since commit 48d07c04b4cc ("rcu: Enable elimination of
Tree-RCU softirq processing"). The tick symbols RCU uses,
tick_dep_set(), tick_dep_clear(), their _cpu and _task
variants, tick_nohz_full_cpu() and TICK_DEP_BIT_RCU, are all
declared in the public linux/tick.h, which tree.c already
includes.
Drop the include.
Signed-off-by: Bradley Morgan <brads@mainlining.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
Correct "upto" to "up to", reported by scripts/checkpatch.pl using the
misspelling list in scripts/spelling.txt. Only touches comments, no code
changes.
Assisted-by: Cursor:claude-opus-5
Signed-off-by: Hemanth Selam <hemanth.selam@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
Currently, rcu_print_task_stall() will dump out the PID, RCU reader
nesting level, the rcu_special structure's flags, and whether or not
that reader is on the ->blkd_tasks list. When debugging RCU priority
boosting, it is also good to know whether the stalled RCU reader is
currently running and whether it is currently being RCU priority boosted.
This commit therefore adds this information to the output.
[ paulmck: Apply kernel test robot feedback. ]
[ paulmck: Apply feedback from Arnd Bergmann and kernel test robot. ]
Link: https://lore.kernel.org/all/20260925133922.1356404-1-arnd@kernel.org/
Link: https://lore.kernel.org/all/202609261005.GrnaXBXT-lkp@intel.com/
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Reviewed-by: Bradley Morgan <brads@mainlining.org>
Reviewed-by: Kunwu Chan <kunwu.chan@gmail.com>
Tested-by: Kunwu Chan <kunwu.chan@gmail.com>
|
|
cpuset_handle_hotplug()
It is reported by sashiko that calling is_in_v2_mode() outside of
cpuset_mutex critical section in cpuset_handle_hotplug() can introduce
a TOCTOU race where cgroup hierarchy may have changed from v1 to v2 or
vice versa after is_in_v2_mode() is called leading to erroneously skip
the allocation of tmpmasks or incorrectly modify cpus_allowed masks,
resulting in cpuset state corruption. Fix that by calling is_in_v2_mode()
after acquiring the cpuset_mutex.
Fixes: b8d1b8ee93df ("cpuset: Allow v2 behavior in v1 cgroup")
Signed-off-by: Waiman Long <longman@redhat.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/printk/linux
Pull printk fix from Petr Mladek:
- Allow using Braille console with a serial console driver converted
to NBCON API
* tag 'printk-for-7.3-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/printk/linux:
braille: nbcon: Allow to use a serial console with NBCON API as Braille console
|
|
Describe SECCOMP_FILTER_FLAG_RESTART_BEFORE_RECV in the UAPI header, as
SECCOMP_FILTER_FLAG_WAIT_KILLABLE_RECV is described above it, and say in
the documentation that supervisors must tolerate ENOENT from
SECCOMP_IOCTL_NOTIF_RECV, which the flag makes routine. Note at the
fget_raw() call that O_PATH files are allowed on purpose.
Build tested ARCH=x86_64 defconfig (kernel/seccomp.o and headers) with
GCC 16.2.0, and SPHINXDIRS=userspace-api htmldocs.
Assisted-by: LLM
Signed-off-by: Kees Cook <kees@kernel.org>
|
|
SECCOMP_IOCTL_NOTIF_ADDFD uses fget() to acquire the supervisor's
source descriptor. fget() rejects FMODE_PATH files, so injecting an
O_PATH descriptor fails with EBADF before the receiving task can
install it.
Use fget_raw() to acquire the source file. This takes the same
reference while allowing FMODE_PATH. The existing receive_fd() and
receive_fd_replace() paths retain their security_file_receive()
checks and preserve the file's O_PATH semantics.
Assisted-by: Codex:gpt-6
Signed-off-by: Cong Wang <cwang@multikernel.io>
Link: https://patch.msgid.link/20260924223018.564548-2-xiyou.wangcong@gmail.com
Signed-off-by: Kees Cook <kees@kernel.org>
|
|
An interrupted user notification wait returns ERESTARTSYS before the
syscall body has run. If the notifying task's handler for the delivered
signal was installed without SA_RESTART, that task sees EINTR even for
calls whose callers do not expect it. This can make fork fail
unexpectedly or make close report EINTR while leaving the descriptor
open. A leaked pipe write end can prevent readers from seeing EOF.
The supervisor cannot repair this result once the interrupted task
removes the notification. If removal happens before receipt, the
supervisor never receives that request; a reply using its ID would fail
with ENOENT. Receiving notifications eagerly only narrows the scheduling
window. WAIT_KILLABLE_RECV protects supervisor processing after receipt,
but deliberately leaves the pre-receive wait interruptible.
A sandbox also cannot transparently fix this in the target. It cannot
require arbitrary workloads to retry calls such as fork and close.
Retrying close on EINTR is unsafe when the native syscall has already
released the descriptor. Forcing SA_RESTART on application handlers
would change cancellation behavior for other blocking calls.
Sandlock encounters this while mediating process creation to enforce
process limits. For example, dash installs its SIGCHLD handler without
SA_RESTART. While dash is creating a pipeline, an earlier child can exit
and generate SIGCHLD while dash waits for a fork notification to be
received. The interrupted wait then makes fork return EINTR, causing
dash to report "Cannot fork". Removing fork from notification mediation
would bypass the process-limit enforcement that sandlock needs.
Add SECCOMP_FILTER_FLAG_RESTART_BEFORE_RECV, requiring NEW_LISTENER.
Under notify_lock, convert an interrupted wait's ERESTARTSYS to
ERESTARTNOINTR only while the notification remains INIT. The notifying
task's signal handler still runs; if it returns normally, syscall entry
and the filter are evaluated again.
Keep this opt-in because commit c2aa2dfef243 ("seccomp: Add
wait_killable semantic to seccomp user notifier") deliberately preserved
pre-receipt interruption so workloads could abandon requests before the
supervisor starts processing them. The flag can be combined with
WAIT_KILLABLE_RECV to defer non-fatal signals after receipt. Neither
supervisor-supplied errors nor the native syscall's restart behavior is
changed.
Assisted-by: Codex:gpt-6
Signed-off-by: Cong Wang <cwang@multikernel.io>
Link: https://patch.msgid.link/20260924204209.477694-2-xiyou.wangcong@gmail.com
Signed-off-by: Kees Cook <kees@kernel.org>
|
|
A data slice is valid only while the specific dynptr that produced it
remains valid. prepare_func_exit() copies a subprogram return value into
the caller and then frees the callee state without applying the normal
destruction semantics to dynptrs in the callee stack.
A callee can therefore derive a slice from a local dynptr, return or spill
the slice into its caller, and then let the dynptr disappear with the
callee frame. The slice retains the id of a dynptr that is no longer in the
verifier state, so the verifier continues to accept accesses through it.
Before the object relationship refactor, bpf_dynptr_data() slices from
referenced dynptrs also carried the shared reference id, allowing a later
release to catch some of these escaped slices. The refactor made those
slices precise children of their source dynptr and exposed the missing
teardown as an accepted stale access even after the shared resource is
released.
Add destroy_dynptrs_in_stack_slots() to apply the existing dynptr teardown
to an inclusive range of stack slots. Reuse it for dynptr initialization,
variable-offset stack writes, and the entire callee stack before freeing
the frame. The teardown rejects losing the last dynptr for a referenced
resource and invalidates the dynptr and all its descendants, including
slices held in the caller. Propagate errors through prepare_func_exit().
Reuse the existing teardown API and diagnostic. Update the affected
callback test expectation in this change so the commit remains test-clean.
Fixes: 308c7a0ae885 ("bpf: Refactor object relationship tracking and fix dynptr UAF bug")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Suggested-by: Amery Hung <ameryhung@gmail.com>
Signed-off-by: Xu Yunxiang <xyx2021@mail.ustc.edu.cn>
Reviewed-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Link: https://lore.kernel.org/r/CAMB2axOmdHwSrHihSRskDywkCmEGQOX+6dZPN_A9u-HhxY3UDA@mail.gmail.com
Link: https://lore.kernel.org/r/CAMB2axMmrTb+UZ84UD48MwMtXbk1s9bWtreU3AOfjzU0fPT0BA@mail.gmail.com
Link: https://lore.kernel.org/bpf/20260927120422.1042107-2-xyx2021@mail.ustc.edu.cn
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
do_exit() closes the task's files in exit_files() and hangs up the
controlling tty of a session leader in disassociate_ctty(1) after
that. Since commit d99d38540bf0 ("fs: make close_files() synchronous")
the final __fput() of every file runs inside exit_files(). So when the
session leader holds the last open of its tty the tty is released
before disassociate_ctty() runs. tty_release() clears signal->tty for
the whole session in session_clear_tty() once the count drops to zero
and sends no signal doing so. disassociate_ctty(1) then finds neither a
tty nor a tty_old_pgrp and does nothing. The foreground process group
loses its SIGHUP:
do_exit()
exit_files()
close_files()
tty_release() tty->count == 0
session_clear_tty() signal->tty = NULL, no signal
disassociate_ctty(1)
get_current_tty() NULL
signal->tty_old_pgrp NULL, nothing sent
That only affects real ttys. For a pty the master's open keeps the
slave's count above zero. And it only affects a foreground job that
holds no descriptor to the tty anymore while its session leader exits.
Everything else is unchanged. The DTR drop on the last close happens in
tty_port_shutdown() regardless, stopped jobs get their SIGHUP from
kill_orphaned_pgrp() and signal->tty is cleared either way.
Before that commit the final __fput() ran from exit_task_work() which
comes after disassociate_ctty(). That order isn't old. Until v3.14
exit_task_work() came right after exit_files() and before v3.6 fput()
was synchronous, so the tty was always released first. Commit
c39df5fa37b0 ("exit: call disassociate_ctty() before
exit_task_namespaces()") moved disassociate_ctty() up to fix a pppd
crash and in front of exit_task_work() as a side effect. The hangup in
this case has worked since then and that's eleven years of userspace
being able to rely on it.
Hang the tty up before closing the files. This is the ordinary hangup
with the file still open: __tty_hangup() swaps in hung_up_tty_fops and
tty_release() runs from the close afterwards as it does when a modem
drops the line. disassociate_ctty() stays in front of
exit_task_namespaces() which the pppd fix needs.
Reported-by: Chris Mason <mason@kernel.org>
Link: https://patch.msgid.link/20260921-work-coredump-fixes-v3-8-8e4adb1619e6@kernel.org
Acked-by: Oleg Nesterov <oleg@redhat.com>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Reorder copy_process() cleanup so sched_fork() taking scx_fork_rwsem
scx_pre_fork() is safe and isn't held around exiting files. Right now, a
fork that fails while another thread closed the bpf link fd will
deadlock against its own read side. It also blocks every fork on the
system:
copy_process() holds scx_fork_rwsem for read
exit_files()
close_files()
filp_close_sync()
bpf_scx_unreg()
kthread_flush_work() waits for the disable work
scx_root_disable()
percpu_down_write(&scx_fork_rwsem)
Simply release the child's files after sched_cancel_fork() dropped the
lock. The task starts out as a copy of its parent so p->files points to
the parent's table until copy_files() replaces it. exit_files() must not
run for a fork that failed before that. Clear p->files up front so
exit_files() is a no-op for those and make copy_files() set it
explicitly for CLONE_FILES.
Reported-by: Chris Mason <mason@kernel.org>
Link: https://patch.msgid.link/20260921-work-coredump-fixes-v3-7-8e4adb1619e6@kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
coredump_wait() sets core_state->nr_threads to the number of tasks
killed and waits for the last thread to enter coredump_task_exit() to
signal completion. Let's just wait on the count directly. The exiting
tasks can use atomic_dec_and_wake_up() and the dumping task sleeps in
wait_var_event_state().
The dumping task must remain freezable since commit f5d39b020809
("freezer,sched: Rewrite core freezer logic"). So keep the wait
TASK_UNINTERRUPTIBLE|TASK_FREEZABLE.
Drop the completion and rename nr_threads to threads_remaining.
No functional changes.
Suggested-by: NeilBrown <neilb@ownmail.net>
Link: https://lore.kernel.org/178899497961.207413.10554121774377911612@noble.neil.brown.name
Link: https://patch.msgid.link/20260910-work-coredump-unlock-self-v4-12-a5c1800dc930@kernel.org
Reviewed-by: NeilBrown <neil@brown.name>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
The core_state->dumper field isn't used anymore. Only its ->next pointer
is. The current task is always the dumping thread and the ->task pointer
is never read. Replace it with a plain pointer to the list of parked
threads.
Historically, core_state->dumper was used. Its ->task pointer was read.
by fill_note_info() started at &core_state->dumper to ensure that the
dumping thread came first in the ELF thread notes. That changed in
commit 4b0e21d64253 ("[elf][regset] simplify thread list handling in
fill_note_info()"). The first iteration was taken out of the loop. So
it's been unused ever since.
No functional changes.
Suggested-by: NeilBrown <neilb@ownmail.net>
Link: https://lore.kernel.org/178900159210.207413.8292125177519817528@noble.neil.brown.name
Link: https://patch.msgid.link/20260910-work-coredump-unlock-self-v4-10-a5c1800dc930@kernel.org
Reviewed-by: NeilBrown <neil@brown.name>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
exec is the only caller left since commit 433967cab51e ("coredump: stop
unsharing the file descriptor table"). All it does is call unshare_fd()
with CLONE_FILES and install the copy. Kill the pointless helper and
open-code it.
No functional changes.
Link: https://patch.msgid.link/20260910-work-coredump-unlock-self-v4-4-a5c1800dc930@kernel.org
Reviewed-by: NeilBrown <neil@brown.name>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Move unshare_fd() where the rest of the descriptor table lifecycle
helpers live.
No functional changes.
Link: https://patch.msgid.link/20260910-work-coredump-unlock-self-v4-3-a5c1800dc930@kernel.org
Reviewed-by: NeilBrown <neil@brown.name>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Add switch_files_struct() to install another table on a task. It
consumes the reference to the new table and puts the old one. Convert
every place that switches a descriptor table except unshare_files().
No functional changes.
Link: https://patch.msgid.link/20260910-work-coredump-unlock-self-v4-2-a5c1800dc930@kernel.org
Reviewed-by: NeilBrown <neil@brown.name>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Since Arm SPE and CoreSight drivers can be built as modules, export
perf_event_itrace_started() so that it can be used by these modules.
Signed-off-by: Leo Yan <leo.yan@arm.com>
Acked-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: James Clark <james.clark@linaro.org>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull perf events fixes from Ingo Molnar:
- Fix race between perf_event_exit_task() and perf_pending_task()
(Luo Gengkun)
- Fix perf header output management regressions (Ian Rogers)
- Require kernel access for text poke events (Zhengchuan Liang)
* tag 'perf-urgent-2026-10-04' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
perf: Require kernel access for text poke events
perf: Replace perf_event_header__init_id with full header init
perf: Fix race between perf_event_exit_task() and perf_pending_task()
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull locking fixes from Ingo Molnar:
- Don't run refcount kunit self-test when !CONFIG_KUNIT_ALL_TESTS
(Kuan-Wei Chiu)
- Fix futex private hash use-after-free on resize (Chris Mason)
* tag 'locking-urgent-2026-10-04' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
futex: Fix private hash use-after-free on resize
irq: Make refcount_interrupt kunit test selectable
|
|
Modern mount tools (util-linux >= 2.39.1) use the new mount API
(fsopen, fsconfig, fsmount, move_mount) instead of the legacy mount(2)
syscall. The generic SYSCALL audit record logs the fsopen syscall but
does not capture the filesystem name string, creating an audit gap for
filesystem mount operations.
Add an FSOPEN auxiliary record that logs the dereferenced filesystem
name string passed to fsopen(2).
type=SYSCALL ... : arch=x86_64 syscall=fsopen ... a1=FSOPEN_CLOEXEC
type=FSOPEN ... : fs_name="tmpfs"
Link: https://github.com/linux-audit/audit-kernel/issues/152
Signed-off-by: Ricardo Robaina <rrobaina@redhat.com>
Reviewed-by: Richard Guy Briggs <rgb@redhat.com>
[PM: Christian Brauner commented "Seems fine"]
Signed-off-by: Paul Moore <paul@paul-moore.com>
|
|
finish_dispatch() uses the qseq embedded in p->scx.ops_state to tell
whether the QUEUED instance of a task it's about to claim is the one
scx_bpf_dsq_insert() saw. qseq is generated from rq->scx.ops_qseq, but
the counters of different rqs are independent, so if a task is dequeued
and re-enqueued on a different rq between scx_bpf_dsq_insert() and
finish_dispatch(), the new QUEUED instance can end up with the same
qseq as the old one:
CPU X CPU Z
----- -----
enqueue p on rq A, qseq = N
ops.dispatch()
scx_bpf_dsq_insert(p)
records qseq N
sched_setaffinity(p)
dequeue p from rq A
enqueue p on rq B, qseq = N
finish_dispatch(p, N)
qseq matches, p is claimed
The claim itself is still atomic so the core stays consistent, but an
insert issued for a previous QUEUED instance gets applied to a new one
which the BPF scheduler has just received through ops.enqueue(). This
breaks the guarantee that dispatches targeting a stale instance are
ignored.
Generate qseq from a per-task counter, p->scx.ops_qseq, instead so that
consecutive QUEUED instances of a task never share a qseq regardless of
which rq they're on. The counter is only updated in
scx_do_enqueue_task() with the task's rq locked, so no additional
synchronization is needed, and it fits in an existing hole in struct
sched_ext_entity on 64bit. Remove the now unused rq->scx.ops_qseq.
Never generate qseq 0. NONE and DISPATCHING don't carry a qseq, so
scx_bpf_dsq_insert() on a task in either state records 0. With a
per-task counter, every task's first QUEUED instance would otherwise get
qseq 0 and could be claimed by such an insert. Wrap the counter where
the QSEQ field wraps so that it can't reach a value that shifts to 0 on
32bit either.
Fixes: f0e1a0643a59 ("sched_ext: Implement BPF extensible scheduler class")
Cc: stable@vger.kernel.org # v6.12+
Assisted-by: Claude:claude-opus-5.5
Signed-off-by: Kuba Piecuch <jpiecuch@google.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
LLVM splits a static of a Rust program into pieces. Every piece is a VAR
with the type of the whole static:
[223] STRUCT 'BpfCell<core::option::Option<...>>' size=32 vlen=1
[236] VAR '..scx_cosmos9TASK_CTXS.0' type_id=223, linkage=static
[248] DATASEC '.bss' size=0 vlen=9
type_id=236 offset=24648 size=1 (VAR '..TASK_CTXS.0')
and the kernel rejects such BTF with "Invalid size".
Allow it. The size of a variable in DATASEC is used in two places:
- btf_find_datasec_var() looks for timers, spin locks, kptrs and other
special fields. btf_find_field_one() skips a variable when its size is
not the size of its type, so there are no special fields in a piece.
- btf_datasec_show() prints variables by their types. It would read
past the piece and past the end of the map value when the piece is the
last one. Don't print the pieces.
"global data test #8" and "#10" in prog_tests/btf.c expect "Invalid
size". BTF of both is loaded now. The map of #10 is not created, since
its value is larger than the section.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20261002124714.180012-10-alexei.starovoitov@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Some static functions in BTF of a Rust program have arguments without
names:
[71] FUNC_PROTO '(anon)' ret_type_id=0 vlen=1
'(anon)' type_id=72
[81] FUNC 'unwrap_failed' type_id=71 linkage=static
and the kernel rejects such BTF with "Invalid arg#1". It's 5 of 25
static functions in scx_cosmos and 6 of 24 in scx_simple.
The verifier looks at types of the arguments. The names are printed only,
as "(anon)" when there is none. Allow such static FUNC. Global functions
are checked as before.
The test "func (Some arg has no name)" in prog_tests/btf.c has a static
function. Make it global to keep the check.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Alan Maguire <alan.maguire@oracle.com>
Link: https://lore.kernel.org/bpf/20261002124714.180012-8-alexei.starovoitov@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Names of types and functions in BTF that LLVM makes for a Rust program
are not C identifiers:
[69] STRUCT 'NonNull<str>' size=16 vlen=1
[147] FUNC 'write_fmt<scx_cosmos::BpfStream>' type_id=146
and the kernel rejects such BTF with "Invalid name". scx_simple and
scx_cosmos schedulers written in Rust have them in STRUCT, FWD and FUNC,
made of letters, digits and " #&()*,:;<>[]{}", 430 characters at most.
Allow any printable character in btf_name_valid_identifier(), like it's
done for DATASEC. It checks names of types, functions, members,
enumerators, variables and arguments, so all of them can have such
characters now. The limit of KSYM_NAME_LEN stays.
The name of FUNC is a part of the name of the program in kallsyms, where
a space would break the parsers. Replace what is not a character of
an identifier with '_' there.
Tests in prog_tests/btf.c expect "Invalid name" for names with '!' and
'*', which are valid now. Put a character that is not printable there.
The type name '?foo' is expected to load.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20261002124714.180012-6-alexei.starovoitov@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
There are no address spaces in Rust. LLVM emits plain loads and stores
for arena memory, without cast_kern:
r3 = 0x20 ll // R_BPF_64_64 .bss, libbpf puts it into arena
r2 = *(u64 *)(r3 + 0) // R3 invalid mem access 'scalar'
lock *(u64 *)(r2 + 8) += r1
Add BPF_F_ARENA_SCALAR flag of BPF_PROG_LOAD. When the program is loaded
with it and has an arena treat ldx, stx, st and atomics through a number
as arena access. It's as safe as access through PTR_TO_ARENA: JIT adds
the base of arena to the low 32 bits of the address. The program has
an arena only with CAP_BPF and CAP_PERFMON.
Registers of the program are not changed, since Rust compares and
stores the address after the access. bpf_do_misc_fixups() copies
the low 32 bits into BPF_REG_AX and the access goes through it:
r2 = *(u64 *)(r3 + 0) -> w12 = w3
r2 = *(u64 *)(r12 + 0)
Constant blinding needs BPF_REG_AX for immediates and leaves insns that
use BPF_REG_AX alone. So the immediate of st through a number is not
blinded, the rest of the program is:
*(u64 *)(r3 + 0) = 1 -> w12 = w3
*(u64 *)(r12 + 0) = 1
The same insn may see PTR_TO_ARENA on another path. It works for both.
It's a flag, since a number is also what the verifier makes of a pointer
that went away when the program has CAP_PERFMON: a ringbuf record after
bpf_ringbuf_submit(), a pointer to the packet after bpf_skb_pull_data().
Programs in C that have an arena keep "invalid mem access 'scalar'" for
such bugs.
JIT tells with bpf_jit_supports_arena_scalar() that it takes BPF_REG_AX
as the address of arena access. With other JITs the flag is rejected
with -EOPNOTSUPP. x86 and arm64 do:
- x86: the fault handler finds the register that holds the address
through reg2pt_regs[]. Add BPF_REG_AX, r10 of x86, there.
- arm64: nothing else is needed. The JIT takes any register as the
address and the fault handler reads it by its number. Compile tested
only.
Any number is an address of arena, NULL and small numbers included, like
it is for PTR_TO_ARENA: arena code in C dereferences NULL and relies on
the fault being handled.
Still rejected:
- insn that sees a number on one path and a pointer that is not
PTR_TO_ARENA on another.
- numbers passed to helpers and kfuncs, except __arena arguments.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20261002124714.180012-4-alexei.starovoitov@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Rust's core::fmt keeps a flag in the low bit of a pointer:
r2 = *(u64 *)(r1 + 0)
r3 = r2
r3 &= 1
r2 >>= 1
The verifier rejects it:
r0 &= 8
R0 bitwise operator &= on pointer prohibited
Negation and byte swap of a pointer already produce a number when
allow_ptr_leaks is set. Do the same for bitwise ops, shifts, *=, /= and
%=, 64-bit and 32-bit. The result is an unknown number. With CAP_PERFMON
the program can store the pointer and load it back as a number already.
+= and -= are not changed: they keep the pointer, 32-bit += is rejected.
Pointers that allow no arithmetic and pointers that may be NULL are
still rejected.
Two tests in verifier_value_illegal_alu store through the result of
&= and /=. They are rejected at the store now. Update expected messages.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20261002124714.180012-2-alexei.starovoitov@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
A freplace attach claims the target prog by bumping
tgt_prog->aux->freplace_link_cnt and setting tr->extension_prog.
bpf_arch_text_poke() then makes the extension take effect. If the
poke fails, the claims are never released: the attach unwinds
through bpf_link_cleanup(), which clears link->prog, so
bpf_trampoline_unlink_prog() never runs.
Drop the link count under ext_mutex on the error path, and set
tr->extension_prog only after the poke succeeded. The count is
still bumped before the poke: it blocks prog_array updates while
the entry is patched.
Fixes: d6083f040d5d ("bpf: Prevent tailcall infinite loop caused by freplace")
Fixes: be8704ff07d2 ("bpf: Introduce dynamic program extensions")
Suggested-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Yuan Chen <chenyuan@kylinos.cn>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Link: https://patch.msgid.link/20260928091121.219333-1-chenyuan_fl@163.com
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
Pull probes fixes from Masami Hiramatsu:
- fprobe: Use guard(rcu_sched_notrace) and check rcu_is_watching()
Exit handlers early if !rcu_is_watching() to prevent potential
use-after-free during unregistration in idle/quiescent states. Switch
to guard(rcu_sched_notrace) to avoid fast-path lockdep overhead and
recursion while ensuring safe grace period synchronization.
- kprobes: Skip disarmed probes when checking optkprobe overlap
Continue past disarmed or unprepared probes in get_optimized_kprobe()
to find active optimized probes. This avoids overwriting active jump
displacements which can lead to panic.
* tag 'probes-fixes-v7.3-rc5' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace:
kprobes: Skip disarmed probes when checking optkprobe overlap
fprobe: Use guard(rcu_sched_notrace) and check rcu_is_watching()
|
|
Currently, kprobe_optimizer() holds kprobe_mutex, text_mutex, and
cpus_read_lock() simultaneously while executing synchronize_rcu_tasks().
Under PREEMPT_LAZY and server workloads with long-running CPU-bound kernel
tasks or heavy cgroup writeback loops, synchronize_rcu_tasks() can block
for seconds to minutes. Because text_mutex and cpus_read_lock() are held
during this entire wait, any concurrent static key updates, module
loading/unloading, CPU hotplug, or tracing updates stall, frequently
triggering hung-task detector panics.
Instead of penalizing the kernel preemption path with invasive hooks and
global hash lookups, decouple kprobe jump optimization from synchronous
waiting entirely:
1. Replace synchronize_rcu_tasks() with call_rcu_tasks(). Locks
(text_mutex and cpus_read_lock()) are held only for the brief moment
needed to patch instructions via arch_unoptimize_kprobes() and
arch_optimize_kprobes() (microseconds), and are completely released
while waiting for the Tasks RCU grace period.
2. Introduce a fixed ring of generations (optprobe_gens[OPTPROBE_GEN_MAX])
to avoid any dynamic memory allocation (kmalloc) or -ENOMEM failure
modes.
3. The last generation slot in the array is reserved as a "waiting room"
and is not dispatched to RCU until another in-flight generation has
finished. While a generation is waiting for its Tasks RCU grace period,
any newly registered or unregistered probes accumulate in the waiting
room generation without blocking or requiring additional slots.
4. When the Tasks RCU callback fires, it marks the generation as ready
and wakes up the optimizer thread to finalize optimization (poking
the jump instructions) and free cleaned probe slots. Once finalized,
the generation is marked idle, allowing the waiting room generation
to be dispatched next.
5. Flushing via wait_for_kprobe_optimizer() waits asynchronously for all
in-flight and queued generations to drain without stalling other
kernel subsystems.
On non-preemptive kernels or configs where CONFIG_TASKS_RCU=n,
call_rcu_tasks() transparently aliases to call_rcu(), preserving full
portability.
Link: https://lore.kernel.org/all/179091652953.526694.3402334249215311330.stgit@devnote2/
Assisted-by: LLM
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
In preparation for converting the kmalloc family of allocators to the
type-aware kmalloc_obj family, we need to make sure that the returned
type from the allocation matches the type of the variable being
assigned. (The kmalloc family returns "void *", which can be implicitly
cast to any pointer type.)
The assigned type is "const char **", but the converted allocation type
would be "char **", which is the same type without the const qualifier.
As there is no general way to safely add const qualifiers, take the size
from the assignment target instead. No change in allocation size
results.
Build tested ARCH=x86_64 allmodconfig with GCC 16.2.0:
kernel/trace/trace_probe.o
Link: https://lore.kernel.org/all/20260917211022.i.677-kees@kernel.org/
Assisted-by: LLM coccinelle
Signed-off-by: Kees Cook <kees+treewide@kernel.org>
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
This enabled knob is a disgusting terrible hack. It has rolled its
own read/write pair since 2007, writing '1' or '0' into a three byte
buffer by hand just to print a single character, with an XXX comment
begging debugfs for write callbacks on bool files.
DEFINE_DEBUGFS_ATTRIBUTE showed up in 2016 and does exactly that, so
the disgusting terrible hack has outlived its excuse for nine years.
Kill it, and the stale comment with it.
The behavior does not change, except the write only accepts 0/1 now
instead of y/n/on/off, and nothing uses anything else.
Link: https://lore.kernel.org/all/20260818015318.24103-1-include@grrlz.net/
Signed-off-by: Bradley Morgan <include@grrlz.net>
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
Cross-merge BPF and other fixes after downstream PR.
Conflicts:
arch/arm64/net/bpf_jit_comp.c
arch/x86/net/bpf_jit_comp.c
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
|
|
Pull bpf fixes from Alexei Starovoitov:
- Fix overflow of backward jump offset in constant blinding
(Alexei Starovoitov)
- Fix packet range of packet pointers sharing an id when var_off
tightens umax of one pointer and not the other (Alexei Starovoitov)
- Fix objects stuck in free_by_rcu_ttrace list of bpf memalloc
(Alexei Starovoitov)
- Fix use-after-free of progs detached from busy trampolines: wait for
an RCU tasks grace period before freeing trampoline progs, and patch
detached progs out of trampoline images that are still in use
(Florent Revest)
- Hold map BTF for the memory allocator destructor record to fix UAF in
deferred bpf_mem_alloc destruction (Kumar Kartikeya Dwivedi)
- Fix missing migration protection in resizable hashtab
lookup_and_delete batch operation (Ömer Mete Kaya)
* tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf:
bpf: Fix missing migration protection in __rhtab_map_lookup_and_delete_batch()
selftests/bpf: Add a test for objects stuck in free_by_rcu_ttrace
bpf: Fix objects stuck in free_by_rcu_ttrace
bpf: Factor out __do_call_rcu_ttrace()
selftests/bpf: Test packet range of pointers sharing an id
bpf: Fix packet range of pointers sharing an id
selftests/bpf: Detach a trampoline prog while a task sleeps before it
bpf: Skip detached progs in trampoline images that are still in use
bpf: Wait for an RCU tasks grace period before freeing trampoline progs
bpf: Hold map BTF for the memory allocator destructor record
bpf: Fix overflow of jump offset in constant blinding
|
|
With commit 2125c0034c5d ("cgroup/cpuset: Make cpuset hotplug processing
synchronous"), the cpuset hotplug operation becomes synchronous. That
commit also removes the code that handles race between cpuset_hotplug_work
and cpu hotplug notifier with the assumption that race is now gone.
Later commit 7a0aabd9ce69 ("cgroup/cpuset: Always use cpu_active_mask")
updates the cpuset code to always use cpu_active_mask instead of
cpu_online_mask in some places including guarantee_online_cpus() which
is renamed to guarantee_active_cpus() in that commit.
In the case of CPU offline operation, cpu_active_mask is updated
first in sched_cpu_deactivate() to remove the offline CPU before
cpuset_handle_hotplug() is called to update the effective_cpus of the
affected cpusets. The cpu_online_mask is updated after that near the end
of the offline operation to remove the offline CPU. As a result, the race
comes back and the top cpuset may not have any active CPU leading to NULL
pointer dereference during the race window when guarantee_active_cpus()
is called after cpu_active_mask is updated to remove the CPU to be torn
down but before cpuset_handle_hotplug() is able to properly update the
effective_cpus of the top cpuset.
Fix this by adding back the NULL cs check to avoid this problem.
Fixes: 7a0aabd9ce69 ("cgroup/cpuset: Always use cpu_active_mask")
Cc: stable@vger.kernel.org # v6.16+
Reported-by: Farhad Alemi <farhad.alemi@berkeley.edu>
Link: https://lore.kernel.org/lkml/CA+0ovChh3VjsKN1g+ZGjwwY2fGTpP7uD+aCCByLj5Qbymw=bfQ@mail.gmail.com
Signed-off-by: Waiman Long <longman@redhat.com>
Tested-by: Farhad Alemi <farhad.alemi@berkeley.edu>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
Add a hook to allow arch code disable hibernation. On arm64, with an RMM
active, there could be pages that are delegated to the "Realm" security
state. Any access to them are fatal (e.g., hibernate save image or
restoring an image to a delegated page). pKVM host also has its
restrictions on enabling hibernation.
Add a weak arch hook to let the architectures have a say.
Cc: "Rafael J. Wysocki" <rafael@kernel.org>
Cc: Len Brown <lenb@kernel.org>
Cc: Pavel Machek <pavel@kernel.org>
Cc: linux-pm@vger.kernel.org
Cc: Sudeep Holla <sudeep.holla@kernel.org>
Suggested-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Reviewed-by: Sudeep Holla <sudeep.holla@kernel.org>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
|
|
This fixes a regression when a console option includes ':'. Commit
7640f1a44eba ("printk: Add match_devname_and_update_preferred_console()")
introduced console=DEVNAME:0.0 hardware style addressing by looking for a
colon. If the colon is part of an option the name is handled as devname
instead of ttyname.
Fix by handling the options first which will add a NULL terminator to the
string. Note that parsing the options first means that also idx parsing
needs changing. Handle the idx parsing by refactoring the code (thanks to
Petr Mladek).
Signed-off-by: David Engraf <david.engraf@sysgo.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Petr Mladek <pmladek@suse.com>
Reviewed-by: John Ogness <john.ogness@linutronix.de>
Link: https://patch.msgid.link/20260924070153.250783-2-david.engraf@sysgo.com
[pmladek@suse.com: Updated the commit message as suggested by Tony Lindgren]
Signed-off-by: Petr Mladek <pmladek@suse.com>
|
