| Age | Commit message (Collapse) | Author |
|---|
|
Instead of migrating one task each time through find_proxy_task(),
we can walk up the blocked_donor ptrs and migrate the entire
current chain in one go.
This was broken out of earlier patches and held back while the
series was being stabilized, but I wanted to re-introduce it.
Signed-off-by: John Stultz <jstultz@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260807035232.1881495-8-jstultz@google.com
|
|
Pull the core of attach_tasks() out into sched.h so
it can be used more generically.
Suggested-by: K Prateek Nayak <kprateek.nayak@amd.com>
Signed-off-by: John Stultz <jstultz@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260807035232.1881495-7-jstultz@google.com
|
|
Similar to commit 04f80f8b12a0 ("sched: Switch rq->next_class on
proxy_resched_idle()"), we should also set the rq->next_class
when we call proxy_reset_donor().
Fixes: f13beb010e4a ("sched: Have try_to_wake_up() handle return-migration for PROXY_WAKING case")
Signed-off-by: John Stultz <jstultz@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260807035232.1881495-6-jstultz@google.com
|
|
Core scheduling chooses a core-wide cookie before __schedule()
installs the next task. With proxy-exec enabled, that task becomes the
donor/scheduling context, and find_proxy_task() may then replace the
execution context with the runnable mutex owner. If its cookie differs
from the selected core cookie, running it would bypass core scheduling's
cookie selection.
When the final mutex owner found by find_proxy_task() does not match the
selected core cookie, stop proxying the donor. If the current execution
context is already in the blocked chain, fall back to idle like the
existing proxy-exec retry paths do. Otherwise deactivate the donor and
let __schedule() pick again. The mutex owner can be picked later under
its own cookie.
Fixes: 7de9d4f94638 ("sched: Start blocked_on chain processing in find_proxy_task()")
[jstultz: Added tweak to ensure we deactivate donor, not runnable owner]
Reported-by: K Prateek Nayak <kprateek.nayak@amd.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: John Stultz <jstultz@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lore.kernel.org/lkml/10282ce9-f4ae-498f-9b57-f4e1e61fffbc@amd.com/
Link: https://patch.msgid.link/20260807035232.1881495-5-jstultz@google.com
|
|
Its not useful to migrate blocked_on tasks from other
runqueues as the proxy logic will just migrate it
back to the owner's rq. So skip blocked_on tasks here.
Signed-off-by: John Stultz <jstultz@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260807035232.1881495-4-jstultz@google.com
|
|
try_steal_cookie() avoids stealing src->core_pick and src->curr before
moving a task with the same cookie via move_queued_task_locked().
With proxy-exec, src->donor is the current scheduling context and may
differ from src->curr. Stealing it migrates a task that the source rq
still treats as current, leaving src's scheduler state for that task
stale. For CFS this means cfs_rq->curr points at the stolen entity,
and the next pick on the source rq hits the WARN_ON_ONCE in
put_prev_entity().
Commit 7de9d4f94638 ("sched: Start blocked_on chain processing in
find_proxy_task()") tweaked the fair class logic so that the donor task
isn't migrated away while we're running the proxy. Do it similarly for
try_steal_cookie() and skip src->donor as well.
Fixes: 7de9d4f94638 ("sched: Start blocked_on chain processing in find_proxy_task()")
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
Signed-off-by: John Stultz <jstultz@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260807035232.1881495-3-jstultz@google.com
|
|
syzbot reported a lockdep splat hitting DEBUG_LOCKS_WARN_ON(1) in
hlock_class() due to an invalid class_idx:
WARNING: kernel/locking/lockdep.c:238 at __lock_acquire+0x382/0x2cf0 kernel/locking/lockdep.c:5203
Workqueue: wg-crypt-wg0 wg_packet_tx_worker
RIP: 0010:hlock_class kernel/locking/lockdep.c:238 [inline]
RIP: 0010:check_wait_context kernel/locking/lockdep.c:4870 [inline]
RIP: 0010:__lock_acquire+0x389/0x2cf0 kernel/locking/lockdep.c:5203
Call Trace:
<IRQ>
lock_acquire+0x106/0x350 kernel/locking/lockdep.c:5886
_raw_spin_lock+0x2e/0x40 kernel/locking/spinlock.c:173
tcp_tsq_handler+0x29/0x200 net/ipv4/tcp_output.c:1291
tcp_tsq_workfn+0x384/0x410 net/ipv4/tcp_output.c:1325
...
When a lock class is zapped (e.g. during module unload or key
unregistration), zap_class() clears the class's bit in
lock_classes_in_use and removes it from the class hash table.
However, existing lockdep_map instances embedded in data structures
may still retain a pointer to the zapped class in their class_cache[]
array.
When __lock_acquire() subsequently runs on such a lock, it finds
lock->class_cache[subclass] != NULL, skipping register_lock_class()
and assigning hlock->class_idx to the index of the zapped class. When
check_wait_context() or hlock_class() inspects the held_lock, it finds
!test_bit(class_idx, lock_classes_in_use) and warns. Furthermore, if
the zapped slot is subsequently re-allocated to an unrelated lock key,
the stale class_cache entry would erroneously match the unrelated
class (ABA issue).
Add lock_class_cache_is_valid() to validate that the cached class is
within lock_classes bounds, still allocated in lock_classes_in_use
(using uninstrumented arch_test_bit() in __always_inline context so it
is safe in noinstr contexts like match_held_lock()), and that
class->key matches the expected subkey (taking lockdep_set_subclass()
overrides into account). Also use READ_ONCE()/WRITE_ONCE() when
accessing class_cache[]. If the entry is invalid or stale, fall back
to register_lock_class() / look_up_lock_class().
Fixes: a0b0fd53e1e6 ("locking/lockdep: Free lock classes that are no longer in use")
Closes: https://lore.kernel.org/netdev/6a8c66dc.4d75e56a.c9a88.0050.GAE@google.com/T/#u
Reported-by: syzbot+2d770620059281e225a4@syzkaller.appspotmail.com
Assisted-by: Gemini:gemini-3.1-pro
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260824155129.676096-1-edumazet@google.com
|
|
perf_mmap_close() drops rb->mmap_count *without* holding
event->mmap_mutex (the refcount_dec_and_test() right before the
refcount_dec_and_mutex_lock() of event->mmap_count). A concurrent
perf_mmap_rb() can slot its entire "revival" path into that window
(perf_mmap holds event->mmap_mutex for its whole duration, including
rb_alloc):
munmap side (perf_mmap_close) mmap side (perf_mmap_rb)
----------------------------------- --------------------------------
rb->mmap_count 1 -> 0 (no lock) (holds event->mmap_mutex)
inc_not_zero(rb->mmap_count) fails
ring_buffer_attach(event, NULL)
rb_alloc() + attach new rb
refcount_set(&event->mmap_count, 1)
lock; event->mmap_count 1 -> 0
ring_buffer_attach(event, NULL)
ring_buffer_put() -> frees the *new* rb
The revival's refcount_set(&event->mmap_count, 1) is an invisible
1 -> 1 write: the close frees the just-revived buffer although the
other process still has it mapped -- a page-level use-after-free
allowing local privilege escalation to root by any unprivileged user
(default kernel.perf_event_paranoid=2).
Swap the order of the two counter updates: event->mmap_count is
dropped first via refcount_dec_and_mutex_lock(), so its 1 -> 0
transition and the ring_buffer_attach() stay serialized with
perf_mmap(). rb->mmap_count == 0 then implies every event using the
buffer is detached already, so the result of the rb->mmap_count drop
can gate the remaining teardown directly and detach_rest is no longer
needed.
An earlier fix for this race from Kyle Zeng and David Lee takes
event->mmap_mutex around both counter updates [0]; here the not-last
close stays lockless.
Fixes: 59741451b49c ("perf: Identify the 0->1 transition for event::mmap_count")
Reported-by: Kimi Security Team <bug-report@moonshot.ai>
Suggested-by: Peter Zijlstra <peterz@infradead.org>
Co-developed-by: Weiming Shi <shiweiming@moonshot.ai>
Signed-off-by: Weiming Shi <shiweiming@moonshot.ai>
Signed-off-by: Yilin Zhang <yilinzhang@moonshot.ai>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lore.kernel.org/linux-perf-users/20260804060931.711308-1-david.lee@trailofbits.com/ [0]
Cc: <stable@vger.kernel.org>
Cc: stable@vger.kernel.org # 6.18+
Link: https://patch.msgid.link/20260831162155.1437652-1-yilinzhang@moonshot.ai
|
|
perf_aux_output_end() emits a PERF_RECORD_AUX when the recorded size is
nonzero or when any flag other than PERF_AUX_FLAG_OVERWRITE is set.
PMU format flags describe how an AUX payload is encoded. TRBE driver
sets PERF_AUX_FLAG_CORESIGHT_FORMAT_RAW for raw trace buffers, causing
an AUX record to be emitted even when no trace data.
This is noticeable when tracing a task with strace. Ptrace stops
repeatedly end empty AUX transactions, producing many zero-sized
PERF_RECORD_AUX records. For example:
perf record -e cs_etm//u -m,128M -- strace ls
perf script -D 2>&1 |
awk '/PERF_RECORD_AUX offset/ {
for (i = 1; i <= NF; i++)
if ($i == "size:" && $(i + 1) == "0")
count++
}
END { print count }'
165
This recording contains 165 zero-sized AUX records which provide no
useful information to userspace.
Ignore PERF_AUX_FLAG_PMU_FORMAT_TYPE_MASK, together with
PERF_AUX_FLAG_OVERWRITE, when deciding whether an empty AUX record is
useful. Zero-sized records carrying TRUNCATED, PARTIAL or COLLISION
are still emitted.
Fixes: 547b60988e63 ("perf: aux: Add flags for the buffer format")
Reported-by: Tamas Petz <tamas.petz@arm.com>
Signed-off-by: Leo Yan <leo.yan@arm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260825-perf_core_fix_zero_aux_records-v1-1-23b95e8d5df3@arm.com
|
|
Cache-aware load balancing biases tasks toward their preferred LLC. On
asymmetric CPU capacity systems (e.g. big.LITTLE) the destination LLC may
contain CPUs that are too small to run the task. Pulling the task there
turns it into a misfit, trading a cache-locality gain for a capacity loss
that's more detrimental to performance.
Guard both cache-aware migration entry points against this:
- can_migrate_llc_task(): forbid the LLC migration when the task fits its
source CPU but would not fit the destination CPU.
- alb_break_llc(): veto the active balance under the same condition so the
runnable task is not pushed onto a CPU that cannot accommodate it.
Both checks are gated with checks for hybrid processors, so symmetric
systems are unaffected. Tasks that already do not fit their source CPU
are left to the existing LLC policy, since the move cannot make their
fitness worse (this also preserves misfit up-migration to bigger CPUs).
Additionally, if there are misfit tasks found in the load balancing
classification phase, prioritize misfit task migrations
over LLC load aggregation on asymmetric systems. A better fitting
CPU will boost performance more than better cache locality.
Reviewed-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
Tested-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com>
Reviewed-by: Chen Yu <yu.c.chen@intel.com>
Signed-off-by: Tim Chen <tim.c.chen@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/edbb2503d554c63dc9b72e201fb4a17e1cb119e7.camel@linux.intel.com
|
|
distribute_cfs_runtime() refreshes the rq clock and accounts elapsed
runtime with update_curr() before redistributing bandwidth, but gates
this on cfs_rq->curr. Since commit 85570f10a4c6 ("sched/eevdf: Move to
a single runqueue") cfs_rq->curr is only maintained on the root
cfs_rq, so for the cgroup cfs_rqs it walks, the check never fires and
the refresh is dead code.
Use cfs_rq->h_curr, the per-level current entity, restoring the
intended behaviour: only refresh when something is actually running at
the throttled level, i.e. within the deferred throttle window.
Without this, runtime consumed by a still-running task of the
throttled hierarchy is not docked before redistribution;
unthrottle_cfs_rq() catches up unconditionally since
commit 28ad5427682b ("sched/fair: Call update_curr() before
unthrottling the hierarchy"), so this is not a correctness hole today,
but the refresh the check was written for is gone.
Fixes: 85570f10a4c6 ("sched/eevdf: Move to a single runqueue")
Signed-off-by: Wanwu Li <liwanwu@kylinos.cn>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Aaron Lu <ziqianlu@bytedance.com>
Tested-by: Aaron Lu <ziqianlu@bytedance.com>
Link: https://patch.msgid.link/20260831101141.391382-3-liwanwu@kylinos.cn
|
|
After commit 85570f10a4c6 ("sched/eevdf: Move to a single runqueue"),
cfs_rq->curr is only maintained on the root cfs_rq (set/cleared from
set_next_task_fair()/put_prev_task_fair()), while cfs_rq->h_curr is
the per-level current entity, set by set_next_entity() at every level
of the hierarchy. For an intermediate cfs_rq (a cgroup), cfs_rq->curr
is always NULL, but cfs_rq->h_curr is the group entity at that level.
throttle_cfs_rq() reads cfs_rq->curr to decide whether there is a
running entity at the throttled level, in which case it should request
a full sched_cfs_bandwidth_slice() of runtime and arm the deferred
throttle task_work via task_throttle_setup_work(). For intermediate
cfs_rqs the check is always false, so bandwidth-controlled cgroups
always get just 1ns of runtime and never arm the deferred throttle
work; the running task then escapes throttling until the next pick
arms the work instead, even though there is an on-rq entity at this
level.
Switch the read to cfs_rq->h_curr so intermediate bandwidth-controlled
cgroups behave consistently with the root cfs_rq, matching the
existing usage of cfs_rq->h_curr in update_curr() and
check_enqueue_throttle().
Fixes: 85570f10a4c6 ("sched/eevdf: Move to a single runqueue")
Signed-off-by: Wanwu Li <liwanwu@kylinos.cn>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Aaron Lu <ziqianlu@bytedance.com>
Tested-by: Aaron Lu <ziqianlu@bytedance.com>
Link: https://patch.msgid.link/20260831101141.391382-2-liwanwu@kylinos.cn
|
|
Commit 4b603f1551a73 ("sched: Update rq->avg_idle when a task is moved
to an idle CPU") moved rq->avg_idle accounting out of the wakeup path and
into put_prev_task_idle(), so that the idle interval is consumed whenever
the idle task is switched out.
The wakeup-side accounting that it replaced only updated rq->avg_idle
when rq->idle_stamp was non-zero. The new helper lost that validity
check and unconditionally computes:
rq_clock(rq) - rq->idle_stamp
If rq->idle_stamp is zero, this uses rq_clock(rq) as the sample. That is
not a valid idle duration and can immediately drive rq->avg_idle to its
clamp.
This can happen when sched_balance_newidle() returns before setting
rq->idle_stamp, for example when this_rq->ttwu_pending is set. In that
case the rq can switch to the idle task with idle_stamp still zero and
leave idle again when the pending wakeup is processed.
Other paths can also switch to the idle task without setting
rq->idle_stamp via newidle_balance(), for example find_proxy_task() or
force-idling.
Restore the idle_stamp validity check in update_rq_avg_idle() and skip
the rq->avg_idle update when there is no measured idle interval.
Fixes: 4b603f1551a73 ("sched: Update rq->avg_idle when a task is moved to an idle CPU")
Signed-off-by: Shubhang Kaushik (Ampere) <sh@gentwo.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
Reviewed-by: Vincent Guittot <vincent.guittot@linaro.org>
Acked-by: John Stultz <jstultz@google.com>
Link: https://patch.msgid.link/20260807-master-v3-1-c328354efed3@gentwo.org
|
|
A migrate_disable()'d RT task cannot be moved to another CPU, but the
scheduler still keeps such a task on that CPU's pushable list
(rq->rt.pushable_tasks) and still marks the runqueue RT-overloaded
(rq->rt.overloaded = 1). So the RT balancer keeps treating this CPU as
having a task to move away, and keeps trying to move the task, but the
push can never succeed. When the head is pinned, push_rt_task() does not
give up either. It falls back to pushing rq->curr instead, using the
per-CPU stopper, as added by commit a7c81556ec4d ("sched: Fix
migrate_disable() vs rt/dl balancing").
The CPU spends tens of milliseconds in this retry loop. The core is
isolated for real-time work, but during the loop nearly half of its time
is consumed by pushes that cannot succeed.
An ftrace capture of the affected CPU, with sched_switch enabled and
commit 94894c9c477e ("sched/rt: Skip currently executing CPU in
rto_next_cpu()") applied, shows where the CPU time went. Two SCHED_FIFO
tasks at equal priority shared the CPU, taskA migrate_disable()'d and
queued, taskB as rq->curr. In one 89 ms window, taskB got only 52 ms of
CPU. The other 37 ms went to the stopper thread.
The scheduler kept trying to push taskA, the pinned head of the pushable
list, fell back to pushing taskB instead, and woke the stopper 5204
times. Every one of those pushes failed and no task was moved. taskA
stayed runnable and queued the whole time, and never ran.
Pushing taskB fails on a re-check. find_lock_lowest_rq() drops the rq
lock to take the target rq lock, then checks again with
"task != pick_next_pushable_task(rq)".
The task being pushed is taskB, but the pick returns taskA, the head of
the pushable list. taskB is rq->curr, and set_next_task_rt() removes the
running task from that list, so taskB can never be the head. The check
expects a candidate taken from the pushable list, but the fallback
pushes rq->curr, which is never on that list. So the check fails every
time.
.--> push-IPI arrives
| |
| v
| pushable head = taskA -> pinned, cannot be pushed
| |
| v
| so push taskB instead -> wake migration/N, a stop-class
| | thread, so it preempts taskB
| v
| re-check compares taskB against the pushable head,
| which is still taskA -> give up
| |
| v
| nothing moved, taskA still queued, rq still overloaded
| |
'----------'
repeats every ~17 us, 5204 times, for 89 ms
The loop cannot stop itself. Every round leaves the runqueue
exactly as it was, so the next push-IPI does the same thing. In
the capture it ended only when taskB went to sleep on its own.
taskA was then picked locally and left the pushable list.
CPU time per task in the window, from sched_switch:
taskB 51.95 ms real work
migration/N 37.18 ms nothing moved
taskA 0.00 ms queued the whole time, never picked
idle 0.01 ms
Counts over the same window:
7667 push-IPIs handled on this CPU
17481 pick_next_pushable_task() returned taskA, still pinned
5204 find_lock_lowest_rq() gave up on the re-check
1 push that actually completed
0 migrations of taskA
The CPU times and the window length come from the standard
sched_switch tracepoint. The counts needed tracepoints added inside
the RT balancer for this investigation.
The self-IPI path is closed by the rto_next_cpu() fix above, and that
part works. But the runqueue is still marked overloaded, because the
pinned task is still advertised as pushable. Other CPUs now send the
push-IPIs during their own RT balancing, and the same loop runs again.
Closing the self-IPI path did not stop a pinned task from triggering
push balancing.
A pinned task should never have been returned as a push candidate in the
first place. A migrate_disable()'d task cannot be migrated, so it
belongs in the same skip that was added for on_cpu tasks by
commit e0ca8991b2de ("sched: Make class_schedulers avoid pushing
current, and get rid of proxy_tag_curr()"). Add is_migration_disabled()
to the skip condition in pick_next_pushable_task() and
pick_next_pushable_dl_task().
With the skip in place, if the pinned task is the only extra runnable
task the helpers return NULL, push_rt_task() and push_dl_task() give up
early, and no stopper is woken. The pinned task then runs locally once
curr yields. If a task that really can be migrated is queued behind the
pinned head, it is now picked and pushed for real.
This makes the fallback that pushes rq->curr unreachable when the
pushable head is migrate-disabled. Nothing is lost, because that path
was always stopped by the re-check described above. In the capture it
ran 5204 times and moved nothing.
Fixes: a7c81556ec4d ("sched: Fix migrate_disable() vs rt/dl balancing")
Signed-off-by: Seiji Nishikawa <snishika@redhat.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260830073746.2189355-1-snishika@redhat.com
|
|
pick_task_fair() and yield_task_fair() call update_curr(&rq->cfs) to bring
curr up to date before they look at the eevdf state. With cgroups that
does not happen: update_curr() reads ->h_curr, which on the root cfs_rq is
the top level group entity, and returns at the !entity_is_task() check
before touching vruntime. Both then read ->curr, so the guard and the
update disagree about which entity they mean.
Counting how often ->h_curr and ->curr differ at pick_task_fair(), on one
CPU for 10s with three busy tasks and one 200us-periodic task:
all tasks in the root cgroup 43321 calls, 0 no-ops
busy tasks in G0, periodic in G1 45211 calls, 45193 no-ops
Whether that matters depends on what precedes the pick. Since
commit 68e37487810a ("sched/fair: Fix flat hierarchy") the tick and
enqueue/dequeue all update curr correctly, so on the normal reschedule
path only the microseconds between those and the pick are missing, and I
could not measure a latency difference there. Three paths have nothing
before them on that rq though:
- pick_task() on the sibling rqs of a core under core scheduling
(kernel/sched/core.c), which updates that rq's clock first for
exactly this reason
- fair_server_pick_task()
- yield_task_fair(), where the stale value feeds the entity_eligible()
test that guards forfeiting the remaining vruntime
There curr can be a full tick behind, as it was before that commit.
No new behaviour for the entity being updated: without cgroups ->h_curr
is already the task, so these two call sites already run the full
update_curr() including update_deadline(), dl_server_update() and the
resched_curr_lazy() at the end. This makes the cgroup case do the same.
Fixes: 85570f10a4c6 ("sched/eevdf: Move to a single runqueue")
Signed-off-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Vincent Guittot <vincent.guittot@linaro.org>
Link: https://patch.msgid.link/20260822105930.2352761-1-zhanxusheng1024@gmail.com
|
|
ops.cgroup_set_idle() is documented to be invoked when a cgroup
transitions between idle and non-idle states, and scx_group_set_weight()
already skips value-preserving writes. scx_group_set_idle() delivers
every write unconditionally, so rewriting an already-correct cpu.idle
value feeds the BPF scheduler a transition callback each time, which
toggle- or accounting-based schedulers miscount. Mirror the weight
guard and only deliver on an actual change.
Verified with a probe scheduler printing each callback: rewriting
cpu.idle=1 twice on an already-idle cgroup delivered two callbacks
before and none after.
Fixes: 347ed2d566da ("sched/ext: Implement cgroup_set_idle() callback")
Link: https://lore.kernel.org/r/b53c61a1-4d7d-4232-941f-d48b0563d4ed@linux.dev
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
scx_bpf_dsq_insert_vtime() orders tasks by time_before64(), which is
only meaningful when the values within a given DSQ stay less than 2^63
apart. This is implicit in how a vtime scheduler works but not spelled
out anywhere. Document it so BPF scheduler authors know the
constraint.
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
Mukesh Pilaniya <mpilaniy@redhat.com> says:
kexec: fix probe error codes and error propagation
While debugging a misleading error on s390x where kexec -s reported
"syscall kexec_file_load not available" instead of the actual EINVAL
from a kernel command line that exceeded the architecture limit, it was
found that some kexec image probe functions return -EINVAL instead of
-ENOEXEC for format mismatches, and kexec_image_probe_default() has a
bug where it returns whatever the last loader returned rather than
always returning -ENOEXEC when no loader matches.
* Fix all probe functions to return -ENOEXEC when they do not recognize
the image format.
* Fix kexec_image_probe_default() to distinguish format mismatches
(-ENOEXEC) from real errors (e.g. -ENOMEM), propagating real errors
immediately and only continuing to the next loader on -ENOEXEC.
kexec-tools patch:
https://lore.kernel.org/all/20260814075329.30203-1-mpilaniy@redhat.com/
Link: https://patch.msgid.link/20260821-mpilaniy-v3-0-76b355399d2e@redhat.com
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
|
|
kexec_image_probe_default() overwrites ret with each loader's probe
return value and returns whatever the last loader returned when no
probe matches. The error code reaching userspace depends on whichever
loader happens to be last in kexec_file_loaders[].
Only continue to the next loader when a probe returns -ENOEXEC.
Propagate any other error such as -ENOMEM immediately. Return -ENOEXEC
only when all loaders have been tried and none matched.
Suggested-by: Pratyush Yadav <pratyush@kernel.org>
Signed-off-by: Mukesh Pilaniya <mpilaniy@redhat.com>
Reviewed-by: Pratyush Yadav <pratyush@kernel.org>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Link: https://patch.msgid.link/20260821-mpilaniy-v3-2-76b355399d2e@redhat.com
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
|
|
Several kexec_file_load() image probe functions return -EINVAL when
they do not recognize the image format. A probe function that rejects
an image should return -ENOEXEC to indicate that the image is not a
recognized executable format. -EINVAL implies a problem with the
syscall parameters, not with image recognition.
kexec_image_probe_default() iterates through registered loaders and
returns the last probe's error code to the caller. That error
propagates as the kexec_file_load() return value to userspace.
Returning -EINVAL from a probe when no loader matches is semantically
incorrect and misleads userspace about the nature of the failure.
Return -ENOEXEC from all probe functions and their helpers when the
image format is not recognized.
Signed-off-by: Mukesh Pilaniya <mpilaniy@redhat.com>
Reviewed-by: Philipp Rudo <prudo@redhat.com>
Reviewed-by: Pratyush Yadav <pratyush@kernel.org>
Reviewed-by: Bradley Morgan <include@grrlz.net>
Link: https://patch.msgid.link/20260821-mpilaniy-v3-1-76b355399d2e@redhat.com
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
|
|
The audit_n_rules and audit_signals counters are incremented when filter
rules are installed and decremented by the explicit rule deletion path.
Rules can also disappear when a watch or tree is removed, or when an LSM
rule cannot be reconstructed, but those paths do not update the counters.
As a result, audit_n_rules can remain nonzero after the last applicable
rule has gone away, causing subsequent syscalls to allocate non-dummy
audit contexts unnecessarily. A stale audit_signals value similarly
causes unnecessary signal auditing work.
This can be reproduced for an inode watch with:
mkdir /tmp/audit-n-rules-bench
touch /tmp/audit-n-rules-bench/watched
auditctl -w /tmp/audit-n-rules-bench/watched -p r \
-k audit_n_rules_bench
rm /tmp/audit-n-rules-bench/watched
rmdir /tmp/audit-n-rules-bench
The rm updates the watch after its inode disappears, and the rmdir causes
audit_remove_parent_watches() to remove the rule. For an audit tree, the
kill_rules() path can be reproduced with:
mkdir /tmp/audit-kill-rules
auditctl -a always,exit -F arch=b64 \
-F dir=/tmp/audit-kill-rules -F perm=r \
-k audit_kill_rules_test
rmdir /tmp/audit-kill-rules
In both cases, auditctl -l reports no rules after the directory is
removed.
Factor the existing counter updates into common rule insertion and removal
helpers and call the removal helper from every automatic removal path. All
of these updates remain serialized by audit_filter_mutex.
Fixes: 471a5c7c8391 ("[PATCH] introduce audit rules counter")
Fixes: e54dc2431d74 ("[PATCH] audit signal recipients")
Signed-off-by: Stanislav Kinsburskii <skinsburskii@gmail.com>
[PM: removed "audit_bench" section as that is not being upstreamed]
Signed-off-by: Paul Moore <paul@paul-moore.com>
|
|
The trace instance files set_ftrace_filter and set_ftrace_notrace was
updated to work with specific trace instances (trace_arrays). The issue is
that when these files are opened, there is a small race window where it
will use the ftrace_ops from the inode->private pointer to get a reference
to the trace_array and then take its reference. The problem is that the
ftrace_ops itself could be freed. If the rmdir on the instance happens at
the same time the set_ftrace_filter file is opened, the rmdir could have
also freed the ftrace_ops and referencing it will cause a use-after-free
bug and crash the kernel.
Instead, pass in the trace_array as the file private data (NULL for the
top level instance), and then pass both the trace_array and the ftrace_ops
to the ftrace_regex_open() function. If the trace_array is NULL, then it
just uses the ftrace_ops without the need to take its reference (like
normal). If the ftrace_ops is NULL, that is only the case for the top
level instance and the global_ops can be used.
This allows the trace_array to have its reference incremented before
touching the ftrace_ops that could also be freed when the instance is.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260828223901.29e26edb@robin
Fixes: 591dffdade9f0 ("ftrace: Allow for function tracing instance to filter functions")
Reported-by: Breno Leitao <leitao@debian.org>
Tested-by: Breno Leitao <leitao@debian.org>
Closes: https://lore.kernel.org/all/apGORjltZgAiAYHT@gmail.com/
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
The newly added files show_event_filters and show_event_triggers that show
all filters or triggers that are set within the trace array do not take a
reference for the trace array it is showing. Without taking a reference,
the trace_array may be freed via "rmdir" while a task is reading one of
theses files. Those files iterate all the events within an instance
(trace_array) and nothing prevents that instance from being freed while
its data is being read. This causes a use-after-free crash.
Have the open of both those files take the trace_array reference via the
trace_array_get() that prevents the trace_array from being freed while the
files are opened.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260828094153.17b95037@gandalf.local.home
Fixes: 729757b96a662 ("tracing: Add show_event_filters to expose active event filters")
Fixes: 6a80838814eea ("tracing: Add show_event_triggers to expose active event triggers")
Reported-by: Farhad Alemi <farhad.alemi@berkeley.edu>
Closes: https://lore.kernel.org/all/CA+0ovCjerKZJLwXScM9bF2ga2rLi4_XOpUfK41NDbENpeu98jA@mail.gmail.com/
Reviewed-by: Aaron Tomlin <atomlin@atomlin.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Only user DSQs use the per-DSQ deferred reenqueue state:
schedule_dsq_reenq() serves the local DSQ through
sch->pcpu->deferred_reenq_local and rejects all other built-in DSQ ids.
Every DSQ, built-in or not, still allocates a struct scx_dsq_pcpu per
possible CPU. As the local, reject, rescue and bypass DSQs are themselves
per-CPU, the wasted memory grows with the square of the CPU count, most of
it at boot before any scheduler is loaded.
Skip the allocation for built-in DSQ ids. While at it, rename ->pcpu to
->pcpu_user so that the per-CPU data reads as user-DSQ only.
Signed-off-by: fangqiurong <fangqiurong@kylinos.cn>
Reviewed-by: Zhan Xusheng <zhanxusheng@xiaomi.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
Sashiko pointed out the some of the scope cleanups for free_uprobe
could get an error pointer. Handle this case in free_uprobe
to prevent a crash.
On the other hand the macro doesn't need the guard because
free_uprobe itself already does the check.
Link: https://lore.kernel.org/all/20260831150651.1134594-2-ak@kernel.org/
Assisted-by: omp:gpt-5.6-luna sashiko
Signed-off-by: Andi Kleen <ak@kernel.org>
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
install_unbound_pwq() fills the pwq slots of a percpu workqueue as well now
that alloc_and_link_pwqs() sends it through the attrs path, and
unbound_pwq_slot() and unbound_pwq() have always addressed and read those
slots for every workqueue, percpu included, from destroy_workqueue().
Their names say otherwise. Rename them to install_pwq(), pwq_slot() and
installed_pwq().
Signed-off-by: Breno Leitao <leitao@debian.org>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
Drop the WQ_PERCPU allocation path (alloc_and_link_percpu_pwqs()).
Now that alloc_pwq() hands out the static percpu pool for a CPU and the
default pwq is optional, the unbound path produces the same result.
Point a percpu workqueue at it and drop alloc_and_link_percpu_pwqs().
Nothing is installed in wq->cpu_pwq when the attrs path fails, since
apply_wqattrs_commit() does not run, so the enomem cleanup only has the
percpu array left to free.
Signed-off-by: Breno Leitao <leitao@debian.org>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
apply_wqattrs_prepare() always creates a dfl_pwq, the fallback a CPU
uses when it has no pwq of its own. This is a CPU unbound field, so, we
don't need it for the per cpu affinity version of workqueue.
A percpu workqueue has no use for one. Skip the allocation for
a WQ_PERCPU workqueue and skip installing what was not allocated.
Signed-off-by: Breno Leitao <leitao@debian.org>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
Unify the pool allocation in alloc_pwq(), instead of only getting the unbound
pool, and add a new parameter for the CPU.
Create a helper for unbound allocations (alloc_unbound_pwq()).
Signed-off-by: Breno Leitao <leitao@debian.org>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
apply_workqueue_attrs_locked() refuses any workqueue without WQ_UNBOUND.
That is the right rule for changing a workqueue's attributes, but it also
blocks the internal callers, and building a percpu workqueue's pwq table
through this path needs them.
Move the check into apply_workqueue_attrs(), the exported entry point,
which is the one that changes attributes on a live workqueue.
Signed-off-by: Breno Leitao <leitao@debian.org>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
A bare "idle" reads as CPU idle state in sched_ext (ops.update_idle(),
idle cpumasks, scx_bpf_pick_idle_cpu()). cpu.idle is the cgroup analog
of the SCHED_IDLE policy, so name the task_group field sched_idle to
match scx_cgroup_init_args.sched_idle.
Pure rename, no behavior change.
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
scx_cgroup_init_args carries the initial weight and bandwidth control
parameters of a cgroup to ops.cgroup_init(), but not its cpu.idle
state. A cgroup that was already configured idle before the scheduler
was loaded (or before it was onlined under it) is presented as
non-idle, and the BPF scheduler only learns about it if cpu.idle is
written again later.
Add the sched_idle state to scx_cgroup_init_args and fill it in all
four places that build the args: scx_tg_online() for cgroups onlined
under the scheduler, scx_cgroup_init() for cgroups that already exist
when the scheduler is loaded, and the sub-scheduler handover paths
scx_cgroup_claim_subtree() and scx_cgroup_return_subtree().
Verified in a VM with a probe scheduler printing the init args: a
cgroup configured cpu.idle=1 before loading shows sched_idle=1 in
ops.cgroup_init(), the default shows 0, and later cpu.idle writes
still come through ops.cgroup_set_idle(). The sub-scheduler paths
are compile-tested only.
Fixes: 347ed2d566da ("sched/ext: Implement cgroup_set_idle() callback")
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq
Pull workqueue fixes from Tejun Heo:
- An unbound worker pool could be freed while still reachable through
the pending-activation list, leading to a use-after-free. Unlink
before dropping the reference
- On PREEMPT_RT, the BH workqueue kick raised softirqs from preemptible
context, tripping a lockdep assertion and possibly losing
concurrently raised softirq bits
- Draining BH work off a dead CPU nests two pools' callback locks,
which lockdep misreported as recursive locking. The nesting cannot
deadlock. Annotate it
- Reject watchdog thresholds that overflow the conversion to jiffies
- Make the drgn workqueue dump script work again on kernels and vmcores
from before the workqueue attrs field rename
* tag 'wq-for-7.3-rc1-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq:
tools/workqueue/wq_dump.py: Support backward compatibility for wq->attrs rename
workqueue: reject watchdog thresholds that overflow jiffies
workqueue: Fix unbound pool lifetime for pending pwqs
workqueue: Use raise_softirq() to trigger softirq in irq_work handler
workqueue: Annotate cb_lock nesting when draining a dead BH pool
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup
Pull cgroup fixes from Tejun Heo:
- After cgroup.kill was written to a cgroup, every child cloned into it
with CLONE_INTO_CGROUP was spuriously killed because the fork path
snapshotted the kill counter before resolving the target cgroup
- Releasing an isolated cpuset partition dropped the isolation of CPUs
isolated on the kernel command line
- Selftest and documentation fixes
* tag 'cgroup-for-7.3-rc1-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup:
selftests/cgroup: test clone3() into a previously killed cgroup
cgroup: fix spurious SIGKILL of CLONE_INTO_CGROUP children
selftests/cgroup: Add test for preserving boot-isolated CPUs
cgroup/cpuset: Preserve boot-isolated CPUs on partition release
selftests/cgroup: Drop invalid boot isolation comparison
docs: cgroup-v2: fix misc.events key format description
selftests/cgroup: Fix cg_run_in_subcgroups ignoring arg parameter
selftests/cgroup: set the test plan after the setup checks
|
|
All namespace types now share the same ns_common infrastructure. Extend
this to include a security blob so LSMs can start managing namespaces
uniformly without having to add one-off hooks or security fields to
every individual namespace type.
Add a ns_security pointer to ns_common and the corresponding lbs_ns blob
size to lsm_blob_sizes. Allocation and freeing hooks are called from the
common __ns_common_init() and __ns_common_free() paths so every
namespace type gets covered in one go. All information about the
namespace type and the appropriate casting helpers to get at the
containing namespace are available via ns_common making it
straightforward for LSMs to differentiate when they need to.
A namespace_install hook is called from validate_ns() during setns(2)
giving LSMs a chance to enforce policy on namespace transitions. The
LSM check runs before ns->ops->install() so the security module can deny
the operation before any type-specific installation effects.
Individual namespace types can still have their own specialized security
hooks when needed. This is just the common baseline that makes it easy
to track and manage namespaces from the security side without requiring
every namespace type to reinvent the wheel.
Cc: Günther Noack <gnoack@google.com>
Cc: Paul Moore <paul@paul-moore.com>
Cc: Serge E. Hallyn <serge@hallyn.com>
Signed-off-by: Christian Brauner <brauner@kernel.org>
Link: https://lore.kernel.org/r/20260216-work-security-namespace-v1-1-075c28758e1f@kernel.org
Signed-off-by: Mickaël Salaün <mic@digikod.net>
[PM: subject tweak]
Signed-off-by: Paul Moore <paul@paul-moore.com>
|
|
free_mnt_ns() skipped ns_common_free() for anonymous mount namespaces
(the "if (!is_anon_ns(ns))" guard) because they carry the reserved inum
MNT_NS_ANON_INO, which proc_free_inum() must never release.
A following change needs ns_common_free() to run for every namespace, to
release per-namespace state attached during __ns_common_init(). Move
the reserved-inum decision into __ns_common_free() and let free_mnt_ns()
call ns_common_free() unconditionally.
__ns_common_free() is shared by all namespace types, so it gates
proc_free_inum() on ns->inum > MNT_NS_INO_SPECIAL_MAX (a new alias for
MNT_NS_ANON_INO) rather than the mount-specific is_anon_ns(). The
reserved inums (MNT_NS_ANON_INO and the *_NS_INIT_INO values just above
it) belong to namespaces that are never freed, except the anonymous
mount namespace; dynamically allocated inums are >= PROC_DYNAMIC_FIRST.
So the comparison frees every dynamic inum and skips exactly the
anonymous mount namespace, matching the previous guard.
Cc: Günther Noack <gnoack@google.com>
Cc: Paul Moore <paul@paul-moore.com>
Signed-off-by: Christian Brauner <brauner@kernel.org>
Co-developed-by: Mickaël Salaün <mic@digikod.net>
Signed-off-by: Mickaël Salaün <mic@digikod.net>
Reviewed-by: Christian Brauner (Amutable) <brauner@kernel.org>
Signed-off-by: Paul Moore <paul@paul-moore.com>
|
|
The `cred` field in task_struct is currently marked as __rcu, but that's
not true: It can point to credentials from access_override_creds(), which
do not get freed with RCU delay.
What actually protects task_struct::cred is that accessing it is only
permitted for the current task (except for setting up a child during fork()
or tearing down a dead process).
(There is currently code in Smack that violates this rule, but that's a bug
and causes UAF, I have sent a separate fix for that.)
Clarify this, remove the __rcu marker, and remove RCU helpers from all
accesses to this field.
Signed-off-by: Jann Horn <jannh@google.com>
Reviewed-by: Serge Hallyn <serge@hallyn.com>
[PM: style tweak in security_init(), applied fixup from JH]
Signed-off-by: Paul Moore <paul@paul-moore.com>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext
Pull sched_ext fixes from Tejun Heo:
- The task ownership check in the dispatch queue move operation raced
against the task exiting or moving to a different sub-scheduler,
spuriously triggering scheduler aborts. Fix by moving the check under
the queue lock
- The cgroup bandwidth change callback runs in a sleepable context but
sleepable implementations were rejected at load time. Allow them and
add a marker so userspace can detect the capability
- Sync tooling headers with the scx repo for accumulated compatibility
improvements
- Example scheduler fixes: ignored timer re-arm failures and vtime
credit loss on cgroup migration
- Documentation and comment fixes
* tag 'sched_ext-for-7.3-rc1-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext:
sched_ext: Fix missing @slice and @vtime descriptions in finish_dispatch() kernel-doc
sched_ext: Fix several comment issues
sched_ext: Check bpf_timer_start return values in scx_qmap
sched_ext: Fix vtime delta loss in scx_flatcg cgroup migration
sched_ext: Fix timer pinning and return value in scx_central
docs/sched_ext: document that cgroup CPU knobs are scheduler-dependent
sched_ext: Fix spurious aborts in scx_bpf_dsq_move() on ownership change races
sched_ext: Sync common and compat headers from the scx repo
sched_ext: Sync tools autogen enum headers from the scx repo
Docs/admin-guide/cgroup-v2: document BPF scheduler callbacks for cpu.max and cpu.idle
sched_ext: Fix nonexistent field in sched-ext.rst example
sched_ext: Allow ops.cgroup_set_bandwidth() to be sleepable
|
|
The netlink header's nlmsg_len was being set using the original skb
instead of the local copy passed to nlmsg_multicast().
While skb->len and copy->len are identical at this point following
skb_copy(), it is conceptually cleaner and safer to use the length
of the skb being actively modified and sent. Thus, this is a minor
cleanup only, no functional change expected.
Signed-off-by: Ricardo Robaina <rrobaina@redhat.com>
Reviewed-by: Bradley Morgan <include@grrlz.net>
Signed-off-by: Paul Moore <paul@paul-moore.com>
|
|
Between the actual process startup (fork systemd) and the executable file
replacement (exec), systemd sets a temporary file name (executable file
name in parentheses). If an auditable system call occurs at this point,
the audit context will latch the temporary process name into the cache.
This name will not change again. The patch clears proctitle into the
audit cache when the exec call is made, allowing the new process name to
be latched.
Suggested-by: Roman Dolgikh <rmd4work@mail.ru>
Link: https://github.com/linux-audit/audit-kernel/issues/170
Signed-off-by: Richard Guy Briggs <rgb@redhat.com>
Reviewed-by: Ricardo Robaina <rrobaina@redhat.com>
Reviewed-by: Bradley Morgan <include@grrlz.net>
Signed-off-by: Paul Moore <paul@paul-moore.com>
|
|
Pull xfs fixes from Carlos Maiolino:
"This contains a few fixes for the zoned storage support, a possible
deadlock vector fix, some code refactoring patches and a quota evasion
fix on XFS while exporting it via NFS.
Please note that for the quota evasion fix, a couple patches for the
capability subsystem are included in the pull request. Those have been
ack'ed by the respective maintainer which also agreed to have them
going through the xfs tree.
This also includes a patch for the quota subsystem to stop issuing
audit messages during quota enforcing. Quota maintainer also ack'ed
and agreed with this going through xfs tree"
* tag 'xfs-fixes-7.3-rc2' of gitolite.kernel.org:/pub/scm/fs/xfs/xfs-linux:
capability: unexport has_capability_noaudit
xfs: replace ns_capable_noaudit
quota: Don't issue audit messages on quota enforcing
capability: Add new capable_noaudit
xfs: fix capability check in xfs
xfs: restore bi_bdev in xfs_zone_gc_write_chunk
xfs: split ioend handling into a separate source file
xfs: factor out a xfs_iomap_set_anon_write helper
xfs: fix zoned write iomap flags assignments
xfs: fix racy open zone caching
xfs: handle NULL open_zone for merged ioends in xfs_ioend_put_open_zones
xfs: use inode_init_always_gfp with __GFP_NOFAIL in xfs_inode_alloc
xfs: remove kmem_to_page()
xfs: don't flush and invalidate internal RT device twice in xfs_shutdown_devices
xfs: split an assert in xfs_trans_log_buf
xfs: don't hold buffer locks across sync transaction commit in xfs_sync_sb_buf
|
|
The watchdog threshold is supplied in seconds but is multiplied by HZ
before being used as a jiffies interval. Reject values that exceed
MAX_JIFFY_OFFSET / HZ so the multiplication cannot wrap and the
time_after() comparisons remain within their supported range.
The check is performed before changing the threshold or watchdog timer.
Zero remains the value used to disable the watchdog.
Fixes: 82607adcf9cdf ("workqueue: implement lockup detector")
Signed-off-by: Jiacheng Xu <stitch@zju.edu.cn>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
kernel-doc
Commit 13f1eae3b662 ("sched_ext: Synchronize slice and dsq_vtime
writes") added the slice and vtime parameters to finish_dispatch() but
did not update its kernel-doc, which produces warnings:
Warning: function parameter 'slice' not described in 'finish_dispatch'
Warning: function parameter 'vtime' not described in 'finish_dispatch'
Describe both parameters using the same wording as
dispatch_to_local_dsq(), which receives the same values.
Signed-off-by: Liang Luo <luoliang@kylinos.cn>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
Fix several comment issues found during review:
__setschduler_prio() -> __setscheduler_class()
scx_iter_scx_dsq_new() -> bpf_iter_scx_dsq_new()
scx_next_task_scx() -> set_next_task_scx()
Signed-off-by: Wanwu Li <liwanwu@kylinos.cn>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
Since commit b69bb476dee9 ("cgroup: fix race between fork and
cgroup.kill"), the fork path snapshots the kill_seq of the child's
future cgroup into kargs->kill_seq, and cgroup_post_fork() SIGKILLs
the child if that cgroup's kill_seq has changed in the meantime, to
catch forks racing with a cgroup.kill sweep.
For CLONE_INTO_CGROUP, however, the snapshot in cgroup_css_set_fork()
is taken before the target cgroup has been resolved: kargs->cgrp is
always NULL at this point (it is only set at the end of the function).
So the "if (kargs->cgrp)" branch is dead code and the snapshot always
records the kill_seq of the parent's cgroup. cgroup_post_fork() then
compares it with the kill_seq of the target cgroup, so the child gets
SIGKILLed whenever the two cgroups have been killed a different number
of times.
As a result, once cgroup.kill has been written to a cgroup, every
child subsequently cloned into it with clone3(CLONE_INTO_CGROUP) is
killed on the spot, for as long as the cgroup exists: kill_seq is not
exposed to userspace and never resets.
Re-snapshot kill_seq from the target cgroup once it has been resolved,
and drop the dead branch at the early snapshot site.
This does not reopen the race fixed by b69bb476dee9. For
CLONE_INTO_CGROUP, everything from the snapshot to the check in
cgroup_post_fork() runs with cgroup_mutex held, and kill_seq is
only ever incremented under cgroup_mutex.
tj: Updated the comment above kill_seq to reflect the new serialization
rules as suggested by Shakeel Butt.
Fixes: b69bb476dee9 ("cgroup: fix race between fork and cgroup.kill")
Cc: stable@vger.kernel.org
Cc: Shakeel Butt <shakeel.butt@linux.dev>
Assisted-by: LLM
Signed-off-by: Etienne Perot <eperot@google.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
Memory failures (such as unrecoverable ECCs errors) are getting more and
more common. The kernel knows how to handle it while running, marking it
as poisoned (and SIGBUS user tasks).
Poisoned memory is removed from the buddy allocator, but, not from
other places. A current problem is that kexec will load new kernel
on top of a bad/poisoned memory, which is undesirable.
If the next kernel's image, initrd or purgatory lands on a poisoned
frame, the relocation copy puts it on memory that is known bad. The
error happens on the first read from a bad page, and that is what we
want to avoid.
Skip hardware-poisoned frames that were detected by the memory failure
subsystem earlier when placing kexec segments.
To do so, add a helper that reports the first or the last poisoned page
in a range: memory is walked top-down by locate_mem_hole_top_down() and
bottom-up by locate_mem_hole_bottom_up(), so each direction needs a
different answer to stay clear of the poison.
kexec_load() gets its destinations from userspace and cannot move them,
so there sanity_check_segment_list() just rejects a segment that happens
to have a poisoned page.
is_page_hwpoison() also covers hugetlb, so a poisoned hugetlb folio is
skipped as a whole.
Suggested-by: Kiryl Shutsemau <kas@kernel.org>
Signed-off-by: Breno Leitao <leitao@debian.org>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Pratyush Yadav <pratyush@kernel.org>
Reviewed-by: Bradley Morgan <include@grrlz.net>
Reviewed-by: Miaohe Lin <linmiaohe@huawei.com>
Link: https://patch.msgid.link/20260812-kexec_posioned-v6-2-e477887086f0@debian.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
|
|
locate_mem_hole_top_down() walks candidates downwards by subtracting
PAGE_SIZE whenever the window conflicts with an existing segment or with
an architecture exclude range.
Nothing stops that subtraction at zero, so a search that reaches the
bottom of the address space wraps temp_start around and continues.
The walk starts inside the range being scanned and only moves down, so
bail out once a candidate ends up above end. That covers every step in
the loop rather than the subtractions alone, and it matches
locate_mem_hole_bottom_up(), which already bounds its candidate on both
sides.
This is a better check than subtracting with check_sub_overflow(), given
that we would have 3 subtractions in this block, and this single fix
would take care of them all (instead of three check_sub_overflow()).
Fixes: cb1052581e2b ("kexec: implementation of new syscall kexec_file_load")
Signed-off-by: Breno Leitao <leitao@debian.org>
Reviewed-by: Bradley Morgan <include@grrlz.net>
Link: https://patch.msgid.link/20260812-kexec_posioned-v6-1-e477887086f0@debian.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
|
|
If a kernel with CONFIG_KEXEC_HANDOVER_ENABLE_DEFAULT=y (or kho=on)
does not receive a handover FDT, it reserves KHO scratch memory in
kho_reserve_scratch().
The kdump kernel always takes this path. Since commit a6715d7ec472
("kho: skip KHO for crash kernel"), the kernel does not add KHO
metadata to the crash kimage. As a result, the kdump kernel never
receives a handover FDT and always reserves scratch memory.
The kdump kernel boots in the small crashkernel= memory reservation.
On our x86 hosts, the kernel and initramfs make approximately 176 MB
of early reservations. The scratch areas then use approximately
650 MB more of the crashkernel memory. When a large early allocation
fails, the kdump kernel panics:
bio: can't create integrity buf pool
A kdump kernel has nothing to hand over: its only task is to dump the
memory of the old kernel and reboot. Fix is to disable KHO at the top of
kho_memory_init() so that the kdump kernel does not reserve scratch
memory.
Fixes: 3dc92c311498 ("kexec: add Kexec HandOver (KHO) generation helpers")
Cc: stable@vger.kernel.org
Signed-off-by: Maxi Saparov <masaparov@coreweave.com>
Link: https://patch.msgid.link/20260811161857.3096-1-maxi.saparov@gmail.com
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
|
|
create_user_ns() jumps to fail_keyring when setup_userns_sysctls()
fails. At that point ns was freshly allocated with
kmem_cache_zalloc() and ns->persistent_keyring_register is only ever
assigned later, lazily, from key_get_persistent()
(security/keys/persistent.c). The key_put() therefore always
receives NULL.
Remove the dead call.
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
Link: https://patch.msgid.link/20260828091659.358700-2-cui.tao@linux.dev
Reviewed-by: Jan Kara <jack@suse.cz>
Reviewed-by: Bradley Morgan <brads@mainlining.org>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
cad_pid is a single kernel-wide struct pid pointer. proc_do_cad_pid()
reads it and passes it to pid_vnr() without protecting the lifetime of
the referenced struct pid. A concurrent writer can replace cad_pid and
drop the final reference to the old struct pid after the reader has
loaded the pointer but before pid_vnr() has finished dereferencing it,
causing a use-after-free.
kill_cad_pid() has the same lifetime race when it passes cad_pid to
kill_pid().
At the time this issue was reported, an unprivileged user could reach the
sysctl through user and PID namespaces because cad_pid was registered in
pid_table[]. Moving cad_pid back to the global reboot sysctl table
corrected that namespace and permission mismatch, but did not fix the
underlying lifetime race.
Fix this by treating cad_pid as an RCU-protected pointer at both read
sites and by waiting for a grace period before dropping the old reference
on the write side.
call_rcu(&old_pid->rcu, ...) cannot be used here because free_pid()
also queues pid->rcu; queueing the same rcu_head twice can corrupt the
RCU callback list.
Original KASAN crash stack:
kernel/pid.c:545 pid_nr_ns() # reads freed pid->level
kernel/pid.c:556 pid_vnr() # calls pid_nr_ns()
kernel/pid.c:775 proc_do_cad_pid() # calls pid_vnr(cad_pid)
Fixes: 9ec52099e4b8 ("[PATCH] replace cad_pid by a struct pid")
Reported-by: AutonomousCodeSecurity@microsoft.com
Closes: https://lore.kernel.org/all/20260717210143.4734-1-blbllhy@gmail.com/
Link: https://lore.kernel.org/all/alz5ZYLE4kaq_v2P@redhat.com/
Link: https://lore.kernel.org/all/al4ICz9biJKtdZc4@redhat.com/
Suggested-by: Mateusz Guzik <mjguzik@gmail.com>
Suggested-by: Bradley Morgan <include@grrlz.net>
Suggested-by: Oleg Nesterov <oleg@redhat.com>
Suggested-by: Eric W. Biederman <ebiederm@xmission.com>
Suggested-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Cc: stable@vger.kernel.org
Signed-off-by: Cen Zhang (Microsoft) <blbllhy@gmail.com>
Link: https://patch.msgid.link/20260814040944.16561-1-blbllhy@gmail.com
Reviewed-by: Bradley Morgan <include@grrlz.net>
Reviewed-by: Oleg Nesterov <oleg@redhat.com>
Reviewed-by: Pavel Tikhomirov <ptikhomirov@virtuozzo.com>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
