| Age | Commit message (Collapse) | Author |
|---|
|
bpf_map_alloc_pages() does not use its map argument. Storing allocated
pages in an array also forces arena callers to allocate a separate
pointer array.
Expose the single-page allocator as bpf_alloc_page(), rename the bulk
helper to bpf_alloc_pages(), and return bulk allocations through an
llist using page->pcp_llist. Add bpf_free_pages() to safely release all
pages remaining on such a list.
Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261002105218.6171-2-emil@etsalapatis.com
|
|
bpf_mem_cache_free_rcu() uses this_cpu_ptr() which requires migration
to be disabled. All callers of rhtab_delete_elem() disable migration
except __rhtab_map_lookup_and_delete_batch(), which calls it under
rcu_read_lock() only.
On CONFIG_PREEMPT_RCU, rcu_read_lock() does not disable preemption or
migration, so the task can migrate between CPUs during the delete loop,
causing this_cpu_ptr() to trigger:
BUG: using smp_processor_id() in preemptible [00000000] code
Fix by wrapping the delete loop in migrate_disable()/migrate_enable()
in __rhtab_map_lookup_and_delete_batch(), matching the migration
protection that the other callers already provide.
Fixes: 818e00848227 ("bpf: Implement iteration ops for resizable hashtab")
Reported-by: syzbot+fd7e415d891073b83e1f@syzkaller.appspotmail.com
Signed-off-by: Ömer Mete Kaya <omermetekaya0@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Mykyta Yatsenko <yatsenko@meta.com>
Link: https://patch.msgid.link/20260929081609.557899-1-omermetekaya0@gmail.com
Closes: https://syzkaller.appspot.com/bug?extid=fd7e415d891073b83e1f
|
|
To confirm the max nesting level of local_interrupt_disable() works, a
kunit test is added to the whole test suite.
Note that when DEBUG_PREEMPT=y, it'll generate a warning which is
expected.
Suggested-by: Shrikanth Hegde <sshegde@linux.ibm.com>
Signed-off-by: Boqun Feng <boqun@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Shrikanth Hegde <sshegde@linux.ibm.com>
Link: https://patch.msgid.link/20261001082519.16195-6-boqun@kernel.org
|
|
poll_state_synchronize_rcu(mm->futex.phash.batches) is used by
futex_ref_drop() to check that a grace period has passed since the
current hash was published. This relies on batches referencing a grace
period which started after the hash pointer was assigned.
__futex_pivot_hash() sets mmph->batches before it replaces mmph->hash:
scoped_guard(rcu) {
mmph->batches = get_state_synchronize_rcu();
rcu_assign_pointer(mmph->hash, new);
}
The scoped_guard(rcu) doesn't stop new grace periods from starting, and
if one starts between those two assignments, futex_ref_drop() can move
forward while a reader still holds a pointer to the old hash.
Fix things by setting mmph->batches after assigning mmph->hash. The
scoped_guard(rcu) isn't needed, so let's drop that as well.
Fixes: 56180dd20c19 ("futex: Use RCU-based per-CPU reference counting instead of rcuref_t")
Assisted-by: kres
Signed-off-by: Chris Mason <mason@kernel.org>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Paul E. McKenney <paulmck@kernel.org>
Link: https://patch.msgid.link/20261001135022.2220288-1-mason@kernel.org
|
|
Currently, refcount_interrupt_test is built unconditionally when
CONFIG_KUNIT is enabled, causing it to run unexpectedly during boot.
Fix this by introducing CONFIG_REFCOUNT_INTERRUPT_KUNIT_TEST so the
test can be configured independently, following standard kunit
practices.
Fixes: 07a88e2bcd5b ("irq: Add KUnit test for refcounted interrupt enable/disable")
Signed-off-by: Kuan-Wei Chiu <visitorckw@gmail.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Lyude Paul <lyude@redhat.com>
Reviewed-by: Radu Rendec <radu@rendec.net>
Reviewed-by: Boqun Feng <boqun@kernel.org>
Link: https://patch.msgid.link/20261001082519.16195-2-boqun@kernel.org
|
|
|
|
The Braille console is not registered in console_list. Instead, it is
integrated with the virtual terminal (VT) and shows what is displayed
on the terminal. It writes the data using con->write*() callback
of the associated serial console driver.
braille_write() is called from the VT code under console_lock().
The associated serial console driver can be converted to the NBCON API
though. The situation is similar to flushing nbcon consoles in the legacy
loop when some boot consoles are still registered, see
nbcon_legacy_emit_next_record().
But there is a big difference though. braille_write() is not directly
called from the code paths flushing registered consoles. The VT code
expects that braille_write() succeeds. It does not replay the message
when it can't acquire the ownership. As a result, braille_write():
+ must try harder to get the ownership.
+ has to be synchronized only against non-printk serial console
which depend nbcon_device_try_acquire() using NBCON_PRIO_NORMAL.
Let's look at it from another side and try to simulate the original
locking using the NBCON API:
1. Take con->device_lock(), aka the port->lock in the legacy serial
console driver.
2. Acquire nbcon context to provide some synchronization for a panic()
context. Use NBCON_PRIO_NORMAL because it contends only with
nbcon_device_try_acquire() users. Do it in a busy loop. It should
always succeed when con->device_lock() succeeded because all other
users do the same. The only exception is when the context get acquired
by a CPU handling panic.
3. In panic, disable interrupts and try to acquire the nbcon context.
Use NBCON_PRIO_PANIC. And try even an unsafe takeover because
otherwise the Braille console won't see the text shown during panic().
It is similar to the oops_in_progress/trylock handling in the legacy
serial console driver.
Finally, avoid the newline prepending logic in the existing serial console
drivers when they are used as a Braille console. As explained above,
the Braille console shows the last modified line on the terminal (VT).
braille_write() is called when single characters are added. Most
messages are not ended by newline. Anyway, the VT code does not have
logic to reply partially printed messages.
Fixes: 13189fa73afa ("printk: nbcon: Rely on kthreads for normal operation")
Reviewed-by: John Ogness <john.ogness@linutronix.de>
[pmladek@suse.com: Remove white space changes in braille_register_console(). Typo fix.]
Link: https://patch.msgid.link/20261001140727.124398-2-pmladek@suse.com
Signed-off-by: Petr Mladek <pmladek@suse.com>
|
|
On x86, an optkprobe at A replaces five bytes with a jump. If a disabled
probe B is at A+2, get_optimized_kprobe() stops at B when arming a new
probe C at A+4. It leaves A optimized:
A A+1 A+2 A+3 A+4
A's jump | e9 | d0 | d1 | d2 | d3 |
after C | e9 | d0 | d1 | d2 | cc |
The INT3 for C overwrites the last byte of A's jump displacement, so
execution can jump to the wrong address. B can have prepared optinsns
while disarmed, but has no jump to unoptimize.
Continue past disarmed and unprepared probes to find the active optimized
probe before arming a probe in its jump.
Link: https://lore.kernel.org/all/20260930053618.104498-1-leon.hwang@linux.dev/
Fixes: afd66255b9a4 ("kprobes: Introduce kprobes jump optimization")
Cc: stable@vger.kernel.org
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
Initially (and perhaps forever), call_srcu() will not be available for
atomic srcu_struct structures. There is therefore no reason to transition
such a structure to big, because the main purpose of such a transition
is to reduce lock contention for concurrent SRCU callback queueing.
This commit therefore adds an is_atomic parameter to both the
check_init_srcu_struct() and init_srcu_struct_fields() functions, which
suppresses the initialization-time transition to big that is enabled by
default on large systems.
It will still be possible to force a transition using rcutorture as a
destructive test. This might (or might not) be adjusted later.
[ paulmck: Apply feedback from Kunwu Chan. ]
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
The new SRCU_READ_FLAVOR_ATOMIC prohibits sleeping in both readers and
grace-period waits in order to permit waiting on an SRCU grace period
within an OOM notifier. This means that srcu_advance_state() cannot
acquire ->srcu_gp_mutex in this case, because mutex_lock() can sleep.
This commit therefore adds an is_atomic parameter to srcu_advance_state().
When this parameter is false, current behavior is maintained, in other
words, ->srcu_gp_mutex is acquired and released as before. But when this
is_atomic parameter is false, the caller is responsible for excluding
other callers.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
In non-preemptible kernels, a Tiny SRCU grace period implies an RCU
grace period because any context switch suffices. But in preemptible
kernels, it is possible for a Tiny SRCU grace period to elapse without
a corresponding RCU grace period.
Which was OK until RCU Tasks Trace was re-implemented in terms
of SRCU-fast, which in Tiny SRCU is implemented as SRCU, which is
already fast. And RCU Tasks Trace grace periods are required to imply
RCU grace periods. This commit therefore adds a synchronize_rcu(),
but only in preemptible kernels.
Because preemptible Tiny SRCU is not in mainline, this added call to
synchronize_rcu() will not slow anything down: The comparison would
instead be with TREE SRCU. But if this added call ever becomes a problem,
the Tiny SRCU srcu_struct structure could track whether or not this is for
SRCU-fast, and to add the synchronize_rcu() only in the SRCU-fast case.
However, at the moment, this is seen as unnecessary complexity.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
call_rcu() and call_srcu() are now safe to invoke from NMI, but
rcutorture never does, leaving the deferral path untested.
Add an ->nmi_capable flag to rcu_torture_ops. For flavors that set it,
arm a per-CPU hardware perf counter whose overflow handler submits a
callback via ->call(). The handler acts only when in_nmi(), so only a
genuine NMI exercises the deferral path. One preallocated callback per
CPU is kept in flight, guarded by an atomic, to avoid allocating in NMI.
The counter uses a fixed sample period rather than a frequency: a
frequency-based event sets TICK_DEP_BIT_PERF_EVENTS and would pin the tick
for the whole run on NO_HZ_FULL kernels.
Report the count issued from NMI ("nmi-calls:") and the count invoked
("nmi-cbs:"). rcu_torture_cleanup() disables the counters and then calls
cb_barrier(), which drains every deferred callback, so the two counts must
then match; a mismatch fails the test. This relies on
rcu_barrier()/srcu_barrier() flushing deferred callbacks, as added earlier
in the series.
Set ->nmi_capable on the NMI-safe flavors: rcu, srcu, srcud, and
tasks-tracing (call_srcu() under the hood). Tasks and Tasks Rude are left
alone, as call_rcu_tasks_generic() is not yet NMI-safe.
Enabled by default; the nmi_calls parameter disables it, which helps rule
NMI handling in or out when triaging a failure. Requires
CONFIG_PERF_EVENTS and a hardware PMU: without one nothing is issued from
NMI and the end-of-test check compares zero against zero, so a pass does
not by itself mean the path ran. That is the case under kvm.sh, which
boots qemu with -cpu kvm64 and no vPMU; rcu_torture_nmi_cleanup() says so
on the console.
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
Vanilla RCU readers, when either non-preemptible or preemptible with
priority-boosting enabled, can handle heavy overload conditions.
Other RCU implementations are more fragile. This commit adds an
rcu_torture_ops structure field named ->rdrs_handle_load that is set
for non-fragile configurations of vanilla RCU, but cleared otherwise.
It also refrains from running fragile readers concurrently with overload
testing, thus avoiding false-positive overload-testing failures.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
Give Tiny call_srcu() the same treatment as Tree SRCU. When interrupts
are disabled and the scheduler is up, stage the callback on the
srcu_struct's lockless list for an irq_work to re-issue later. Tiny SRCU
is uniprocessor, so there is no CPU-offline drain. A draining flag drops
a deferring call_srcu() that re-enters mid-drain (unless from an NMI), as
in Tree SRCU; such a drop can strand state the caller tied to the callback,
not just the callback itself.
Interrupts stay off for the whole batch. TINY_SRCU implies !SMP, where
arch_irq_work_has_interrupt() is false, so the drain always waits for the
tick and a batch is whatever one tick's worth of interrupts-disabled
call_srcu()s staged. Unlike the other three flavors srcu_do_enqueue() here
has no debug_rcu_head_queue(), so nothing reports a double call_srcu();
termination of the drain rests on srcu_do_enqueue() clearing ->next, and
the callback list self-links at the tail exactly as a double call_srcu()
made it before.
srcu_barrier() (now out of line) and cleanup_srcu_struct() drain the
deferred list first, so a deferred callback is re-issued onto the callback
list and invoked by the grace-period work that cleanup_srcu_struct()
flushes, rather than stranded on a soon-to-be-freed srcu_struct.
cleanup_srcu_struct() also syncs ->defer_iw, since that irq_work is
embedded in the srcu_struct the caller is about to free.
The draining flag is global rather than per-srcu_struct: a re-entrant
call_srcu(B) inside a drain of A raises B's own ->defer_iw, whose drain can
stage back onto A, so a per-srcu_struct flag would not break the chain.
The cost is that a drain of A also drops a non-NMI call_srcu() to any
other srcu_struct for its duration.
Gated by CONFIG_RCU_DEFER like Tree SRCU, though the srcu_struct members
are unconditional.
Suggested-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
call_srcu() has the same constraint as call_rcu():
srcu_gp_start_if_needed() enqueues under raw_spin_lock_irqsave() and may
walk the srcu_node tree, as do callback invocation and grace-period work,
so a call_srcu() with interrupts already disabled can race an operation in
flight on this CPU. call_rcu_tasks_trace() is call_srcu() under the hood,
so a sleepable BPF program freeing an object can reach this.
Defer as call_rcu() does: stage the callback on the srcu_data's
->defer_cbs, chain that srcu_data onto a per-CPU list, and raise a per-CPU
irq_work that re-issues it straight to the enqueue helper, never back
through __call_srcu(). The irqs-enabled path is unchanged; as for
call_rcu() the gate is bare irqs_disabled(), so callers that merely hold
interrupts off are deferred too and pay one irq_work hop, including
call_rcu_tasks_trace() from the BPF memalloc irq_work.
The irq_work is per-CPU rather than per-srcu_struct and statically
initialized, so deferral never runs check_init_srcu_struct(); it is
IRQ_WORK_INIT_HARD as for call_rcu(). srcu_barrier() flushes it first, and
rcutree_migrate_callbacks() calls srcu_offline_drain() for an outgoing CPU.
cleanup_srcu_struct() drains before its "just leak it" early returns, and
srcu_module_going() before freeing any ->sda, since a staged srcu_data left
chained on a per-CPU list would dangle. Staging is two steps, the callback
onto
->defer_cbs and then the srcu_data onto the per-CPU list, so a flusher can
find the per-CPU list empty while a callback whose call_srcu() has not
returned sits on ->defer_cbs; the staging CPU's own irq_work takes that
one.
The per-CPU srcu_defer ->lock, not any srcu_data's, is held with interrupts
off across the whole nested drain: the chain of srcu_datas staged on that
CPU and, for each, its callbacks, with srcu_do_enqueue() taking that
srcu_data's ->lock and possibly starting a grace period for every one.
That is what serializes the drainers. The bound is as for call_rcu(): what
one interrupts-disabled region could stage, normally a single callback, or
whatever accumulates before the tick where arch_irq_work_has_interrupt() is
false. Both lists are drained newest-first; nothing depends on call_srcu()
ordering.
The drain clears ->next before re-issuing, which bounds a node self-linked
by a double call_srcu(); a longer cycle is not bounded, and a double
call_srcu() stays undefined, as for call_rcu().
A callback deferred past the CPUHP_AP_SMPCFD_DYING irq_work flush leaves
that CPU's srcu_defer ->iw claimed with its self-IPI lost, as for
call_rcu(). srcu_offline_drain() still re-issues the callback, but the
irq_work cannot be un-queued, and here the claim is shared by every
srcu_struct on the CPU.
As in call_rcu(), the re-issue runs with interrupts disabled and can be
re-entered by instrumentation, so a per-CPU flag, set only while that CPU
is inside its own irq_work drain, drops a deferring call_srcu() seen
mid-drain unless it comes from an NMI. Such a drop can strand state the
caller associated with the callback, not just the callback itself.
Staging records only the callback, so a deferred expedited call_srcu()
completes as a normal grace period. Only srcu_expedite_current() can hit
that, and only when invoked with interrupts already disabled.
Gated by CONFIG_RCU_DEFER, though the srcu_data members and the per-CPU
srcu_defer are unconditional. Under CONFIG_PROVE_RCU, warn if the direct
path is reached from an NMI.
Suggested-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
Give Tiny call_rcu() the same treatment as Tree RCU. When interrupts are
disabled and the scheduler is up, stage the callback on a lockless list
that an irq_work re-issues later. One global list and irq_work suffice
since Tiny RCU is uniprocessor, and there is no CPU-offline drain.
The re-issue runs with interrupts disabled and can be re-entered by
instrumentation, so a draining flag drops a deferring call_rcu() seen
mid-drain (unless from an NMI), as in Tree RCU. Gated by CONFIG_RCU_DEFER,
though the deferral state is unconditional.
Interrupts stay off for the whole batch, but the re-issue is a tail append
with no locks. TINY_RCU implies !SMP, where arch_irq_work_has_interrupt()
is false, so the drain always waits for the tick and a batch is whatever
one tick's worth of interrupts-disabled call_rcu()s staged. As in Tree
RCU the drain clears ->next before re-issuing, which bounds a node
self-linked by a double call_rcu(): rcu_do_enqueue()'s duplicate path
returns without clearing it. A longer cycle is not bounded; a double
call_rcu() stays undefined.
The idle-task reschedule moves out of the enqueue helper so that a drain
does it once for the batch rather than once per callback, which would
otherwise take the runqueue lock N times with interrupts disabled.
Suggested-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
RCU's per-CPU callback list is only touched with interrupts disabled: the
enqueue runs under local_irq_save() (and the nocb locks when offloaded),
as do callback invocation and grace-period work. A call_rcu() that
arrives with interrupts already disabled, whether from an NMI or from
instrumentation that re-enters RCU, can interrupt one of those and corrupt
the list or deadlock.
Defer instead: stage the callback on a per-CPU llist and raise an irq_work
that re-issues it once interrupts are on, straight to the enqueue so it
cannot defer again. The gate is bare irqs_disabled(), so callers that
merely hold interrupts off are deferred too and pay one irq_work hop.
Skip it while the scheduler is down (RCU_SCHEDULER_INACTIVE): irq_work is
not usable that early, rcu_init() already calls call_rcu(), and the per-CPU
deferral state is not initialised until rcu_init_one() runs later in it.
rcu_barrier() drains every CPU's ->defer_head before it scans the lists,
and rcutree_migrate_callbacks() drains an outgoing CPU's. A drain
re-issues onto the draining CPU, so a barrier moves other CPUs' staged
callbacks onto
its own ->cblist; call_rcu() promises no CPU affinity for invocation.
->defer_lock is held across llist_del_all() and the whole re-issue so the
drainers
serialize: one that finds the list empty can conclude that everything
staged before it is already on a callback list. Interrupts stay off for
the batch. Where the arch has an irq_work self-IPI that is what one
interrupts-disabled region could stage, normally a single callback; where
arch_irq_work_has_interrupt() is false the drain waits for the tick, so
several regions can accumulate first.
The drain clears ->next before re-issuing. A double call_rcu() on a head
that is already debug-object-active self-links the staged node, and
rcu_do_enqueue()'s duplicate path returns without clearing it, so the
drain would spin. A re-add behind other staged callbacks makes a longer
cycle, which that does not bound; a double call_rcu() stays undefined.
llist_del_all() yields newest-first, so a batch is re-issued in reverse
call order; nothing depends on call_rcu() ordering. The re-issue drops
the lazy hint, since staging records only ->func, so a deferred callback
loses its batching on CONFIG_RCU_LAZY. kasan_record_aux_stack() moves to
__call_rcu_common() so a use-after-free report names the caller rather
than the irq_work.
The re-issue runs with interrupts disabled, so instrumentation on the
enqueue path can re-enter call_rcu(), stage another callback and re-raise
the irq_work, livelocking the drain. A per-CPU flag guards it: a deferral
that arrives while this CPU is draining, and is not from an NMI, is
dropped. The WARN_ONCE() is under CONFIG_PROVE_RCU, so a production kernel
drops it silently. That leaks the callback and can strand state
the caller tied to it, since a one-shot flag only the callback clears
never resets, but the alternative is an unbounded loop.
A callback deferred past the CPUHP_AP_SMPCFD_DYING irq_work flush leaves
->defer_work claimed with its self-IPI lost. rcutree_migrate_callbacks()
still re-issues the callback, but the first deferral after that CPU comes
back raises no IPI and waits for the next irq_work there, or for
rcu_barrier(). Unqueueing an irq_work is not something the API offers.
The irq_work is IRQ_WORK_INIT_HARD so the re-issue stays prompt on
PREEMPT_RT, where a non-HARD irq_work runs in a kthread that can be
delayed under load. A hidden CONFIG_RCU_DEFER gates the deferral code and
its IRQ_WORK dependency, though the rcu_data members are unconditional;
without it call_rcu() enqueues directly as before. Under
CONFIG_PROVE_RCU, warn if the direct path is reached from an NMI.
Suggested-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Puranjay Mohan <puranjay@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
|
|
The WARN_ON() added by commit 78a38cbf6f20 ("srcu: Queue sdp->work
when the delay timer is successfully deleted") uses rcu_segcblist_n_cbs()
to detect callbacks that srcu_barrier() failed to wait for. However, the
->len counter is decremented only at the end of srcu_invoke_callbacks(),
after the invoking loop has finished. Since srcu_barrier() can return
right after the barrier callback is invoked, cleanup_srcu_struct() can see
a non-zero n_cbs even though the cblist is already physically empty,
falsely triggering the WARN_ON() together with a still-pending delay_work
timer.
This can be triggered as follows, as seen in the syzbot report against
kvm_destroy_vm() -> cleanup_srcu_struct(&kvm->srcu):
1. call_srcu(&kvm->srcu, &bus->rcu, __free_bus) starts SRCU grace
period GP1.
2. GP1 ends: a delay timer is armed and sdp->work is queued, but
sdp->work has not run yet.
3. Another call_srcu(&kvm->srcu, &bus->rcu, __free_bus) call invokes
srcu_segcblist_advance(), which moves the GP1 callback to
RCU_DONE_TAIL, and starts SRCU grace period GP2.
4. srcu_barrier() is called. It queues its barrier callback after the
GP2 callback and waits for srcu_invoke_callbacks() to invoke it.
5. GP2 ends: another delay timer is armed, and the sdp->work queued in
step 2 begins to run. Its srcu_invoke_callbacks() call invokes
srcu_segcblist_advance() again, moving the GP2 and barrier callbacks
to RCU_DONE_TAIL as well, and then invokes all of them. However,
rcu_segcblist_add_len(), which updates srcu_cblist's ->len, has not
run yet at this point.
6. srcu_barrier() returns once its callback has been invoked, and
cleanup_srcu_struct() starts running. It finds the delay timer
armed in step 5 still pending and srcu_cblist's ->len still
non-zero (because step 5 has not reached rcu_segcblist_add_len()
yet), and WARN_ON() fires even though every callback has actually
been invoked.
Use rcu_segcblist_empty(), which checks the actual head of the cblist,
instead of rcu_segcblist_n_cbs(), which checks the racy ->len counter.
Callbacks that have genuinely not been invoked yet still leave the list
non-empty, so the WARN_ON() still catches callers that skip srcu_barrier()
or queue callbacks after it.
Link: https://lore.kernel.org/rcu/e6350377085ddd85d6ef00d8e9a67bd50c762d3c@linux.dev/T/#t
Reported-by: syzbot+d4faf7db59e11f6fd1ab@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=d4faf7db59e11f6fd1ab
Fixes: 78a38cbf6f20 ("srcu: Queue sdp->work when the delay timer is successfully deleted")
Suggested-by: Zqiang <qiang.zhang@linux.dev>
Signed-off-by: Sunho Park <shpark061104@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Tested-by: Sean Christopherson <seanjc@google.com>
|
|
|
|
cgroup_e_css() and cgroup_get_e_css() find the effective css by walking
up the hierarchy until they reach a cgroup that has the subsystem
enabled. Per-I/O users such as nvmet need this lookup to be O(1).
Track the effective css in each cgroup and make cgroup_e_css() and
cgroup_get_e_css() use it. When a css is brought online or offline,
update the effective css of its cgroup and of all descendants.
Signed-off-by: Peng Yu <yupeng0921@gmail.com>
Assisted-by: Claude:claude-fable-5 [Claude Code]
Assisted-by: Claude:claude-opus-5-5 [Claude Code]
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
do_call_rcu_ttrace() returns early when call_rcu_ttrace_in_progress is set
and leaves the objects in free_by_rcu_ttrace. __free_rcu() frees
waiting_for_gp_ttrace only and clears the flag. Hence the objects that
free_bulk() or __free_by_rcu() added while RCU tasks trace GP was in flight
stay in free_by_rcu_ttrace until free_bulk() or alloc_bulk() is called for
the same bpf_mem_cache again, which may never happen. The number of such
objects is not bounded.
Turn call_rcu_ttrace_in_progress into three states:
0 - idle
1 - __free_rcu() is queued
2 - __free_rcu() is queued and free_by_rcu_ttrace got more objects since
do_call_rcu_ttrace() sets 2. __free_rcu() does cmpxchg(1 -> 0) and starts
the next GP when it fails. It cannot clear the flag first and check
free_by_rcu_ttrace later, since bpf_mem_alloc_destroy() frees bpf_mem_cache
without waiting for RCU callbacks when the flag is zero.
Now __free_rcu() queues itself, so the one that didn't see 'draining' may
do call_rcu_tasks_trace() after rcu_barrier_tasks_trace() in
free_mem_alloc(). Queue it under rcu_read_lock() and do synchronize_rcu()
before the barriers. Calling rcu_barrier_tasks_trace() twice works too, but
creating and destroying hash maps in a loop on many cpus slows down to one
free_mem_alloc() per GP and kworkers pile up.
Fixes: 8d5a8011b35d ("bpf: Batch call_rcu callbacks instead of SLAB_TYPESAFE_BY_RCU.")
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20260930095920.601738-3-alexei.starovoitov@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Move the part of do_call_rcu_ttrace() that runs after
call_rcu_ttrace_in_progress is set into __do_call_rcu_ttrace().
The next patch will call it from __free_rcu().
No functional change.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20260930095920.601738-2-alexei.starovoitov@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Since commit 022ac0750883 ("bpf: use reg->var_off instead of reg->off
for pointers"), find_good_pkt_pointers() sets the range of all packet
pointers sharing an id from the umax of the compared pointer, and
check_packet_access() requires umax + off + size <= range. That assumes
the umax of two such pointers differ by exactly their constant distance.
reg_bounds_sync() breaks it when var_off tightens one umax and not the
other:
r4 &= 0x38
if r4 > 50 goto exit ; umax 50, var_off (0x0; 0x38)
r5 = pkt + r4 ; umax 50
r6 = r5
r6 += 8 ; umax 56, not 58
Comparing r6 with pkt_end sets the range to 56, and the valid 8-byte
load at r5 is rejected (50 + 8 > 56). Comparing r5 sets it to 50, and
the out-of-bounds 1-byte load at r6 - 7, i.e. r5 + 1, is accepted
(56 - 7 + 1 <= 50).
Don't call reg_bounds_sync() on a packet pointer that keeps its id (a
constant was added or subtracted) or its range (an unknown non-negative
value was subtracted), so that var_off cannot tighten its umax. Only
update the 32-bit bounds from var_off: reg_bounds_sanity_check() wants
them constant when the lower half of var_off is, e.g. for pkt + 8.
This relies on nothing else changing the 64-bit bounds of a packet
pointer, which holds today.
var_off of such a pointer is no longer narrowed by its bounds. Adjust
three verifier_align expectations; the low bits, which the alignment
checks use, don't change. veristat on the selftests shows no verdict
changes and +0.8% insns in test_cls_redirect_subprogs.
Fixes: 022ac0750883 ("bpf: use reg->var_off instead of reg->off for pointers")
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20261001145255.855630-1-alexei.starovoitov@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Two things in __cgroup_bpf_query() work only because CGROUP_TCP_SOCK_OPS is
the only struct_ops attach type.
It calls cgroup_bpf_enabled(atype) with an atype that
find_atype_by_struct_ops_id() works out at runtime. That macro is an asm
goto and needs a constant. Today the compiler can see there is only one
value; add a second type and the build breaks with "impossible constraint
in 'asm'". The check is not needed anyway. When the key is off nothing is
attached, so progs[atype] and effective[atype] are empty and the count loop
leaves total_cnt at 0. The other attach types already use that loop with
no such check. Drop the check and the skip_count label.
And find_atype_by_struct_ops_id() matches on type_id alone. An attach type
whose subsystem is not built keeps type_id 0, so a query for type 0 finds
it and returns success with nothing instead of -ENOENT. Skip such slots.
Fixes: 369d9dcd8fb8 ("bpf: Add infrastructure to support attaching struct_ops to cgroups")
Signed-off-by: Shakeel Butt <shakeel.butt@linux.dev>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Reviewed-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Yafang Shao <laoar.shao@gmail.com>
Acked-by: Tejun Heo <tj@kernel.org>
Link: https://patch.msgid.link/20261001141901.3225830-1-shakeel.butt@linux.dev
|
|
linux/module.h appears in roughly 15k #include directives across the
kernel. This makes it a "hot" header, so it should avoid pulling in
unnecessary definitions.
The header currently includes linux/error-injection.h to obtain the
definition of `struct error_injection_entry`. However, this is unnecessary
because the type is only referenced in the file as a pointer, for which an
incomplete type is sufficient.
Remove the linux/error-injection.h include from linux/module.h and add it
to kernel/module/main.c instead, where
`sizeof(struct error_injection_entry)` is actually needed.
Reviewed-by: Aaron Tomlin <atomlin@atomlin.com>
Signed-off-by: Petr Pavlu <petr.pavlu@suse.com>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/sysctl/sysctl
Pull sysctl fixes from Joel Granados:
"Fix sysctl jiffies conversions errors introduced in 2dc164a48e6f
("sysctl: Create converter functions with two new macros")
- Ensure that mult_hz does *not* wrap
- Ensure we pass just the magnitude for the negative branch in
proc_int_k2u_conv_kop"
* tag 'sysctl-7.03-fixes-rc6' of git://git.kernel.org/pub/scm/linux/kernel/git/sysctl/sysctl:
time/jiffies: Saturate in mult_hz() instead of wrapping
sysctl: Negate before converting in the int read path
|
|
Active references on the namespaces of an nsproxy are taken when the
nsproxy is installed into a task in switch_task_namespaces() and
copy_namespaces() and dropped again by put_nsproxy() through
deactivate_nsproxy().
But ksys_unshare() calls put_nsproxy() on an nsproxy that was never
installed when set_cred_ucounts() fails. The new namespaces of that
nsproxy go from zero to minus one and the namespaces shared with the
caller lose a reference that belongs to the nsproxy the caller keeps
using:
WARNING: kernel/nscommon.c:171 at __ns_ref_active_put+0x1cd/0x230
nsproxy_ns_active_put
deactivate_nsproxy
ksys_unshare
Afterwards the namespaces the caller lives in aren't listed by listns()
anymore. set_cred_ucounts() only fails when alloc_ucounts() can't
allocate and it only allocates when the real uid of the caller differs
from its effective uid.
Commit cefd55bd2159 ("nsproxy: fix free_nsproxy() and simplify
create_new_namespaces()") separated the two cases on purpose.
nsproxy_free() frees an nsproxy that was prepared but never installed
and that's what a failed setns() uses in put_nsset(). Export it and use
it for a failed unshare() as well.
Fixes: a98621a0f187 ("unshare: fix nsproxy leak in ksys_unshare() on set_cred_ucounts() failure")
Cc: stable@vger.kernel.org # v7.1+
Link: https://patch.msgid.link/20260930-work-mount-fixes-3-v1-16-be34c83956ae@kernel.org
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Perf events with exclude_kernel=1 can be opened without kernel perf
access. However, exclude_kernel does not suppress text-poke sideband
records. Every PERF_RECORD_TEXT_POKE is marked PERF_RECORD_MISC_KERNEL
and contains a raw kernel instruction address.
An unprivileged task can therefore open and mmap a task-local software
event with text_poke=1. Both opening a count-only tracepoint event and
configuring UDP GRO for ESP-in-UDP cause updates to inline static calls;
the observer receives the relocated addresses of the modified instructions.
For a known kernel image, any such address reveals the runtime kernel
text base despite KASLR.
Call perf_allow_kernel() whenever attr.text_poke is set, regardless of
exclude_kernel. Events that neither monitor kernel execution nor request
text-poke records retain their existing permissions.
Fixes: e17d43b93e54 ("perf: Add perf text poke event")
Assisted-by: LLM
Signed-off-by: Zhengchuan Liang <zcliangcn@gmail.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/99131354c41e23188f778b92f90363775b482395.1790573390.git.zcliangcn@gmail.com
|
|
perf_iterate_sb() invokes its callback for each matching perf_event on
the CPU and task context, passing a shared caller-allocated event
structure.
perf_event_header__init_id() mutated header->size in place by adding
event->id_header_size, requiring sideband output callbacks to save and
restore header fields across iterations. Three sideband callbacks failed
to save and restore header.size around perf_event_header__init_id():
- perf_event_ksymbol_output()
- perf_event_bpf_output()
- perf_event_text_poke_output()
When multiple events with attr.ksymbol, attr.bpf_event, or
attr.text_poke and sample_id_all are active on the same CPU, each
subsequent event receives a record whose header.size is inflated by all
preceding events' id_header_size values while only a single id_sample is
written, leaving uninitialized ring-buffer bytes at the end of the
record and causing userspace perf to fail with -EFAULT ("Bad address")
when parsing the sample_id trailer.
Similarly, perf_event_mmap_output() set PERF_RECORD_MISC_MMAP_BUILD_ID
in mmap_event->event_id.header.misc when event->attr.build_id was
enabled, but only saved and restored header.size and header.type. If an
event with attr.build_id was followed by an event with attr.mmap2 and
!attr.build_id, the second event received PERF_RECORD_MISC_MMAP_BUILD_ID
in header.misc while its payload contained maj/min/ino/ino_generation
instead of a build ID.
Rather than splitting header initialization between callers and output
callbacks and saving/restoring mutated header fields, replace
perf_event_header__init_id() with perf_event_header__init(), which
initializes header->type, header->misc, and header->size alongside the
sample_id fields on each invocation.
Fixes: 76193a94522f ("perf, bpf: Introduce PERF_RECORD_KSYMBOL")
Fixes: 6ee52e2a3fe4 ("perf, bpf: Introduce PERF_RECORD_BPF_EVENT")
Fixes: e17d43b93e54 ("perf: Add perf text poke event")
Fixes: 88a16a130933 ("perf: Add build id data in mmap2 event")
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260929222332.973435-1-irogers@google.com
Cc: stable@vger.kernel.org
|
|
A race condition exists between perf_event_exit_task() and
perf_pending_task() during begin_new_exec().
During begin_new_exec(), perf_event_exit_task() may be called, and the
PF_EXITING flag is not set on task. So perf_sigtrap() continues to execute
and triggers WARN_ON_ONCE(event->ctx->task != current).
Since both task exit and exec paths can call perf_event_exit_task() which
sets ctx->task to TASK_TOMBSTONE, fix this by explicitly checking if
event->ctx->task equals TASK_TOMBSTONE and dropping the redundant
PF_EXITING check.
Fixes: 97ba62b27867 ("perf: Add support for SIGTRAP on perf events")
Signed-off-by: Luo Gengkun <luogengkun2@huawei.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260920075026.990582-1-luogengkun2@huawei.com
|
|
Once a smpboot thread gets work assigned it receives a wake up. If the
thread had no chance to run and receives a parking requesst during a
CPU-hotplug event then it is parked without running the callback first.
As a result the enqueued work remains stuck until the CPU gets back
online.
There is nothing wrong with invoking the thread function first and
parking in the following iteration. This would ensure that the callbacks
are processed before the shutdown.
Honor the park request once thread function does not need to run.
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260911143815.997254-4-bigeasy@linutronix.de
|
|
PREEMPT_RT invokes IRQ_WORK_LAZY callbacks and callbacks which are not
explicitly marked IRQ_WORK_HARD_IRQ from thread context. If a CPU gets
shutdown, the irq_work is flushed during smpcfd_dying_cpu() which is
invoked on the target CPU with disabled interrupts.
The callbacks enqueued on lazy_list on PREEMPT_RT are not flushed because
they require thread context. The callbacks remain and get processed once
the CPU gets back online.
per-CPU irq_work items continue to work for enqueues on their local-CPU,
only the work on the "offline" CPU get stuck. "Single" irq_work are
worse because they remain "claimed" and can not be used again. One
"symptom" is the late printk() in CPU down path which enqueues the
irq_work for printing but it never gets scheduled. Further prints don't
print to the console because the irq_work is "pending" of the offline
CPU.
Another problem is the canceling/ flushing of the irq_work which is
stuck. Since the irq_work will not continue, the task waiting for its
completion will block waiting.
While looking through the users of irq_work, it does not matter if the
callback is enqueued on another CPU. This makes it possible to invoke
the callbacks from a remote CPU by smpcfd_dead_cpu(). This is invoked
from the control CPU with enabled interrupts shortly after the
hotplugged CPU is shutdown so the llist can be accessed safely.
Add irq_work_run_cpu() which invokes the irq_work callbacks of the
specified dead CPU. Invoke it from smpcfd_dead_cpu().
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260911143815.997254-3-bigeasy@linutronix.de
|
|
hotplug_cfd() no longer exists and the flow changed a bit since the
rework of CPU hotplug as of commit 31487f8328f20 ("smp/cfd: Convert core
to hotplug state machine").
Update the comment to reflect reality as of today.
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://patch.msgid.link/20260911143815.997254-2-bigeasy@linutronix.de
|
|
Architectures can use SD_ASYM_PACKING to describe preferred CPU ordering
at the SMT scheduling domain. Some systems benefit from this policy, but
their firmware cannot describe the preference and inferring it from the
CPU model would embed a platform-specific policy in the kernel.
Add sched_smt_asym_packing=on boot option to force SD_ASYM_PACKING at
the SMT level. Omitting the option preserves the topology provided by
the architecture or firmware.
Apply the override centrally to domains with SD_SHARE_CPUCAPACITY so it
also covers architectures with custom SMT topology callbacks, including
powerpc.
When enabled, priority remains defined by arch_asym_cpu_priority(). The
weak default prefers lower-numbered logical CPUs, while architecture
overrides remain authoritative. Siblings with equal priorities remain
unordered. In particular, x86 with CONFIG_SCHED_MC_PRIO normally gives
both SMT siblings the same core priority, so enabling the option there
does not prioritize a sibling.
Signed-off-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Tested-by: Breno Leitao <leitao@debian.org>
Link: https://patch.msgid.link/20260929165538.726616-3-arighi@nvidia.com
|
|
POWER7 uses SD_ASYM_PACKING at the shared-capacity SMT level to order
hardware threads, and NVIDIA Olympus benefits from the same policy. Idle
CPU selection does not consult that order, so a task can wake on an
arbitrary sibling and remain there until load balancing corrects the
placement. On these systems, that initial choice can prevent the core
from entering its preferred lower-thread resource mode and cause a large
and persistent performance loss.
When idle selection finds an available CPU in an SMT core, choose the
highest-priority available sibling. On SMT2 Olympus this only changes
selection on fully idle cores. A partially idle core has only one
available CPU. On wider SMT systems such as POWER7, it also fills
available siblings in priority order while the core is partially busy.
Apply the preference to idle-core and idle-CPU scans,
asymmetric-capacity scans, target, previous, recently-used CPU fast
paths and the slow path. Inspect the lowest scheduling domain directly,
but require both CPUs to share its span because isolcpus can split
hardware siblings across scheduling domains.
Keep physical-core capacity selection independent from SMT sibling
ordering. SD_ASYM_CPUCAPACITY first selects among cores with different
maximum capacities, then SD_ASYM_PACKING selects the preferred available
sibling inside the chosen core, whose siblings continue to share equal
capacity.
Signed-off-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Srikar Dronamraju <srikar@linux.ibm.com>
Reviewed-by: K Prateek Nayak <kprateek.nayak@amd.com>
Reviewed-by: Vincent Guittot <vincent.guittot@linaro.org>
Reviewed-by: Kayra Cizmeci <kayracizmeci@gmail.com>
Tested-by: K Prateek Nayak <kprateek.nayak@amd.com>
Tested-by: Kayra Cizmeci <kayracizmeci@gmail.com>
Tested-by: Breno Leitao <leitao@debian.org>
Link: https://patch.msgid.link/20260929165538.726616-2-arighi@nvidia.com
|
|
set_tsk_need_resched() tests TIF_NEED_RESCHED, calls
__trace_set_need_resched() if the flag is clear, then sets it via
set_tsk_thread_flag(). A BPF raw_tp program attached to
sched_set_need_resched executes synchronously inside __bpf_trace_run().
On return, __bpf_trace_run() drops the RCU lock with
rcu_read_unlock_migrate(), which on the preempt-or-BH-disabled path
calls set_need_resched_current() -> set_tsk_need_resched() again.
set_tsk_thread_flag() follows the tracepoint call, so every re-entrant
frame sees TIF_NEED_RESCHED clear and calls __trace_set_need_resched()
again:
BUG: TASK stack guard page was hit at ffffc9001224ff98
Oops: stack guard page: 0000 [#1] SMP KASAN PTI
RIP: 0010:__bpf_trace_sched_set_need_resched_tp+0x1c/0x190
Call Trace:
trace_sched_set_need_resched_tp+0x110/0x130
set_tsk_need_resched include/linux/sched.h:2076
set_need_resched_current include/linux/sched.h:2094
rcu_read_unlock_special+0x43a/0x440
__rcu_read_unlock+0x9e/0x120
rcu_read_unlock_migrate+0xa9/0x240
__bpf_trace_run+0x131/0x180
bpf_trace_run3+0x333/0x430
__bpf_trace_sched_set_need_resched_tp+0x13a/0x190
trace_sched_set_need_resched_tp+0x110/0x130
set_tsk_need_resched include/linux/sched.h:2076
...
__resched_curr() has the same ordering, firing the tracepoint before
setting the flag via set_ti_thread_flag() or set_nr_and_not_polling().
Fix it for consistency.
Replace the separate test_tsk_thread_flag() + set_tsk_thread_flag() pair
in set_tsk_need_resched() with test_and_set_tsk_thread_flag(). In
__resched_curr(), move the tracepoint call after the flag is set in
each path.
Fixes: adcc3bfa8806 ("sched: Adapt sched tracepoints for RV task model")
Signed-off-by: Sechang Lim <rhkrqnwk98@gmail.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Acked-by: Gabriele Monaco <gmonaco@redhat.com>
Link: https://patch.msgid.link/20260630084750.2792851-1-rhkrqnwk98@gmail.com
|
|
Nathan reported build failure with clang23 when CONFIG_PREFERRED_CPU=y.
kernel/sched/core.c:11298:3: error: releasing raw_spinlock 'rq_lockp(rq)' that was not held [-Werror,-Wthread-safety-analysis]
11298 | rq_unlock(rq, &rf);
| ^
kernel/sched/core.c:11303:1: error: raw_spinlock 'rq_lockp(__this_rq())' is not held on every path through here [-Werror,-Wthread-safety-analysis]
context_unsafe_alias(rq) was placed after rq_lock(), by which point it had
already associated the lock with the original rq alias.
Move the alias barrier before rq_lock() so that the changing rq alias is
accounted for before acquiring the lock.
Fixes: 74699f56ebcf ("sched/core: Push current task from non preferred CPU")
Closes: https://lore.kernel.org/all/20260929121838.GA1814129@ax162/
Reported-by: Nathan Chancellor <nathan@kernel.org>
Signed-off-by: Shrikanth Hegde <sshegde@linux.ibm.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Tested-by: Nathan Chancellor <nathan@kernel.org> # build
Link: https://patch.msgid.link/20260929164712.1054883-1-sshegde@linux.ibm.com
|
|
ssh://gitolite.kernel.org/pub/scm/linux/kernel/git/jic23/iio into char-misc-next
Jonathan writes:
IIO: 1st set of device support, features and cleanup for 7.4
2 merges
- v7.3-rc4 to get a x86 build fix needed by the ad9910 driver.
- devm_notifier_chain_register to pick up the IIO specific users of this
new infrastructure.
New device support
------------------
Major refactors or new drivers needed:
adi,ad3530r
- Support for the AD5710R and AD5711R 8 channel IDAC / VDAC parts.
Needed substantial driver rework to support channel type controls.
adi,ad5529r
- New driver for this 16 channel DAC with 12 and 16 bit variants.
adi,ad7768
- New driver for the 8 (AD7768) and 4 (AD7768-4) channel variants of this
24-bit simultaneous sampling ADC family. They are very different from
the 1 channel variant that has as separate driver.
adi,ad9910
- New driver for this DDS (Direct Digital Synthesizer). This was a large
undertaking is it is a complex chip requiring quite a lot of new IIO ABI.
Delayed from last cycle by an x86 bug that was fixed in rc4.
adi,ade9000
- Add support for the ADE9078 polyphase energy metering device.
Required significant changes to introduce multiple part support into
this driver.
adi,adis16201
- Add support for the ADIS16203 device, replacing the driver in staging.
axiado,saradc
- New driver for the SARADC found on the AX3000 and ADX3005 SoCs.
liteon,ltr501
- Add support for the LTR-329ALS-01 ambient light sensor. Included
various bits of driver modernization.
microchip,mcp48fev02
- Split previously I2C specific mcp47feb02 into core and I2C specific
parts.
- Add an SPI specific part for the many parts in the MCP48FxBy1/2/4/8
series of SPI DACS 24 different part numbers covering:
- 8, 10 and 12 bit resolutions
- 1, 2, 4, and 8 channels
- EEPROM and no non volatile storage variants.
- Substantial driver modification and cleanup was needed prior to the new
support.
maxim,max40080
- New driver for this current-sense amplifier with integrated ADC.
renesas,rzg2l_adc
- Support the RZ/G3L ADC 1 that is dedicate to the on-chip thermal
sensor unit. The thermal parts of this support will be going through
that tree.
sensortek,stk3310
- Add support for the STK36C61 Ambient Light, proximity and RGB color
sensor. Handling had to be added for RGB elements and the driver in
general had to be made ready for support multiple parts.
ti,ads1100
- Add support for the ADS1110 with faster data rate and 2.048V reference.
ti,ads112c04
- New driver to support this 16bit sigma-delta ADC.
vishay,veml6031x00
- New driver for this ambient light sensor that was built in in stages.
- Add triggered buffer data capture.
- Add support for events and triggers.
- Enable mysterious 'internal calibration' setting.
Minor additions such as IDs, compatibles or device specific data.
adi,ad3530r
- Support the AD3536R which is low resolution equivalent of the already
supported AD3532R.
adi,ad4080
- Support the AD4885 - only needed chip specific data.
aosong,am2315
- Add support for the AM2320, fully compatible wth the AM2315.
invensense,icm42600
- Minor tweaks + addition of compatible for the ICM42630 IMU.
mediatek,mt2701-adc
- Compatible for the MT6572
rockchip,saradc
- Support the RV1106.
Features
--------
IIO Core
- Add a helper to discover if timestamps are enabled. In a few cases
(typically when there is batching involved) we may have to refuse
to start the buffer if timestamps are enabled. Provided a helper
to allow that to be queried.
IIO backend
- CRC support.
infineon,dps310
- Add triggered buffer support.
linear,ltc2497
- Add 2x conversion speed mode.
- Add the internal temperature channel.
renesas,r9a09g077
- Add missing DMA properties to DT and enable use of DMA
- Expose the sampling frequency as per channel attributes.
ti,ad1015
- Add per channel labels from DT support.
ti,ads112c14
- Add dataready trigger using continuous mode if only one channel is
enabled.
- Add burnout current control including new IIO ABI.
- Add filter control.
- Add new ABI for settling time.
Cleanup and minor or late breaking fixes
----------------------------------------
Only calling out the more significant changes or ones that were carried
on out on several drivers.
multiple drivers
- Add sanity check for spi_device_get_match_data() returning NULL due
to use of driver_override.
- Add some useful __counted_by_ptr markings.
- Add some missing MODULE_DEVICE_TABLE() calls.
- Use dmaenginge_get_dma_device() instead of chan->device->dev
- A few precursor cleanups for planed devm_ object allocators.
- Disable autosuspend on exit. Note there is work ongoing to make
this unnecessary.
- Header reorders and IWYU being applied to what was included.
- Use devm_blocking_notifier_chain_register() to replace open coded
equivalent.
- Ensure some clk related structures are always fully initialized.
- Use the size checked iio_push_to_buffers_with_ts() replacement for
iio_push_to_buffers_with_timestamp().
- Avoid shadowing various error codes from store() callbacks.
adi,axi-adc
- Fix missing mutex_init()
adi,ad4080
- Ensure autoincrement is set for multi register reads.
adi,ad5758
- Fix wrong offset calculation
- Reject out of range values rather than truncating values.
- Make sure buffer used for bulk acceses is DMA safe.
adi,ad7816
- Local lock to protect state by serializing accesses. A few extra locations
were added much later in the cycle.
- Stop using non DMA safe buffers by using spi_write_then_read()
to ensure the data was bounce buffered.
adi,adxl367
- Ensure interrupt mapping is resolved before device setup reducing work
done before potential deferral.
- Support INT2 pin.
asahi-kasei,ak8975
- Use BIT() and GENMASK() to improve readability of defines.
- Add an enum for scan mask elements.
- Switch to devm_ for all resources.
avago,apds9306
- Fix the default sampling frequency.
awinc,aw96103
- Fix an early return that stopped all channels being checked.
broadcom,iproc-adc
- General modernization and conversion to devm managed cleanup.
- Use dev_err_probe(), local dev pointer and drop some duplicate error
messages.
capella,cm32181
- Add device specific dt-binding as trivial-devices doesn't provide
vdd-supply or interrupt properties.
freescale,mma8452
- Modernize driver via guard(), IIO specific cleanup helpers and
bulk regulator handling.
infineon,dps310
- Add context analysis lock markings.
- Single local acquire / release per raw read.
- Replace opencoded get_unaligned_be24().
- Fix incorrect CFG_REG bit definitions.
invensense,icm42600
- Ensure ODR is updated when switching power mode to paper over a hardware
bug.
- Fix detecting ODR change in invalid data packets.
- Optimize high frequency data readback by not bothering to read the
FIFO count when the watermark is known to have been passed.
- Various other minor fixes.
isil,isl29028
- Fix a runtime pm reference leak.
liteon,ltr501
- Make the dt-binding property proximit-near-level only available
on parts that support it.
kionix,kx022a
- Use as structure to make the channel layout explicit and use that for
all cases (previously there were two similar buffers).
pulsed-light,lidar-lite-v2
- Move binding from trivial-devices to it's own file as there are power
enable and mode control pins that needs describing.
renesas,r9a09g077
- Drop some interrupt descriptions for interrupts that aren't connected to
the interrupt controller.
- Avoid direct reads when buffer enabled.
- Store the device PA and IRQ in state ready for the DMA support.
st,hts221
- Fix a potential division by zero if calibration data corrupted.
- Allow an unknown Who Am I value to support fallback dt-compatibles.
- Replace opencoded oversampling_ration_available handling with read_avail
callback.
st,stm32-adc
- Fix up some wrong minimum sample times.
ti,ads1100
- Fix readings after datarate change by ensuring a long enough wait for
a new sample.
ti,tmp117
- Fix case where cached state was updated before we know if the write
to the device succeeds.
vishay,veml3328
- Reshape the scale array to improve readability as it is made up
of value pairs.
* tag 'iio-for-7.4a' of ssh://gitolite.kernel.org/pub/scm/linux/kernel/git/jic23/iio: (270 commits)
iio: dac: ad5758: Fix the offset calculation
iio: dac: ad5758: Reject out-of-range raw values
iio: dac: ad5758: Fix alignment for DMA safety
iio: pressure: dps310: check the lock markings with context analysis
iio: core: add an accessor for scan_timestamp
iio: pressure: dps310: add triggered buffer support
iio: pressure: dps310: take the lock once per raw read
iio: pressure: dps310: use get_unaligned_be24() for the 24-bit results
iio: pressure: dps310: use a local device pointer in probe
iio: pressure: dps310: fix CFG_REG bit definitions
staging: iio: adc: ad7816: Protect sysfs attributes with mutex
dt-bindings: iio: Add dedicated CM32181 schema
iio: adc: stm32: fix minimum sampling time values
iio: light: veml6031x00: Enable internal calibration
iio: dac: add support for Microchip MCP48FEB02
dt-bindings: iio: dac: add support for MCP48FEB02 SPI
iio: dac: mcp47feb02: refactor MCP47FEB02 I2C driver into two modules
iio: dac: mcp47feb02: protect EEPROM store sequence with mutex
iio: dac: mcp47feb02: Avoid unjustified probe error on missing label
iio: adc: rzg2l_adc: Add RZ/G3L ADC support for TSU
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/pcmoore/audit
Pull audit fix from Paul Moore:
"A single audit fix for a potential UAF error in some audit filter
configurations"
* tag 'audit-pr-20260930' of git://git.kernel.org/pub/scm/linux/kernel/git/pcmoore/audit:
audit: fix exe mark UAF in kill_rules()
|
|
|
|
cpuset_update_flag() is only used by the legacy (v1) interface: its
callers are the write handlers of cpuset1_files via cpuset_write_u64(),
and cpuset_css_offline() which invokes it only on the legacy hierarchy
(!cpuset_v2() && is_sched_load_balance()).
Move it into cpuset-v1.c, where its sole helper
cpuset1_update_tasks_flags() already lives, so both can go back to being
static and drop out of cpuset-internal.h. As a result:
- dup_or_alloc_cpuset(), free_cpuset() and validate_change() are now
referenced from cpuset-v1.c, so they lose their static qualifier and
gain declarations in cpuset-internal.h.
- The v2 branch in cpuset_update_flag() that called
cpuset_force_rebuild() is dead on the v1-only call paths and is
dropped; the remaining rebuild_sched_domains_locked() path is kept.
callback_lock, a static in cpuset.c, is taken through the existing
cpuset_callback_lock_irq() wrapper.
- The open-coded "!cpuset_v2() && is_sched_load_balance()" test in
cpuset_css_offline() is wrapped in a new cpuset1_offline_css() helper
(a no-op stub when CONFIG_CPUSETS_V1=n), removing the last
cpuset_v2() branch from that path in cpuset.c.
No functional change intended.
Assisted-by: LLM
Signed-off-by: Ridong Chen <chenridong@xiaomi.com>
Reviewed-by: Waiman Long <longman@redhat.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
Hierarchical BPF schedulers may need to know how many CPUs a cgroup can
run on when distributing group-wide weights.
For example, fair's group share calculation scales a group's weight by
the smaller of its estimated runnable task count and the number of CPUs
in its effective cpuset. Without that count, a BPF scheduler can only
use the machine-wide CPU count and may give a group confined to a few
CPUs excessive shares.
A task's effective affinity can include per-task restrictions, so it
cannot reliably represent the group's cpuset. Reading the cpuset
directly from BPF would require CO-RE access to private cgroup and
cpuset structures, including inherited cpuset resolution, making the
approach fragile across kernel versions.
Add scx_bpf_cgroup_nr_cpus(), which returns the same count fair uses
through cpuset_num_cpus(). The count is independent of CPU or CID
numbering, so the kfunc is available to both cpu-form and cid-form
schedulers from any context. Also add a compat wrapper which falls
back to nr_cpu_ids on older kernels.
Signed-off-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
Pull to receive cc07fae1de5d ("selftests/sched_ext: Add a test for
ops.dequeue() on remote local DSQ moves") and resolve its conflict with
f73ec6791424 ("sched_ext: Add selftest for blocked donor admission") here
so that for-next merges clean.
Conflicts: kept both auto-test-targets entries in the selftests Makefile,
where each side added one after dequeue_iter.
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
When a task in the BPF scheduler's custody is moved to another CPU's
local DSQ, ops.dequeue() is only called once the task is picked for
execution, from set_next_task_scx() with SCX_DEQ_CORE_SCHED_EXEC, even
though no core-sched pick took place. It should instead be called
without flags when the task is inserted into the destination DSQ, as it
is for same-rq dispatches.
move_remote_task_to_local_dsq() sets p->scx.sticky_cpu across the
migration so that deactivate_task() on the source rq doesn't end custody.
Since commit b75aaea24c9f ("sched_ext: Properly mark SCX-internal
migrations via sticky_cpu"), enqueue_task_scx() only clears it after
inserting the task, so task_leave_custody() still sees the migration in
progress and skips the custody exit.
Clear p->scx.sticky_cpu as soon as enqueue_task_scx() has read it, as
was done before that commit. The source side is unaffected.
Fixes: ebf1ccff79c4 ("sched_ext: Fix ops.dequeue() semantics")
Cc: stable@vger.kernel.org # 7.1.x: 18d62044cda7: sched_ext: Preserve rq tracking across local DSQ dispatch
Cc: stable@vger.kernel.org # 7.1.x
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Assisted-by: Claude:claude-opus-5.5
Signed-off-by: Kuba Piecuch <jpiecuch@google.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
|
|
After seeing the patch [1] to guard is_in_v2_mode() with RCU, it makes
me realize that is_in_v2_mode() may be called in a context where a new
cgroup filesystem is being rebound with stale cpuset_cgrp_subsys.root
pointer. Avoid this potential UaF situation by adding a new cpuset_v2_mode
flag which is set when the cpuset_v2_mode mount option is used. This
flag is written into only when cpuset_bind() is being called with a
stable cpuset_cgrp_subsys.root value. The is_in_v2_mode() helper is
modified to read the new cpuset_v2_mode flag instead of accessing
cpuset_cgrp_subsys.root directly.
[1] https://lore.kernel.org/lkml/20260929084124.626693-2-arighi@nvidia.com
Fixes: b8d1b8ee93df ("cpuset: Allow v2 behavior in v1 cgroup")
Signed-off-by: Waiman Long <longman@redhat.com>
Reviewed-by: Ridong Chen <ridong.chen@linux.dev>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
|
|
The cpuset code has 2 different helpers for determining if it is in a
v1 or v2 working environment - cpuset_v2() and is_in_v2_mode().
cpuset_v2() checks the binding of the cpuset subsystem is on a v1
or v2 hierarchy. It can also help to optimize out v1 specific code
if CONFIG_CPUSETS_V1 isn't defined.
is_in_v2_mode() layers on top of cpuset_v2() and returns true if it is in
a v2 hierarchy or in a v1 hierarchy with "cpuset_v2_mode" mount option
enabled to make "cpuset.cpus" and "cpuset.mems" behave in a way similar
to their v2 counterparts. However the cpuset hierarchy may still be a
standalone one separated from the other cgroup hierarchies.
Commit 90ac22ffef48 ("sched/fair: Add cgroup_mode: max") adds a new
cpuset_num_cpus() helper to return the number of effective CPUs in a given
cgroup. It is part of the "sched: Flatten the pick" patch series to help
flatten the cpu cgroup runqueue hierarchies. cpuset_num_cpus() currently
uses is_in_v2_mode() to determine if it is in a v2 environment. However
is_in_v2_mode() can be true in a v1 environment where the cpuset and cpu
cgroup subsystems can be in two separate hierarchies. The current use
case [1] for cpuset_num_cpus() assumes that it can return the number
of effective CPUs present in the current cpu cgroup which may not be
the case.
Fix that by switching to use cpuset_v2() which signifies that it is in
a unified hierarchy where both cpuset and cpu cgroup subsystems share
the same cgroup hierarchy if it returns true.
[1] https://lore.kernel.org/lkml/20260930081125.GF88198@noisy.programming.kicks-ass.net
Fixes: 90ac22ffef48 ("sched/fair: Add cgroup_mode: max")
Signed-off-by: Waiman Long <longman@redhat.com>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
Reviewed-by: Michal Koutný <mkoutny@suse.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
KCSAN's skip_watch, udelay_task and udelay_interrupt parameters can be
changed at runtime through /sys/module/kcsan/parameters/, where 0 means
watching every access or not delaying at all. With the default
CONFIG_KCSAN_SKIP_WATCH_RANDOMIZE=y and CONFIG_KCSAN_DELAY_RANDOMIZE=y,
writing 0 to any of them makes the next watchpoint setup divide by zero:
# echo 0 > /sys/module/kcsan/parameters/udelay_task
Oops: divide error: 0000 [#1] SMP NOPTI
CPU: 0 UID: 0 PID: 93 Comm: sh Not tainted 7.3.0-rc5 #3 PREEMPT(lazy)
RIP: 0010:kcsan_setup_watchpoint+0x1b5/0x410
Code: ... 65 89 05 ad 8d 98 02 d3 ee <f7> f6 29 d7 e8 22 0e 0f 01 ...
RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
...
__show_trace_log_lvl+0x1bd/0x315
__die_body.cold+0x19/0x2b
die+0x34/0x50
...
BUG: TASK stack guard page was hit at ffffd3ab001fbfe8 (...)
reset_kcsan_skip() and delay_access() pass these values unchecked to
kcsan_prandom_u32_max(), which returns "state % ep_ro". delay_access()
halves the delay for compound and assert accesses, so
udelay_interrupt=1 is enough there.
The oops path runs instrumented code, which faults again until the stack
overflows. With skip_watch=0 the guest hangs without printing anything.
The division came in with commit 71a076f4a61a ("kcsan: Rewrite
kcsan_prandom_u32_max() without prandom_u32_state()"). Before it, the
helper scaled the random value with a multiply-shift, which yields 0 for
ep_ro == 0. Go back to that via reciprocal_scale().
Tested in an x86-64 KCSAN guest with virtme-ng: after this patch,
writing 0 to each parameter, or 1 to udelay_interrupt, no longer
oopses. skip_watch=0 now watches every access, which is very slow with
the default delays but completes when they are set to 0.
Fixes: 71a076f4a61a ("kcsan: Rewrite kcsan_prandom_u32_max() without prandom_u32_state()")
Cc: stable@vger.kernel.org
Assisted-by: LLM
Reviewed-by: Marco Elver <elver@google.com>
Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
Signed-off-by: Marco Elver <elver@google.com>
|
|
Program Structure reports have two attribution gaps. A missing jump
table is reported at the beginning of its subprogram rather than at
the gotox that needs the table, and the subprogram-layout checks run
before func_info and line_info are validated and installed, so their
reports cannot name the source function or line.
Report the missing jump table at the gotox instruction that looks it up,
rather than at the start of its subprogram.
func_info and line_info validation only needs the subprogram boundaries
found by add_subprogs() and the LD_ABS and tail-call properties that
check_subprogs() collects while scanning instructions; it does not
depend on the layout checks themselves. Split the property collection
into its own nested subprogram and instruction scan, together with the
program's callx marker, and run bpf_check_btf_info() before
check_subprogs(). The jump-boundary and fallthrough reports then carry
validated source information without changing either check.
CO-RE relocations are unaffected: commit c26e97721b17 ("bpf: Apply
CO-RE relocations before subprogram validation") applies them before
add_subprogs(), and they stay there. Only the func_info and line_info
validation moves.
Fixes: a8f427835394 ("bpf: Report Program Structure CFG errors")
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/cf2f420c2b21de440a7dc51b1565c0f06d4b539640ee5c03384e5d77bcfb5686@mail.kernel.org/
Link: https://lore.kernel.org/bpf/20260926133048.2962553-3-memxor@gmail.com
|
