| Age | Commit message (Collapse) | Author |
|---|
|
git://git.kernel.org/pub/scm/linux/kernel/git/driver-core/driver-core
Pull more driver core fixes from Danilo Krummrich:
- Revert TAINT_FORCED_BIND, introduced to flag manual sysfs bind/unbind
operations associated with misleading fuzzing reports. It also taints
established workflows, including VFIO device passthrough; a more
targeted approach will be revisited after 7.4.
- Rate limit synthetic uevent failure messages to prevent log flooding
from repeated invalid writes to sysfs uevent files
* tag 'driver-core-7.4-rc1-3' of git://git.kernel.org/pub/scm/linux/kernel/git/driver-core/driver-core:
Revert "driver core: add TAINT_FORCED_BIND for when userspace manually messes with devices and drivers"
driver core: Limit the messages on a refused uevent
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm
Pull power management fixes from Rafael Wysocki:
"These address assorted issues in the power management code and update
documentation:
- Fix potential out-of-bounds access in deferred reset in the cpufreq
statistics code (Zhongqiu Han)
- Cancel HWP notify work in the intel_pstate driver cleanup to
prevent possible use-after-free from occurring (Tristan Madani)
- Adjust state names and descriptions for ACPI _LPI idle states in
the intel_idle driver to make them follow the driver's idle state
naming convention (Rafael Wysocki)
- Document the new pm_runtime_use_autosuspend() undo mechanism that
triggers automatically on driver exit (Joshua Crofts)
- Do not call suspend_ops->begin()/end() for suspend-to-idle to avoid
driver confusion (Ovidiu Panait)
- Make the energy model management code allocate the cpumask for
non-CPU perf domains to prevent out-of-bounds reads from occurring
in the EM netlink interface (Karl Mehltretter)"
* tag 'pm-7.4-rc1-2' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm:
PM: runtime: clarify pm_runtime_use_autosuspend() undo mechanism
cpufreq: stats: Fix potential out-of-bounds access in deferred reset
PM: suspend: Do not call suspend_ops->begin()/end() for s2idle
intel_idle: Adjust state names and descriptions for _LPI states
PM: EM: allocate the cpumask for every perf domain
cpufreq: intel_pstate: cancel HWP notify work in driver cleanup
|
|
Pull more KVM updates from Paolo Bonzini:
"ARM64:
- Fix a couple of embarrassing merge mistakes in kvmarm-7.4
- Cleanup the HCR_EL2 bit fields to exclusively use the generated
definitions, fixings a nasty regression introduced in 6.16, where
barriers were not upgraded from NSH to ISH anymore
- Fix publication of state when a vcpu is forced out of the guest
- Let vcpu pausing nest correctly
- Sanitise ID_AA64MMFR3_EL1.S1P[IO]E to follow the rules of the
architecture
- Don't warn on SMCCC filter reserved range allocation failure
- Prevent GICv5 and TLBI hypercalls to be used from the host when
pKVM is active
- Enforce that only GICv3 guests are supported for pKVM
- Don't try to access GICv5 EL2 sysregs from EL1
- Compute architectrally correct values for ID_AA64MMFR3_EL1.S1P[IO]E
by checking the feature dependencies
- Require matching VGIC and ITS model
- Cleanup TRACE_REMOTE dependencies
- Fix random comments
ARM64 selftests:
- selftests: Skip unwritable ID fields in the failed-run test
- Move pkvm_selftests() after all CPUs have initialized EL2
RISC-V:
- Allow Zicclsm, Ziccamoa, Ziccif, Za64rs for KVM guest
- Implement KVM_PRE_FAULT_MEMORY ioctl()
- Replace imsic switch-case macros with range checks
RISC-V fixes:
- Avoid redundant interrupt check for pending timers
- Reject vCPU creation after AIA initialization
- Fix build with vector support disabled
- Fix IPI delivery for out-of-order vcpu_id
- Use a raw lock for IMSIC wakeup state
- Reject vCPU creation after AIA initialization
RISC-V selftests:
- Lazy V-extension enablement for KVM selftests
- Show sepc/scause/stval CSRs in KVM selftests vcpu dump"
* tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (42 commits)
KVM: selftests: Reject GICv3 ITS with a GICv5 VGIC
KVM: arm64: vgic: Require matching VGIC and ITS models
KVM: arm64: gic-v5: Drop unused addr variable
KVM: arm64: Sanitise ID_AA64MMFR3_EL1.S1P[IO]E
KVM: arm64: selftests: Skip unwritable ID fields in the failed-run test
tracing/remotes: Fix TRACE_REMOTE build when TRACING=n
KVM: arm64: Move pkvm_selftests() after all CPUs have initialized EL2
KVM: arm64: Get rid of compatibility HCR_EL2 aliases
KVM: arm64: Convert HCR_xyz to HCR_EL2_xyz
KVM: arm64: Set HCR_EL2.BSU to Inner Shareable for guests
KVM: arm64: Fix stale VGICv3 comments in the nVHE world switch
KVM: arm64: vgic: Do not access the GICv5 CPU interface from EL1
KVM: arm64: Validate the host-provided vgic model in pKVM
KVM: arm64: Reject the stage-2 flush hypercalls under pKVM
KVM: arm64: Return on kvm_gmem_get_pfn() failure in gmem_abort()
KVM: riscv: selftest: add Zicclsm, Ziccamoa, Ziccif, Za64rs
RISC-V: KVM: Allow Zicclsm, Ziccamoa, Ziccif, Za64rs in guest
MAINTAINERS: name the kvm-riscv/linux riscv_kvm_next branch
RISC-V: KVM: Don't write MMIO load result to x0
RISC-V: KVM: Use a raw lock for IMSIC wakeup state
...
|
|
Merge assorted power management fixes for 7.4-rc1:
- Fix potential out-of-bounds access in deferred reset in the cpufreq
statistics code (Zhongqiu Han)
- Cancel HWP notify work in the intel_pstate driver cleanup to prevent
possible use-after-free from occurring (Tristan Madani)
- Adjust state names and descriptions for ACPI _LPI idle states in the
intel_idle driver to make them follow the driver's idle state naming
convention (Rafael Wysocki)
- Document the new pm_runtime_use_autosuspend() undo mechanism that
triggers automatically on driver exit (Joshua Crofts)
- Do not call suspend_ops->begin()/end() for suspend-to-idle to avoid
driver confusion (Ovidiu Panait)
* pm-cpufreq:
cpufreq: stats: Fix potential out-of-bounds access in deferred reset
cpufreq: intel_pstate: cancel HWP notify work in driver cleanup
* pm-cpuidle:
intel_idle: Adjust state names and descriptions for _LPI states
* pm-runtime:
PM: runtime: clarify pm_runtime_use_autosuspend() undo mechanism
* pm-sleep:
PM: suspend: Do not call suspend_ops->begin()/end() for s2idle
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull vfs fixes from Christian Brauner:
- writeback:
- Update dirtied_when instead of dirtied_time_when when a wb switch
puts an inode at the newest end of the target b_dirty list. The
list was left out of order so move_expired_inodes() could stop
before older inodes, and the lazytime expiration timestamp was
reset
- Clear the old wb's dirty-I/O state and bandwidth contribution when
a batch of inode switches leaves its dirty lists empty. An empty
wb stayed scheduled for periodic writeback and inflated
bdi->tot_write_bandwidth
- Take wb_switch_rwsem across the inode scan of
cleanup_offline_cgwb(). The v7.3-rc5 fix that rotates scanned
inodes let a scan racing a sync()'s queue_io() move lazytime
inodes past the sync's cutoff, so sync() returned without
persisting their timestamps
- Don't kick writeback for an IOCB_DONTCACHE write on an inode that
has no wb yet. With cgroup writeback an O_DIRECT RWF_DONTCACHE
write to a block device dereferenced NULL
- statmount/listmount/listns:
- Refuse statmount() string requests with a buffer that isn't larger
than struct statmount
- Don't wrap around with a U64_MAX cursor in listmount() and
listns()
- Don't end listns() at an inactive namespace
- Only add init_ipc_ns and init_net to the namespace trees if
CONFIG_IPC_NS and CONFIG_NET_NS are enabled
- iomap:
- Only mark blocks uptodate in __iomap_write_begin() after reading
them and let __iomap_write_end() mark the zeroed blocks uptodate
once the copy from userspace succeeded
- Check a chunk against max_integrity_io_size() on its own before
the bio size check
- Only enforce block alignment for direct I/O into holes and
unwritten extents if IOMAP_DIO_FSBLOCK_ALIGNED is set
- backing-file:
- Check S_APPEND and S_IMMUTABLE on the backing inode. FUSE
passthrough opens the backing file with vfs_open() which skips
may_open(), so anyone who may open the FUSE file for writing could
overwrite an append-only backing file anywhere or write to an
immutable one. Open, mmap and writes now check the backing inode
the way may_open() and do_mmap() do
- Always call kiocb_end_write() for a queued write
- Let backing files inherit FMODE_NONOTIFY from the user file. The
event fds fanotify creates for permission events on a stacked
filesystem raised FS_OPEN_PERM on the real path from inside the
listener's read() and deadlocked listeners with marks on both
layers
- pidfs:
- Look the task up before checking for exit information in
PIDFD_GET_INFO and start over if the task no longer owns the pid
once all fields have been read. A caller racing release_task() got
neither PIDFD_INFO_PID nor PIDFD_INFO_EXIT or ESRCH although the
exit information was available, and a caller racing a sub-thread's
exec reported that thread's tid as the pid
- Clear the mask bits of fields that lie beyond the struct size the
caller passed. The uapi header promises that the mask names no
such field but a caller passing PIDFD_INFO_SIZE_VER0 got the
coredump and supported mask bits set for fields that were never
written
- aio:
- Give the aio ring a ->read_folio that fails with EIO so a fault on
a ring whose file was truncated gets SIGBUS instead of jumping to
NULL
- Check nr_events against the ring size before doubling it in
ioctx_alloc()
- splice:
- Only write back an implicit file position in do_splice() when the
splice helper succeeded. splice doesn't take f_pos_lock so a
failed splice restored a stale snapshot over a position that a
concurrent getdents() had moved. On ext2 the next readdir then
trusted an offset inside a directory entry
- Retry tee() when the input pipe was emptied or the output pipe
filled between the prep steps and link_pipe() instead of returning
0 while the input pipe still has writers, as splice_pipe_to_pipe()
does
- ucount:
- Take the reference that pins a ucounts level on the 0 -> 1
transition before checking the rlimit
- Read the parent ucounts before dropping the rlimit count in
do_dec_rlimit_put_ucounts()
- inode:
- Rotate inodes whose page cache couldn't be dropped to the tail of
the LRU
- Skip inodes that another LRU walker is isolating
- setns:
- Set nsset->flags before anything in prepare_nsset() can fail so
the error path drops the cred prepared for CLONE_NEWUSER. A
failing setns() with CLONE_NEWUSER and CLONE_NEWNS leaked it
together with its user namespace
- Refuse setns(CLONE_NEWNS) for a task that shares its fs_struct
again when more namespaces are requested
- minix:
- Restore uptodate on the pinned bitmap buffers before redirtying
them after a failed writeback instead of tripping the warning in
mark_buffer_dirty(), as ext2 and ext4 do for their superblock
buffers
- Reject rmdir() of a directory with a corrupted link count with
EFSCORRUPTED instead of hitting the drop_nlink() warning
- userns: Keep the exec securebits and their locks when creating or
joining a user namespace. set_cred_user_ns() reset all securebits to
SECUREBITS_DEFAULT so anyone who may create a user namespace could
shed SECBIT_EXEC_RESTRICT_FILE and SECBIT_EXEC_DENY_INTERACTIVE with
a single unshare(CLONE_NEWUSER)
- fs: Honor FOP_UNSIGNED_OFFSET in sys_llseek() and in the pread64(),
pwrite64(), preadv() and pwritev() wrappers
- proc/kcore: Advance the vmalloc retry in read_kcore_iter() by the
last vread_iter() copy instead of the running total
- eventpoll: Return the error when the wakeup source for EPOLLWAKEUP
can't be created in EPOLL_CTL_MOD instead of reporting success
without suspend protection
- coredump: Mark do_coredump() noinline_for_stack
- nsfs: Check the trailing bytes of a namespace file handle in bytes
instead of words
- netfs: Restore the per-level and per-point decoding of the debug mask
- exportfs: Release the directory file get_name() opened with
__fput_sync(). From nfsd's kernel threads the deferred fputs arrived
faster than delayed_fput retired them and each pinned the directory's
readdir state, growing unreclaimable slab by about 90 MB per second
under a parallel tree walk
- fs: Call ramfs_kill_sb() for a ramfs rootfs. Since rootfs can be
unmounted as of v7.0 its struct ramfs_fs_info was leaked
- signal: Check the blocked mask for every entry
dequeue_synchronous_signal() looks at. The scan could select a
blocked synchronous signal and deliver it while blocked
- super: Convert the superblock device table from an rhltable to a
plain hashtable under a spinlock. This fixes a bug from v7.3 where
fs_bdev_freeze() froze a btrfs filesystem through a stale member that
open_ctree() had already released while the freeze was waiting for
SB_BORN
- fork: Leave rcu_users at zero until copy_process() can't fail
anymore. bpf_task_acquire() handed out trusted references to the new
task from the task_alloc LSM hook and sched_ext's ops.init_task() and
a failing fork freed the task behind them
- xattr: Always run cap_convert_nscap() for a security.capability
write. A zero-length setxattr() skipped it and let an unprivileged
caller without CAP_SETFCAP or write permission on the file plant an
empty security.capability xattr that made every execve() of the
binary fail with EINVAL
- namei: Skip the S_ISDIR assertion in
- proc: Refuse to mount on entries below /proc/<pid>/net like fd/,
fdinfo/ and map_files/. proc_net_d_revalidate() always fails so the
first lookup by anyone with search permission ran d_invalidate() on
the dentry and detached every mount on it, locked mounts included
- libfs: Hold one mount reference for all pins instead of one per pin.
The pin count is what keeps the mount alive, so the per-pin reference
only cost a mntget() and a mntput() through
mntput_no_expire_slowpath() for every debugfs, tracefs, securityfs or
configfs file that comes and goes
- iov_iter: Return 0 from iov_iter_alignment() for an empty iterator of
any type
- watch_queue: Give info_id its own storage instead of overlaying it
with rcu_head
- filelock: Take an active superblock reference while waiting for a
delegation break. After a lazy umount closing the delegation holder
could shut down the superblock under the waiter whose final iput()
then ran shmem_evict_inode() on freed state
- fs: Skip the copy_file_range() write limit checks when there is no
data to copy
- bpf: Allow bpf_d_path() from fentry on filp_close_sync(). Since
close_range() became synchronous this cycle close(), close_range()
and the closes on exec and exit no longer go through filp_close().
fexit stays rejected since the file may be gone by then
- fs_pin: Dequeue the wait entry unconditionally in pin_kill(). A
spuriously woken waiter kept its on-stack entry queued, took that as
proof that the pin was alive and used the freed acct structure when
umount() of the accounting file raced acct(2)
- dcache: Don't fail __d_obtain_alias() for an inode without links and
aliases. The ESTALE check added this cycle broke open_by_handle_at()
and nfsd file handle decoding on NFS and FUSE where i_nlink is just
what the server last reported and the handle may well still be valid
- selftests/ovl: Fix the mountinfo expectations of the detached layer
test which has failed since it was added
* tag 'vfs-7.4-rc1.fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs: (60 commits)
fs: don't fail __d_obtain_alias() for unlinked inodes
ns: don't add init_ipc_ns and init_net without namespace support
aio: avoid request count overflow in ioctx_alloc()
selftests/ovl: fix detached layer mountinfo expectations
bpf: Allow bpf_d_path() from filp_close_sync()
fs: skip copy_file_range write limits when there is no data
minix: reject rmdir of a directory with a corrupted link count
fs_pin: do not trust the pin after a spurious wakeup in pin_kill()
ucount: read the parent ucounts before dropping the rlimit count
splice: retry tee() instead of returning 0 when it races
selftests/pidfd: test that fields beyond the struct size are not reported
pidfs: do not report fields that do not fit into the caller's struct
fs: let backing files inherit FMODE_NONOTIFY from the user file
splice: avoid stale f_pos writeback on error
backing-file: always end a queued write
filelock: keep superblock active through delegation wait
watch_queue: keep info_id separate from rcu_head
iomap: only follow dio alignment requirement if specified
iov_iter: fix NULL deref in iov_iter_alignment()
backing-file: enforce append-only and immutable backing inodes
...
|
|
The d_path selftest closes its descriptors with close_range() so that
its fentry program on filp_close() runs.
Commit 056069eb2b66 ("fs: make close_range() synchronous") switched
close_range() to filp_close_sync(), so the program still attaches but
is never called:
test_d_path_basic:FAIL:close trampoline for filp_close was not called
With that series, close(), close_range() and the closes on exec and
exit all go through filp_close_sync(). Few paths still reach
filp_close(), dup2() being one of them, so a program on filp_close()
no longer sees most file closes.
Allow bpf_d_path() from filp_close_sync(), but only from fentry.
filp_close_sync() drops the last file reference inline, so by the time
an fexit program runs, file->f_path may already be freed, with no RCU
delay for NORCU dentries such as pipes and sockets.
Attach the selftest program to filp_close_sync() and keep triggering
it with close_range().
Add a test rejecting bpf_d_path() from fexit/filp_close_sync.
Fixes: 056069eb2b66 ("fs: make close_range() synchronous")
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Link: https://patch.msgid.link/20261021201400.5409-1-ihor.solodrai@linux.dev
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
do_dec_rlimit_put_ucounts() drops its share of iter->rlimit[type] and
only then loads iter->ns->ucounts. If the decrement leaves the count
non-zero, a concurrent caller can take it to zero and put the last
reference, which frees the ucounts and possibly its user namespace with
kfree_rcu(). An exiting task frees its pending signals without siglock
or any RCU read-side section, so when it is preempted between the
decrement and the load for a grace period it reads freed memory and
then decrements a counter through the pointer it found there.
A pending signal holds no reference on its ucounts: the reference is
taken by whoever moves the count from 0 to 1 and dropped by whoever
moves it back to 0, so the caller's share of the count is all that
keeps iter alive, and only until it is dropped.
Load the parent first; it stays pinned by the caller's share of its
own count.
Fixes: 15bc01effefe ("ucounts: Fix signal ucount refcounting")
Signed-off-by: Nickolai Zeldovich <nickolai@csail.mit.edu>
Link: https://patch.msgid.link/20261024015513.830312-1-nickolai@csail.mit.edu
Reviewed-by: Bradley Morgan <brads@mainlining.org>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
struct watch overlays info_id with rcu_head. A notification poster reads
info_id under rcu_read_lock(), but a concurrent final put can enqueue the
watch with call_rcu(). Enqueueing the callback writes rcu_head before the
grace period has elapsed, so it can corrupt info_id while the object
remains legally visible to the reader.
__post_watch_notification() then merges that value into the notification
info word without masking it. Pointer bits can corrupt WATCH_INFO_LENGTH
and make post_one_notification() copy beyond the posting task's stack
record.
Give info_id storage separate from rcu_head so RCU enqueueing cannot modify
it. Place it after the kref, which uses existing tail padding on x86-64.
Mask info_id with WATCH_INFO_ID at the point of use as an additional
invariant check.
An unprivileged shell and keyctl reproducer was run for 30 seconds in QEMU.
The vulnerable baseline produced malformed notification lengths, while this
change completed with 288631 key notifications and no malformed record.
Fixes: c73be61cede5 ("pipe: Add general notification queue support")
Reported-by: Tong Liu <lyutoon@gmail.com>
Reported-by: Qi Tang <tpluszz77@gmail.com>
Reported-by: Zhiyu Zhang <zhiyuzhang999@gmail.com>
Closes: https://lore.kernel.org/lkml/CALf2hKveij_sHgANY9Aktm1dPa4wfKjKNkHyp9yPbjORWFXagA@mail.gmail.com/
Link: https://lore.kernel.org/netdev/CALf2hKvp0NPVLwr=a3+jUHtPohxbra8gox+aYAfWathJnn-J+Q@mail.gmail.com/
Signed-off-by: Zhiyu Zhang <zhiyuzhang999@gmail.com>
Link: https://patch.msgid.link/20261029054136.1174216-1-zhiyuzhang999@gmail.com
Reviewed-by: Bradley Morgan <brads@mainlining.org>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/danielt/linux
Pull kgdb updates from Daniel Thompson:
"Varied updates this cycle.
Naveen Kumar Chaudhary fixed a memory leak when the environment is
already full, eliminated to weird historical cruft (CMD_BUFLEN having
different values in different files) and removed some dead code.
Fang Xieyan tackled a family of related buffer overflows in the gdb
remote protocol parsing (great fixes from a defensive coding point of
view but not security relavent; gdb can already do anything it wants
to)"
* tag 'kgdb-7.4-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/danielt/linux:
kgdb: Fix buffer overflow in the qRcmd packet handler
kgdb: Fix buffer overflow in the 'M' and 'X' packet handlers
kgdb: Fix buffer overflow in the 'm' packet handler
kgdb: remove unreachable return in kgdb_reenter_check()
kdb: unify CMD_BUFLEN definition into kdb_private.h
kdb: fix memory leak in kdb_setenv() when environment is full
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/hyperv/linux
Pull hyperv updates from Wei Liu:
- Update to documents and code comments (Michael Kelley)
- Harden Vmbus (Kameron Carr)
- Rework MSHV driver page deposit logic (Mukesh Rathor)
- Vmbus kexec bug fix (Shradha Gupta)
- Miscellaneous bug fixes (Daehyeon Ko, Jiale Yao, Sasha Levin)
* tag 'hyperv-next-signed-20261028' of git://git.kernel.org/pub/scm/linux/kernel/git/hyperv/linux:
Drivers: hv: Remove leftover support for old Hyper-V versions
mshv: Remove unused *_old memory deposit functions
mshv: Redesign hypervisor memory deposit logic
mshv: Rename memory deposit memory functions to _old
mshv_vtl: reject poll files with multiple waitqueues
mshv_vtl: detach poll waiter on POLLFREE
mshv_vtl: initialize poll file mutex before device registration
Documentation: hyperv: Update information on guest CPU offlining
Documentation: hyperv: Remove support for guests on Hyper-V 2012/2012R2
Drivers: hv: Run VMBus kexec cleanup from syscore shutdown
kexec: Add helper to report an active kexec reboot
tools: hv: remove unused rte_smp_mb()
Documentation: hyperv: Update documentation for sending VMBus packets
Drivers: hv: vmbus: Check ACPI companion before use
Documentation: hyperv: Describe validation of the ring buffer indices
Drivers: hv: vmbus: Keep the ring byte counts sane for a bad index
Drivers: hv: vmbus: Compute ring byte counts from a caller-held snapshot
Drivers: hv: vmbus: Use *_ONCE() to access shared ring buffer indices
Drivers: hv: vmbus: Bounds check the shared ring buffer indices
|
|
messes with devices and drivers"
This reverts commit fcbfaffee51a452cc8efcbaa3b88ec4595678424.
Turns out that vfio is a crazy beast and people have been "relying" on
the bind/unbind debugging feature as a way to handle passing through
devices to guest instances. Because of this (https://xkcd.com/1172/),
let's revert the feature for now until we can figure out a more agreeable
way to handle this such that normal workflows do not taint the kernel.
Odds are something like what Uwe proposes at:
https://lore.kernel.org/lkml/cover.1790612298.git.u.kleine-koenig@baylibre.com/
will be the best way forward and will be revisited after for after 7.4.
The feature was originally intended to stem the flow of unnecessary
fuzzing reports of "broken" drivers where random devices were being
bound to random drivers and of course things were going wrong. The main
syzbot tool is now not doing this anymore, and hopefully no one else
starts it up again (but in this madness of LLMs these days, you can't be
sure.)
Thanks to everyone for telling me just how wrong I was to implement
this, I appreciated the feedback :)
Link: https://lore.kernel.org/all/DLIL9H50MALI.3JROXYEEUM3KU@kernel.org/
To: Jonathan Corbet <corbet@lwn.net>
To: Shuah Khan <skhan@linuxfoundation.org>
To: Randy Dunlap <rdunlap@infradead.org>
To: "Rafael J. Wysocki" <rafael@kernel.org>
To: Danilo Krummrich <dakr@kernel.org>
To: Johan Hovold <johan@kernel.org>
To: Steven Rostedt <rostedt@goodmis.org>
To: Masami Hiramatsu <mhiramat@kernel.org>
To: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Reported-by: Leon Romanovsky <leon@kernel.org>
Reported-by: Jason Gunthorpe <jgg@ziepe.ca>
Reported-by: Michal Pecio <michal.pecio@gmail.com>
Reported-by: "Uwe Kleine-König" <u.kleine-koenig@baylibre.com>
Link: https://patch.msgid.link/20261028-bind_taint-revert-v1-1-7c6b9cef30c9@linuxfoundation.org
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
[ Revert the change in Documentation/admin-guide/tainted-kernels.rst.
- Danilo ]
Signed-off-by: Danilo Krummrich <dakr@kernel.org>
|
|
Pull bpf fixes from Alexei Starovoitov:
- Remove BPF_F_ARENA_SCALAR flag and .arena.data section in libbpf that
were added for rust-bpf in this merge window. We found a cleaner
approach that does it on compiler side instead of kernel/libbpf. No
released kernel has them (Alexei Starovoitov)
- Fix bpf_send_signal_task() returning -EPERM when it is called from a
kernel thread (Aditya Sharma)
- Various fixes (Alexei Starovoitov):
- Use emit_rsb_call() to call bpf_unwind_target() in x86 JIT to fix
objtool warning during the build
- Fix scalar evolution: clear delta of a widened register, don't
widen a pointer to a buffer, don't clamp a register whose type
changed in the loop, and reset the range of a clamped packet
pointer
- Finish the insn in the state that maybe_fork_scalars() splits
off, so that pruning doesn't drop it at the head of a counted
loop
- Preserve the registers that carry stack arguments across
arch_bpf_timed_may_goto() on x86-64, arm64 and riscv64
- Don't re-sort subprog_info[] for every new subprogram
- Fix flaky map_kptr and rcu_reentry selftests
- Use memmove() in bpf_skb_load_bytes(), bpf_skb_load_bytes_relative()
and bpf_xdp_load_bytes(), since the destination can be a slice of the
same packet (Ayushman Rout)
- Fixes and follow-ups for scalar evolution: reject pointer operands in
32-bit recurrences and latches, don't intersect with the current
register range on clamp to avoid a false 'verifier bug', avoid false
infinite loop reports, and recognize a 32-bit copy of the induction
variable (Eduard Zingerman)
- Fix out-of-bounds write to btf->types[] for split BTF and propagate
btf_add_type() errors in btf_check_all_metas() (Feng Yang)
- Fix use-after-free in bpf_timer callback that fires after its inner
map was freed (Mani Tofigh)
- Fix x86 JIT stopping before jumps converge: size backward jumps by
the insns they jump over (Mykyta Yatsenko)
- Fix use-after-free of struct_ops trampoline: dissociate struct_ops
progs before the grace period of the map (Qiliang Yuan)
- Validate stack argument reads against the caller state to fix
out-of-bounds read in the verifier, and reject callbacks with stack
arguments (Weiming Shi)
- Tie borrowed iterator results and pointers loaded from trusted fields
to the lifetime of their source, so that iterator next, destroy and
reference release invalidate them (Xu Yunxiang)
* tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf: (48 commits)
bpf: Don't re-sort subprog_info[] for every new subprogram
bpf: Fix out-of-bounds btf->types write for split BTF
selftests/bpf: Test stack arguments across may_goto
bpf: Keep stack arguments in arch_bpf_timed_may_goto()
selftests/bpf: Test maybe_fork_scalars() at the head of a counted loop
bpf: Finish the insn in the state that maybe_fork_scalars() splits off
selftests/bpf: Test the clamp of pointers in scalar evolution
bpf: Reset the range of a packet pointer that scalar evolution clamps
bpf: Don't clamp a register whose type changed in the loop
selftests/bpf: Test a loop that moves a pointer to a buffer
bpf: Don't widen a pointer to a buffer in scalar evolution
selftests/bpf: Avoid a page fault in rcu_reentry while the handler is armed
selftests/bpf: Wait for the last maps of map_kptr to be freed
selftests/bpf: Test that a widened register is not linked
bpf: Clear delta of a register that scalar evolution widens
bpf, x86: Remove r0_on_stack from fetching atomics in arena
bpf: Remove BPF_F_ARENA_SCALAR
selftests/bpf: Remove tests for arena access through numbers
libbpf: Keep format strings of bpf_printk() in .rodata
libbpf: Remove .arena.data
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks
Pull kthread fix from Frederic Weisbecker:
"Fix a bug where dynamic CPU isolation through cpuset isolated
partitions can move a kthread out of the housekeeping range.
The fix makes sure the fallback affinity is always a set of online
housekeeper"
* tag 'kthread-fix-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks:
kthread: Fall back when no preferred housekeeping CPU is online
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext
Pull sched_ext fixes from Tejun Heo:
"This is a big pull for fixes, but much of it was in the works before
the merge window.
Most of these fixes are in sub-scheduler support, which nests BPF
schedulers under a parent that hands them CPUs. It was heavily
reworked over recent cycles and is now functionally complete. Porting
real schedulers on it drove the latest additions and keeps finding
corner cases.
For example, once CPUs are shared and can be taken back, the kernel
must keep every task moving when its scheduler can't run it or is
failing, and those fallbacks interact subtly with each other and with
other new core features.
- The fallback paths meant to keep a task moving when its scheduler
can't run it or is failing could themselves leave it stuck
- The stall watchdog could blame a scheduler for time it wasn't
responsible for, even taking down all of sched_ext
- Sub-schedulers were told late, or not at all, which CPUs they could
use
- Deferred callbacks could stay queued or be dropped under core
scheduling and proxy execution, with a sched/core fix to the
duplicate check
- Hardening: tracing programs can no longer call kfuncs that could
deadlock the CPU"
* tag 'sched_ext-for-7.4-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext: (34 commits)
sched_ext: Reject scx_bpf_sub_grant/revoke() in tracing programs
sched_ext: Fix the rescue overload pick ejecting the wrong sub under bypass
sched_ext: Fix use-after-free of a NONE rescuee leaving its sched while running
sched_ext: Fix a lone IMMED task waiting behind a kept prev
sched_ext: Fix queued bypassed tasks stalling when the host loses SCX_CAP_BASE
sched_ext: Let the bypass load balancer move tasks homed on other cpus
sched_ext: Factor out bypass_lb_move() and insert with the donor DSQ unlocked
sched_ext: Fix the bypass load balancer stranding tasks on cpus where the host lacks SCX_CAP_BASE
sched_ext: Fix bypassed tasks stalling when the host lacks SCX_CAP_BASE
sched_ext: Expire prev's slice in dispatch_one() under bypass
sched_ext: Fix a cpu idling after put_prev_task_scx() queues its task on the local DSQ
sched_ext: Refuse extending a kernel slice
sched_ext: Mark the rescue grant as a kernel slice with SCX_TASK_KERNEL_SLICE
sched_ext: Fix a cpu idling with a runnable rescuee
sched_ext: Restore a rescuee's slice after ops.stopping()
sched_ext: Fix rescue served time disagreeing with the slice under proxy execution
sched_ext: Move refill_task_slice_dfl() below apply_slice_vtime()
sched_ext: Fix bypassed tasks looping on a host without SCX_CAP_ENQ
sched_ext: Fix bypassed tasks stranded by the reenqueue limit
sched_ext: Clear holding_cpu where the claim is confirmed
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup
Pull cgroup fixes from Tejun Heo:
- cgroup.kill did not kill a process whose thread group leader had
exited while other threads still ran, nor those threads once moved to
another cgroup, letting a process survive its container being killed
- A cgroup.procs write under favordynmods could migrate a newly forked
process before fork linked it to its cgroup, corrupting cgroup task
lists
- cpuset partition fixes for nested hierarchies: invalidation, revival,
shrinking and switches between root and isolated could leak isolated
CPUs, skip or miscompute the housekeeping check, or leave wrong
effective CPUs:
- A root partition can no longer be created under an isolated
parent or stay valid under one, as disabling it could leave no
housekeeping CPUs
- Enabling a partition failed when a grandchild held tasks,
although the parent had none of its own
- Separately, an allocation racing with a cpuset.mems update could
drop its mempolicy nodemask and land on the wrong node, and a
memoryless node in cpuset.mems triggered a spurious "movable
nodes only" warning
- Two no-functional-change cleanups of the effective CPU
computation
* tag 'cgroup-for-7.4-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup: (27 commits)
cgroup/cpuset: Integrate compute_partition_effective_cpumask() into compute_effective_cpumask()
cgroup/cpuset: Relocate compute_*effective_cpumask() and compute_effective_nodemask()
cgroup/cpuset: Release invalid isolated CPUs with a shrunk root partition parent
cgroup/cpuset: Release invalid isolated CPUs in compute_partition_effective_cpumask()
cgroup/cpuset: Make sure compute_partition_effective_cpumask() is only called for valid partition
cgroup/cpuset: Prevent recursive call of remote_cpus_update() via update_cpumasks_hier()
selftests/cgroup: Check for a single range nohz_full CPUs before running test matrix
cgroup/cpuset: Minor fixes to prstate_housekeeping_conflict()
cgroup/cpuset: Fix incorrect effective_cpus bug in update_cpumasks_hier()
cgroup/cpuset: Properly compute effective xcpus in update_parent_effective_cpumask()
cgroup/cpuset: Find furthest isolated ancestor for invalidated isolated partition
cgroup: don't kill migrated processes through their exiting threads
cgroup/cpuset: check effective mems for a movable-only configuration
cgroup/cpuset: Fix task checking when enabling a partition
cgroup: serialize migrations of newly forked processes
selftests/cgroup: test cgroup.kill with an exited thread group leader
cgroup: fix cgroup.kill for exited thread group leader
selftests/cgroup: Skip test_cpuset_prs.sh test if some required CPUs are offline
selftests/cgroup: Add tests for nohz_full housekeeping check
selftests/cgroup: Add tests for isolcpus housekeeping check
...
|
|
listns() resumes after the last returned id by looking up the first
namespace with a larger id with lookup_ns_id_after(), or with
lookup_ns_owner_after() when filtering by owner. Both also take a
reference with ns_get_unless_inactive(). If that namespace is inactive,
e.g. a network namespace that is only pinned by a socket or a user
namespace that is only pinned by a file's credentials, or if it is being
torn down, the lookup returns NULL and listns() fails with -ENOENT.
Userspace takes that as the end of the list, so every namespace after
it is silently missed:
ids 10 (active), 11 (inactive), 12 (active)
listns(ns_id = 0, nr = 1) -> 10
listns(ns_id = 10, nr = 1) -> lookup_ns_id_after(10) -> NULL, -ENOENT
12 is never listed
A call without a cursor doesn't have this problem because the loop just
skips the namespaces that legitimize_ns() rejects.
The reference only kept the starting point valid until the RCU walk
began. Look it up under rcu_read_lock() instead, without taking a
reference. The namespace can't be freed before the RCU read-side section
ends, and if it is unlinked in the meantime its ->next still leads back
into the list, which the walk already relies on for every other entry.
The starting namespace then goes through legitimize_ns() like every
other one, and kls->first_ns isn't needed anymore.
Reported-by: shaos <shawnq1@163.com>
Link: https://lore.kernel.org/r/20261020110746.959806-1-shawnq1@163.com
Fixes: 76b6f5dfb3fd ("nstree: add listns()")
Cc: stable@vger.kernel.org
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
SECBIT_EXEC_RESTRICT_FILE and SECBIT_EXEC_DENY_INTERACTIVE tell script
interpreters to only run what AT_EXECVE_CHECK allows and to refuse
interactive input. A service manager sets them together with their lock
bits so that nothing below it can turn them off again.
But set_cred_user_ns() resets all securebits to SECUREBITS_DEFAULT. So
any task that can create a user namespace sheds the locked exec bits
with a single unshare(CLONE_NEWUSER) and joining one with setns() does
the same. The lock is pointless if anyone who may create a user
namespace can drop it, and that is pretty much everyone.
Resetting the other bits is fine. They are about how capabilities are
computed and a new user namespace starts out with a full set of
capabilities anyway. The exec bits grant nothing though. They are
restrictions a task placed on itself, just like no_new_privs which a
user namespace doesn't touch either. So keep them and their locks.
Reproducer:
// SPDX-License-Identifier: GPL-2.0
/* Lock the exec securebits, then create a user namespace and look at them again. */
#define _GNU_SOURCE
#include <sched.h>
#include <stdio.h>
#include <sys/prctl.h>
#define SECBIT_EXEC_RESTRICT_FILE (1 << 8)
#define SECBIT_EXEC_RESTRICT_FILE_LOCKED (1 << 9)
#define SECBIT_EXEC_DENY_INTERACTIVE (1 << 10)
#define SECBIT_EXEC_DENY_INTERACTIVE_LOCKED (1 << 11)
int main(void)
{
int bits = SECBIT_EXEC_RESTRICT_FILE | SECBIT_EXEC_RESTRICT_FILE_LOCKED |
SECBIT_EXEC_DENY_INTERACTIVE | SECBIT_EXEC_DENY_INTERACTIVE_LOCKED;
if (prctl(PR_SET_SECUREBITS, bits, 0, 0, 0)) {
perror("PR_SET_SECUREBITS");
return 1;
}
printf("before: 0x%x\n", prctl(PR_GET_SECUREBITS, 0, 0, 0, 0));
printf("clear locked bits: %s\n", prctl(PR_SET_SECUREBITS, 0, 0, 0, 0) ? "refused" : "ALLOWED");
if (unshare(CLONE_NEWUSER)) {
perror("unshare(CLONE_NEWUSER)");
return 1;
}
printf("after unshare(CLONE_NEWUSER): 0x%x\n", prctl(PR_GET_SECUREBITS, 0, 0, 0, 0));
return 0;
}
On v7.1 this prints "before: 0xf00", "clear locked bits: refused" and
"after unshare(CLONE_NEWUSER): 0x0". With this patch the last line
reads 0xf00 as well.
Fixes: a0623b2a1d59 ("security: Add EXEC_RESTRICT_FILE and EXEC_DENY_INTERACTIVE securebits")
Cc: stable@vger.kernel.org # v6.14+
Link: https://patch.msgid.link/20261026-work-securebits-userns-v1-1-b35473fa1a34@kernel.org
Reviewed-by: Mickaël Salaün <mic@digikod.net>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
dup_task_struct() sets rcu_users to 2. From then on bpf_task_acquire()
hands out references to the new task, which BPF programs see as a trusted
pointer long before it is sure to live: in the task_alloc LSM hook and in
ops.init_task() of sched_ext.
If copy_process() fails later on, it undoes what it has done and frees
the task without a look at the count. A program that has stored its
reference in a map is left with a pointer to freed memory:
BUG: KASAN: slab-use-after-free in put_task_struct_rcu_user+0x22/0x90
The program is loaded by the administrator, but any user can make a fork
fail, for example at the pids.max of a cgroup.
Honouring the count would not do: the task would stay behind with its mm,
files and credentials freed. Leave rcu_users at zero until copy_process()
cannot fail any more. bpf_task_acquire() returns NULL until then, which
its callers have to be prepared for, and which they get for init_task,
whose count has always been zero. The other users of rcu_users deal with
tasks that have run.
Fixes: 90660309b0c7 ("bpf: Add kfuncs for storing struct task_struct * as a kptr")
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
gdb_cmd_query() decodes the hex payload of a qRcmd packet (the GDB
"monitor" command passthrough to kdb) into remcom_out_buffer and hands
it to kdb_parse(). It sizes the decode with strlen() of the payload:
int len = strlen(remcom_in_buffer + 6);
...
kgdb_hex2mem(remcom_in_buffer + 6,
remcom_out_buffer, len);
len = len / 2;
kgdb_hex2mem() takes count as the number of bytes to produce and
decodes them in place at buf, reading and writing 2 * count bytes
there. len here is the hex character count, twice the decoded size,
so the decode runs over 2 * len bytes at remcom_in_buffer + 6, off
the end of the buffer. On x86 BUFMAX is 1024, so a 256-byte monitor
command (512 hex characters) overruns remcom_in_buffer during the
in-place decode:
BUG: KASAN: global-out-of-bounds in kgdb_hex2mem+0x131/0x160
Read of size 1 at addr ffffffff8765ef05 by task sh/1
...
kgdb_hex2mem+0x131/0x160
gdb_serial_stub+0x16d4/0x3170
kgdb_cpu_enter+0xb68/0x1520
...
The buggy address belongs to the variable:
remcom_in_buffer+0x405/0x420
Halve len before the call so kgdb_hex2mem() gets the decoded byte count.
It then reads the len hex characters already in remcom_in_buffer and
writes len / 2 bytes, all inside the buffer, and remcom_out_buffer holds
the same command as before.
Fixes: a0de055cf613 ("kgdb: gdb "monitor" -> kdb passthrough")
Cc: stable@vger.kernel.org
Suggested-by: Douglas Anderson <dianders@chromium.org>
Link: https://lore.kernel.org/all/CAD=FV=V1-gNZ8yTsC+D8inePWv=A+9H09hWJojQf5NRpcNvK+Q@mail.gmail.com/
Assisted-by: Qoder:Qwen3.8-Max
Signed-off-by: Fang Xieyan <fangxy@xiaopeng.com>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
Link: https://lore.kernel.org/r/20260916151753.6722-1-fangxy@xiaopeng.com
Signed-off-by: Daniel Thompson <daniel@riscstar.com>
|
|
write_mem_msg() passes the length claimed by an 'M' or 'X' packet directly
to kgdb_hex2mem() or kgdb_ebin2mem(). A malformed packet can claim more
data than was actually received, causing the decoders to read past
remcom_in_buffer.
Record the received payload length in get_packet() and use it to bound
the decoders. 'M' carries two hex characters per byte, while 'X' carries
one byte per output byte except that a 0x7d escape consumes an additional
input byte. Pass the packet end to kgdb_ebin2mem() so both reads are
checked against the received data.
Keep zero-length 'X' packets valid, as GDB uses them to probe binary
download support.
Fixes: dc7d55270521 ("kgdb: core")
Cc: stable@vger.kernel.org
Assisted-by: Hawkeye:GLM-5.3-flash
Assisted-by: Qoder:Qwen3.8-Max
Signed-off-by: Fang Xieyan <fangxy@xiaopeng.com>
Link: https://lore.kernel.org/r/20260922154907.90535-1-fangxy@xiaopeng.com
Signed-off-by: Daniel Thompson <daniel@riscstar.com>
|
|
gdb_cmd_memread() passes the length of the 'm' packet straight to
kgdb_mem2hex(), which hex-encodes the reply into
remcom_out_buffer[BUFMAX] without checking that the reply fits. A
client on the KGDB I/O console can ask for a read of any size, and
the hex encoding runs off the end of the buffer. kgdb_mem2hex() takes
that length as an int count, so 2^31 truncates to a negative value
and copy_from_kernel_nofault() is handed buf + count as its
destination.
On x86 BUFMAX is 1024, so a read of 512 bytes overruns by the single
byte that terminates the hex string:
BUG: KASAN: global-out-of-bounds in kgdb_mem2hex+0x1dd/0x230
Write of size 1 at addr ffffffff87651780 by task sh/1
...
The buggy address belongs to the variable:
remcom_out_buffer+0x400/0x420
Bounds check the length before the hex encoding:
if (length > (BUFMAX - 1) / 2) {
error_packet(remcom_out_buffer, -EINVAL);
return;
}
gdbstub_msg_write() already bounds the count it hex-encodes
against BUFMAX. Reads over the limit never returned valid data,
so an error reply loses nothing.
Fixes: dc7d55270521 ("kgdb: core")
Cc: stable@vger.kernel.org
Assisted-by: Hawkeye:GLM-5.3-flash
Assisted-by: Qoder:Qwen3.8-Max
Signed-off-by: Fang Xieyan <fangxy@xiaopeng.com>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
Link: https://lore.kernel.org/r/20260914180344.15485-1-fangxy@xiaopeng.com
Signed-off-by: Daniel Thompson <daniel@riscstar.com>
|
|
Remove the dead 'return 1' after panic() in kgdb_reenter_check().
panic() is marked __noreturn so the subsequent return statement can
never execute.
Signed-off-by: Naveen Kumar Chaudhary <naveen.osdev@gmail.com>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
Link: https://lore.kernel.org/r/uddjzmr3oqbf2ly5iyx4ajomreub7firbrzwinhj35ppix6jlo@tt6ejkxmk4tj
Signed-off-by: Daniel Thompson <daniel@riscstar.com>
|
|
CMD_BUFLEN was defined separately in kdb_io.c (256) and kdb_main.c
(200), causing kdb_main.c to use the wrong size when formatting the
prompt string into kdb_prompt_str (which is 256 bytes).
Move CMD_BUFLEN (256) into kdb_private.h so all users share a single
consistent definition, and remove the local definitions from both
files.
Fixes: 5d5314d6795f ("kdb: core for kgdb back end (1 of 2)")
Signed-off-by: Naveen Kumar Chaudhary <naveen.osdev@gmail.com>
Link: https://lore.kernel.org/r/l6b6bomrw3bvkehlv73fyghk6loz6jpdwqk5c4nthnhl7j7kzk@4jjhfpzv5zon
Signed-off-by: Daniel Thompson <daniel@riscstar.com>
|
|
When kdb_setenv() fails to find a free slot in __env[], the function
returns KDB_ENVFULL without freeing the previously allocated string
buffer 'ep', leaking memory on every failed call.
Add the missing kfree(ep) before returning KDB_ENVFULL.
Fixes: a30d4ff8193e ("kdb: remove usage of static environment buffer")
Signed-off-by: Naveen Kumar Chaudhary <naveen.osdev@gmail.com>
Reviewed-by: Douglas Anderson <dianders@chromium.org>
Link: https://lore.kernel.org/r/mbwhul34esbxkdbmlbjaquv5lgyzznn6pdapohry3lsiidbz6n@zucvm5os363q
Signed-off-by: Daniel Thompson <daniel@riscstar.com>
|
|
add_subprog() appends new subprog to env->subprog_info[] and calls sort()
on the whole array, though only the last entry can be out of order.
Move subprogs that start after the new one up by one entry instead.
'start' is unique, so the order of the array doesn't change. The new entry
can land in a used slot, so initialize all of it.
add_subprog() returns the index of the new subprog instead of the last
one. The callers check for error only.
It's a trivial prep for rust-bpf [1] that bumps BPF_MAX_SUBPROGS to 1024.
Rust crates have more than 256 functions that are not inlined.
Time in add_subprogs() when veristat loads sched_ext schedulers from scx
repo, 8 cpu VM, usec:
subprogs before after
arena_selftest 101 386 27
lavd_enqueue 52 96 16
lavd_dispatch 48 78 11
[1] https://github.com/4ast/rust-bpf
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Reviewed-by: Mahe Tardy <mahe.tardy@gmail.com>
Link: https://patch.msgid.link/20261023150302.4097865-1-alexei.starovoitov@gmail.com
|
|
btf_add_type() stores the types of a BTF in btf->types[], which can hold at
most BTF_MAX_TYPE entries. The types of a split BTF get IDs start_id ..
start_id + nr_types - 1, so only BTF_MAX_TYPE - start_id of them fit, but
neither the limit check nor the size used to grow the array accounts for
start_id: the check is an equality test against BTF_MAX_TYPE
('start_id + btf->types_size == BTF_MAX_TYPE') and the new size is clamped
to BTF_MAX_TYPE.
The check therefore only fires if the array happens to grow to exactly
BTF_MAX_TYPE - start_id entries. Otherwise, if the module has more than
BTF_MAX_TYPE types, the array is filled up to BTF_MAX_TYPE entries. The
next type that needs to be added reallocates the array to the same size,
after which btf->types_size no longer equals btf->nr_types and the growth
path is not taken anymore, so every following type is written past the end
of the array:
BUG: unable to handle page fault for address: ffffc90006200000
RIP: 0010:btf_add_type+0x74/0x270
Call Trace:
btf_check_all_metas
btf_parse_module
Bound both the array size and the limit check by BTF_MAX_TYPE - start_id.
Base BTF (start_id == 0) is unaffected.
Fixes: 951bb64621b8 ("bpf: Add in-kernel split BTF support")
Signed-off-by: Feng Yang <yangfeng@kylinos.cn>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Reviewed-by: Mahe Tardy <mahe.tardy@gmail.com>
Acked-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Link: https://patch.msgid.link/20261028031933.277550-1-yangfeng59949@163.com
|
|
When a register is -1 or 0, maybe_fork_scalars() splits the state in two
before an 'and' or an 'or' with a constant. The result is more accurate for
each value alone. The verifier continues with -1. It pushes a copy of the
state with 0 and verifies the copy later, from the same insn.
today this patch
N: r1 &= 8 r1 is -1 or 0 N: r1 &= 8 r1 is -1 or 0
/ \ / \
r1 = -1 r1 = 0 r1 = -1 r1 = 0
r1 &= 8 push to N r1 &= 8 r1 &= 8
| | push to N+1
N+1: ... N+1: ...
At insn N the copy is a part of the state that it came from: 0 is in
[-1, 0]. If pruning compares the two states, it drops the copy, and no one
verifies the case r1 = 0.
The first state has branches that are not explored. Usually pruning does
not use such states. In a loop that scalar evolution counted it does, and
the head of such a loop always keeps its state. Thus a fork at the head of
a counted loop loses the case r1 = 0:
r9 = a map value of 8 bytes
r6 = -1
1: r6 &= 8 r6 is -1 or 0, only -1 is verified: 8
r6 ^= 8 0. It is 8 when r6 was 0.
r1 = r9
r1 += r6
r0 = *(u8 *)(r1 + 0)
r6 = -1 or 0
r7 += 1
if r7 < 100 goto 1b
from 11 to 11: safe
The verifier accepts this prog. The prog reads the byte that follows the
map value. With a constant of 64 Tbyte in a register it reads an address
that is not mapped, and the kernel crashes.
Do the insn in the copy too, and push the copy to the next insn. All other
code that pushes a state does the same.
No verdict changes in selftests, Cilium and scx. Each fork processes one
insn less, and the sequence in which the verifier explores states changes
slightly. 62 of 161 progs of Cilium process a different number of insns,
from -0.7% to +3.3%. The total is 986970 -> 987013. 2 of 563 progs of scx
process one insn less.
Fixes: 6ccf49f79327 ("bpf: Use SCEV to widen bounded loops")
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261027190314.1440199-17-alexei.starovoitov@gmail.com
|
|
A compare with pkt_end sets 'range' in all packet pointers that have the
same id. Then 'range - offset' bytes from the pointer are in the packet.
When a state returns to the head of a counted loop, bpf_clamp_scev_regs()
gives a new id to a packet pointer, and the offsets of all iterations. It
does not change the range. But the range is valid only together with the
old id and the old offsets.
1: ; R6=pkt(id=1,r=0,off=0..100)
r5 = r6
r5 += 100
if r5 > pkt_end goto 2f ; R6=pkt(id=1,r=200,off=0..100)
r6 += 100 ; R6=pkt(id=1,r=200,off=100..200)
r9 += 1
if r9 < 2 goto 1b ; clamp: R6=pkt(id=2,r=200,off=0..100)
after bytes from r6 that the verifier accepts
---------------------- ---------------------------------------
the compare 100, correct
r6 += 100 0, correct
the clamp 100, incorrect. Nothing compared this r6
Usually the verifier prunes the state at the head, and the range has no
effect. The verifier continues with the state when a different part of it
is not within the state at the head. A flag in a 32-bit spill is
sufficient. Then the prog reads and writes after pkt_end. A write to
skb_shared_info crashes the kernel.
Take the range that the pointer has at the entry of the loop. No compare in
the loop found this range, so it is valid in all iterations. The state at
the head has the same range, so the verifier prunes as before.
Fixes: 6ccf49f79327 ("bpf: Use SCEV to widen bounded loops")
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261027190314.1440199-15-alexei.starovoitov@gmail.com
|
|
When a state returns to the head of a counted loop, bpf_clamp_scev_regs()
replaces the range of each register that changes by a constant step. The
new range is the range over all iterations. The function calculates it from
the register in the state that entered the loop. For a pointer this range
is the range of its offsets.
A call in the loop can invalidate the pointer. bpf_skb_pull_data() does it
to packet pointers, bpf_ringbuf_discard() does it to ringbuf memory. Then
the register is an unknown scalar. Scalar evolution does not see that, and
the clamp gives the offsets of the pointer to the scalar.
r6 = bpf_ringbuf_reserve() ; R6=ringbuf_mem
r7 = r6
1: if first iteration goto 2f
r1 = map_value
r1 += r6 ; accepted: R6 is 0 or 1
r0 = *(u8 *)(r1 + 0)
goto 3f
2: bpf_ringbuf_discard(r7) ; R6=scalar()
3: r6 += 1
r9 += 1
if r9 < 2 goto 1b ; clamp: R6=scalar(umax=1)
At run time r6 is the address of the ringbuf memory plus 1. The prog adds
it to a pointer to a map value, and the kernel crashes in the access.
Do not clamp a register that has a different type than at the entry of the
loop. Such a state is not within the state at the head. Thus the verifier
continues with it, and r6 stays an unknown scalar.
Fixes: 6ccf49f79327 ("bpf: Use SCEV to widen bounded loops")
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261027190314.1440199-14-alexei.starovoitov@gmail.com
|
|
Scalar evolution widens counted loops. Since that change the verifier
rejects a loop over the per-cpu values of a map element:
SEC("iter/bpf_map_elem")
...
for (i = 0; i < n; i++) {
out[i] = *(__u32 *)pptr;
pptr += 8;
}
loop header at 12, widening r2 to 0..56 step 8
R2 invalid variable buffer offset: off=0, var_off=(0x0; 0x38)
pptr is a pointer to a buffer (PTR_TO_BUF). Scalar evolution replaces its
offset with the range over all iterations. But __check_buffer_access()
accepts only a constant offset. Thus a prog cannot use a widened
PTR_TO_BUF.
Do not widen it. Then the verifier verifies the loop one iteration at a
time, as it did before.
loop before SCEV with SCEV this patch
-------------------------------- ----------- ---------- ----------
counts in i, reads through pptr loads rejected loads
counts in p ('p < end'), no read 1M insns loads 1M insns
counts in p, reads through p 1M insns rejected 1M insns
The verifier cannot decide a comparison of two pointers to a buffer. Thus
it follows 'p < end' until the limit of 1M insns.
bpf_iter_bpf_percpu_array_map.c and bpf_iter_bpf_percpu_hash_map.c have the
first loop. They load today because num_cpus is volatile. The prog reads
the bound again in each iteration. Scalar evolution cannot count such a
loop, and it widens nothing.
Fixes: 6ccf49f79327 ("bpf: Use SCEV to widen bounded loops")
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261027190314.1440199-12-alexei.starovoitov@gmail.com
|
|
After 'r3 = r8; r3 += 4' the verifier knows that r3 == r8 + 4. The two
registers have the same id, and r3 keeps the difference in reg->delta. When
the prog compares one of them with a constant, sync_linked_regs()
calculates the range of the other register from the difference of the two
deltas.
At a loop head scalar evolution widens a register: it replaces the register
with its range over all iterations. Then the register is not linked to
other registers. scratch_widened_reg_id() clears reg->id, but it does not
clear reg->delta. A subsequent copy of the register gets the stale delta.
Then the range of the copy is incorrect by this delta.
r3 = r8
r3 += 4 ; R3=scalar(id=1+4)
1: ; r3 is widened: no id, delta is still 4
r4 = r3 ; id=2 and delta 4 in both
r4 += 1 ; R4=scalar(id=2+1)
if r3 > 10 goto 3f ; R3 <= 10, R4 = R3 + 1 - 4 <= 7
if r4 <= 7 goto 2f
... ; not verified, r4 is up to 11 here
The verifier does not verify code that can run. Use clear_scalar_id(). It
clears id and delta together. Move it to bpf_verifier.h, so that scev.c can
call it.
Fixes: 6ccf49f79327 ("bpf: Use SCEV to widen bounded loops")
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261027190314.1440199-8-alexei.starovoitov@gmail.com
|
|
Rust has no address spaces. Thus a prog in Rust accesses arena through
scalars. With BPF_F_ARENA_SCALAR the verifier accepts each load and each
store through a scalar as an access to arena.
The verifier rewrites one insn at a time. Thus it must add one insn to each
access. The compiler knows the source of each pointer. It can emit one
addr_space_cast for each pointer, as it does for the __arena pointers of C.
flag: one move for each access cast: one cast for each pointer
ax = (u32)r1 r1 = addr_space_cast(r1, 0, 1)
r2 = *(u64 *)(ax + 0) r2 = *(u64 *)(r1 + 0)
ax = (u32)r1 r3 = *(u64 *)(r1 + 8)
r3 = *(u64 *)(ax + 8)
20 progs of rust-bpf, with the flag and with the casts that a prototype of
an LLVM pass emits:
flag casts
moves or casts 1568 571
insns 14864 13938
jited bytes 72351 69111
No released kernel has the flag. LLVM does not emit the casts for Rust at
this time. Until it does, Rust progs that access arena do not load.
scx_mavd and scx_nitosis set the flag too. They are experimental schedulers
in C in sched-ext/scx, and they use the flag to omit __arena. They must use
__arena again.
This reverts commit 4c651a91bdfc
("bpf: Treat load and store through a number as arena access"). One part
stays: BPF_REG_AX in reg2pt_regs[] of the x86 JIT. The fetching AND/OR/XOR
atomics in arena use it now.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261027190314.1440199-6-alexei.starovoitov@gmail.com
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
Pull tracing updates from Steven Rostedt:
- Add "within" to filter logic
Add a way to trace events by what function they fall in.
echo 'within == "vfs_read" > events/sched/sched_switch/filter
Will only trigger sched_switch events that happen within the vfs_read
function. It uses a stacktrace at the event and checks if any entry
is within the specified function and if so, it prints the event.
- Remove a duplicate definition of ftrace_set_clr_event() from the
header
- Do not have ftrace_set_clr_event() modify its input buffer
Have the buffer passed into ftrace_set_clr_event() to be const char *
and not allow modifications to it that might cause side effects.
- Add kunit tests for ftrace_parse_event_string()
- Add filter argument to __ftrace_set_clr_event()
Make it possible to set a filter on an event at the same time it is
enabled.
- Add event filtering logic to the kernel command line
To allow filtering events on boot, allow filtering to be specified on
the kernel command line when enabling events.
- Rename "ring_buffer_size()" to "ring_buffer_capacity()"
"size" refers to how much is allocated whereas the actual return
value is the amount of available data in the ring buffer minus its
meta data.
- Rename "trace_array_entries" to "trace_array_capacity"
There's several functions that use "entries" and returns the amount
of data that that is usable. "entries" should be only for the number
of events, not the amount of data.
- Print the histogram "bucket" size as unsigned.
If a bucket size is LONG_MAX, then it shows up as a negative number.
- Show bucket size in output of histograms
The output of what the histogram is shown doesn't show the size of
the buckets when buckets are used. The output should be the same as
the input and since specifying buckets without a size is an invalid
command, the output to display what was entered should also show the
size.
- Fix stacktrace modifier output
Only show the stacktrace modifier when it was specified in creation.
- Unregister filed variable histograms when the command fails
If one histogram references a variable of another histogram it
registers that other histogram. But if the first histogram fails in
setup after it registered the other histogram it doesn't unregister
it. Have the first histogram add the reference to the other histogram
so the failure clean up will unregister the second histogram too.
- Fix 64-bit signed histogram fields on 32-bit kernels
A fix to convert char * to u64 with added casting also incorrectly
added an intermediate cast to convert a long value to u64 which ended
up truncating the upper 32 bits for 32-bit kernels. Remove the
intermediate typecast.
- Print signed histogram keys as signed
Histogram keys were being printed as unsigned and with signed keys it
was giving the wrong output.
- Use copy_from_user() and not __copy_from_user() in syscall tracing
The syscall tracing incorrectly was using __copy_from_user() which
could be an issue if the system trusts perf doing the trace.
- Take trace_types_lock when recording a new module in persistent
buffer
When a new module is loaded, the persistent ring buffer saves it in
its persistent memory. All the trace_arrays are iterated looking for
any persistent buffer to add the module to. But it did not hold the
trace_types_lock meaning a trace_array could be added or removed
while iterating the list.
- Only create tracepoint guest_halt_poll_ns when it is used
The tracepoint guest_halt_poll_ns is defined in the power management
code but it is only used when CONFIG_CPU_IDLE_GOV_HALTPOLL is
enabled. This wastes memory when that config isn't enabled so wrap
the tracepoint around that as well.
- Handle empty fields for events in trace_event_get_offsets()
The user events could have empty fields and this causes
trace_event_get_offsets() to crash as it expects events to always
have fields. Make the function handle events with no fields.
- Reject incomplete event filter expressions
If a filter has an incomplete expression (common_pid == 1111 || ), it
will cause a out-of-bounds bug. Have the predicate_parse() only
process the expression if it is complete.
* tag 'trace-v7.4' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace: (27 commits)
tracing: Reject incomplete event filter expressions
tracing: Guard trace_event_get_offsets() against an empty fields list
cpuidle: haltpoll: Only define guest_halt_poll_ns tracepoint when used
tracing/osnoise: Add __printf attribute to osnoise_print()
tracing/histograms: Verify event_call identity when updating synth_event refcount
tracing: Save the event file instead of the compatible histogram
tracing: Hold trace_types_lock when resetting last boot data from readers
tracing: Hold trace_types_lock when updating modules in persistent instances
tracing/syscalls: Use copy_from_user for the faultable syscall buffer
tracing: Print a signed histogram key as signed
tracing: Fix 64-bit and signed histogram fields on 32-bit kernels
tracing: Clamp the printed end of a histogram bucket at U64_MAX
tracing: Unregister field variable histograms when the command fails
tracing: Only report the stacktrace modifier on a real field
tracing: Add the bucket size to expr_field_str()
tracing: Print the bucket size as unsigned
tracing: Rename tracing_entries_fops to tracing_capacity_fops
tracing: Rename trace_array::entries to capacity
ring-buffer: Rename ring_buffer_size() to ring_buffer_capacity()
tracing: Parse filter from boot event string
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
Pull more MM updates from Andrew Morton:
- "mm: make VMA flag semantics explicit, eliminate VM_SPECIAL" (Lorenzo
Stoakes):
Replace VM_SPECIAL with helpers describing mapping ownership and
behavior. Enforce the mmap flag rules and convert affected drivers,
removing ambiguous tests shared by driver mappings and ordinary
memory.
- "support kselftest on nommu platform" (Hajime Tazaki):
Add opt-in mmap and mremap tests for NOMMU systems. Teach the runner
to avoid its fork-dependent timeout command and document the build
and runtime requirements.
- "mm/collapse: separate a collapse from its callers" (Kiryl
Shutsemau):
Separate huge-page collapse from caller policy. Replace khugepaged
identity checks with explicit permissions, and separate scanning from
collapse with the mmap_lock drop in one place.
- "mm: implement and use vma_has_anon_rmap(), silence KCSAN" (Lorenzo
Stoakes):
Provide a common helper for testing whether a VMA has anonymous
reverse mappings. Document the locking rules, annotate intentional
concurrent reads for KCSAN, and convert the core MM callers.
- "mm/sparse: remove SECTION_MARKED_PRESENT and further cleanups"
(David Hildenbrand):
Remove SECTION_MARKED_PRESENT in favor of the early-section flag.
Simplify sparse memory initialization, section metadata handling and
related PFN validity checks.
- "sh: remove NUMA and SPARSEMEM support" (Mike Rapoport):
Remove SuperH NUMA and SPARSEMEM support. The extra memory banks have
been unusable since the NO_BOOTMEM conversion, leaving only the code
and data overhead.
- "mm: Unify device DAX and HugeTLB vmemmap population paths" (Muchun
Song):
Unify the generic device DAX and HugeTLB vmemmap population paths and
shared tail-page handling. Remove the duplicate DAX machinery while
retaining the powerpc radix implementation.
- "kselftest: mm: fix some failure of split_huge_page_test" (Yeoreum
Yun):
Fix huge-page split tests racing with khugepaged recollapse. Use
pagemap and page flags instead of VMA-boundary assumptions which fail
when glibc has already enabled THP.
- "memcg: move memcgid refcount to objcg to unpin dying memcgs"
(Bingfang Guo):
Let dying memcgs be freed while swapped-out shmem still holds their
IDs. Move ID references to reparentable objcgs, with the necessary
swap accounting and RCU lifetime changes.
- "mm/truncate: fix data loss when truncating straddling large folios"
(Zhang Yi):
Preserve out-of-range data when truncating straddling large folios.
Fix a lookup race and split-failure handling, and handle minimum
folio orders for the forthcoming ext4 iomap buffered I/O path.
- "mm/damon: introduce damos quota goal target metric complement flag"
(SJ Park):
Let DAMOS quota goals use a metric's complement, so utilization can
also express free-memory goals without adding separate metrics. Add
the sysfs interface, documentation and tests.
- "arm64/mm: Standardize printing for pgtable entries" (Anshuman
Khandual):
Use the generic formatter for arm64 page-table entries, ready for
future wider entries. Move the safe-PTE-update check out of the
header to avoid an include cycle.
- "mm: kmsan: fix iounmap metadata teardown" (Dima Koziuk):
Fix KMSAN metadata leaks on iounmap and failed ioremap allocations.
Free backing pages before removing their mappings, and share the
teardown with partial-allocation cleanup.
- "mm/damon/core: preserve quota state when constructing schemes" (Karl
Mehltretter):
Preserve running DAMOS quota state while validating parameter
updates. Even rejected updates could reset accounting and exceed the
quota window; add tests for both accepted and rejected updates.
- "mm/vmalloc: fix vmalloc_dump_obj VA lookup" (Ye Liu):
Fix vmalloc_dump_obj() lookups for last-page addresses and
allocations spanning vmap zones. These allocations were being missed
in diagnostic output.
- "mm: various (urgent?) fixes" (David Hildenbrand):
Fix mmap charge leaks and hugetlb use-after-free, wrong-address
unmapping, stuck locks and endless faults. Also preserve reservations
on failed mremap and handle stale userfaultfd bits.
- "maple_tree: fix maple_range_64 crashes in RCU mode" (Dimitris
Charisis):
Fix pointer and metadata confusion during RCU destruction of
maple_range_64 nodes, avoiding double frees and invalid accesses. Add
a regression test; no in-tree user currently uses this combination.
- "mm: fix hugetlb NR_HUGETLB accounting on folio migration" (Hongfu
Li):
Fix hugetlb migration and preallocation accounting so /proc/vmstat
retains the correct count and memory.numa_stat follows the folio to
its new node.
- "mm/damon: fix a minor bug, clean code, and add a test" (SJ Park):
Fix 32-bit DAMON age overflow which made old regions look young.
Share hugetlb folio acquisition code and add a regression test for
the previously fixed last-region skipping bug.
- "mm/damon/core: cleanup code, reduce stack usage, and add kunit" (SJ
Park):
Reduce kdamond stack usage and remove unused enums. Add deterministic
PSI quota-goal tests which do not depend on the machine's current
memory pressure.
- "mm/damon: fix the temporal goal tuner's size quota conversion"
(Donggeun Yoo):
Bound the temporal tuner's quota conversion to prevent overflow and
zero effective quotas, which could stop schemes making progress. Add
boundary tests.
- "Don't use GFP_DMA when calling dma_alloc_coherent" (Baoquan He):
Remove redundant GFP_DMA flags from coherent DMA allocation calls.
The DMA API already selects the zone from device limits and ignores
the caller's flag.
- "kselftest: mm: fix intermittent failure khugepaged test" (Yeoreum
Yun):
Propagate child-process failures in the khugepaged tests and prevent
background collapse from racing their checks. This removes false
passes and intermittent failures.
- "MAINTAINERS: cover all hugetlb headers under HUGETLB SUBSYSTEM"
(Enze Li):
Cover all hugetlb headers in MAINTAINERS so get_maintainer.pl finds
the right maintainers. Also label the closing hugetlb_cgroup.h
include guard.
Summary of the 65 singletons in this merge:
The singletons include a few worthwhile performance fixes. Avoiding
unnecessary reverse-map tree updates gave a reported 15.9% improvement
in UnixBench's execl test on a 192-core x86 system. A reclaim retry
fix stops defrag_mode from spinning under memory.low protection; a
reported file-creation stress test ran 5.7 times faster, with far less
memory pressure. Narrower hugetlb invalidations also spare KVM guests
needless refaults: the reported Redis-in-VM P99 QPS penalty while
punching holes fell from 7.09% to 1.45%.
Several fixes prevent lost data or corrupted memory. Lazyfree
huge-page races with mremap()/munmap() could lose data or access freed
memory, and MADV_COLD could lose a concurrent write's dirty bit.
Another fix prevents stale migration entries from outliving large
folios. On filesystems with blocks smaller than a page, extending a
file by 4 GiB or more could expose bytes written past the old EOF
through mmap instead of zeroes. That is fixed, along with
use-after-free races in PAGEMAP_SCAN during process exit and in
memory-cgroup accounting on PREEMPT_RT kernels.
Other fixes stop UFFDIO_MOVE spinning forever on pinned large folios,
which could wedge other memory operations, and stop mremap() leaking
commit charges until allocations fail under strict overcommit. A
further fix keeps userfaultfd tracking intact on neighboring mappings.
NUMA binding no longer prevents long-term pins when the chosen node
has only movable memory, fixing QEMU/VFIO startup and io_uring buffer
registration failures. MGLRU also gets a fix for early-boot crashes
with memory cgroups disabled at boot, notably in small crash kernels.
* tag 'mm-stable-2026-10-25-19-27' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (213 commits)
mm: multi-gen LRU: fix NULL lruvec->pgdat deref with memcg disabled on cmdline
mm/page_counter: use WRITE_ONCE() for emin/elow when parent is the root
mm/memcontrol: publish null pointer before dropping css reference
test_maple_tree: test maple_range_64 node in RCU mode
maple_tree: fix maple_range_64 crashes in RCU mode
mm/vma: avoid broken uffd merge on mremap
tools/testing/shared: catch a failed posix_memalign()
mm/damon/core: remove unreachable damos charge_{target,addr}_from
mm/shrinker: allocate shrinker_info_unit from a dedicated kmem_cache
riscv: mm: exclude invalid THP PMDs from page table check
mm/damon/core: filter out damos address filter-split failed region
fs/proc/task_mmu: pin mm_users in do_pagemap_scan()
mm/mempolicy: apply bind policies against the scoped allocation zone
mm/vmscan: reclaim every zone in per-node proactive reclaim
mm/mremap: correctly uncharge accountable VMAs in error path
mm/mremap: populate calculated fields in vrm state correctly
MAINTAINERS: add an entry for MEMORY MANAGEMENT - MEMORY MAP
mm: bdi: reset min_ratio after the sysfs attributes are gone
mm/page_vma_mapped: continue past nonmatching huge PMDs
selftests/mm: skip hugetlb cgroup v2 tests without the needed controllers
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
Pull non-MM updates from Andrew Morton:
- "ocfs2: exit recovery thread on mount error path" (Joseph Qi):
Stop OCFS2 recovery before freeing failed-mount state and replay
slots, closing use-after-free and double-free races. Defer
suballocator reclaim to break a transaction-lock deadlock with
dismount.
- "ocfs2: validate suballoc slot and bit of metadata blocks" (Joseph
Qi):
Validate OCFS2 metadata suballocator slots, bits and directory index
block locations. Reject corrupt metadata with filesystem errors
instead of BUGs or out-of-bounds accesses.
- "ocfs2: deal with legacy signed name hash values" (Joseph Qi):
Restore OCFS2 lookups for non-ASCII names stored with legacy signed
hashes. Use stored xattr hashes during bucket splits to avoid
duplicate entries and spurious ENOSPC failures.
- "ocfs2: stabilize dentry names in file and setattr traces" (Cen
Zhang):
Snapshot OCFS2 names during file and setattr tracing to prevent
rename races using freed storage or mismatched names and lengths.
Preserve trace formats and the locked release open-count sample.
- "panic: fix panic_force_cpu= redirect races and NMI bypass" (Bradley
Morgan):
Fix panic_force_cpu races and NMI redirection so crash-kernel CPU
selection is honored. Preserve variable arguments on failed redirects
and use a static message buffer.
- "fat: allocate multiple clusters in fallocate" (Matheus Alves de
Almeida):
Allocate FAT clusters in batches for FALLOC_FL_KEEP_SIZE and batch
FAT updates. This removes repeated single-cluster allocation
overhead, particularly on synchronous mounts.
- "scripts/gdb: fix radix tree lookup and stack depot diagnostic"
(Ting-Han Hou):
Fix GDB radix-tree lookup and its handling of XArray sibling, retry
and zero entries. Also repair the stack-depot diagnostic for an
out-of-range pool index.
- "hung_task: Improve warning budget handling and task reporting"
(Aaron Tomlin):
Reset the hung-task warning budget after recovery so later hangs
remain visible. Keep the configured limit intact and emit one summary
per scan when detailed reports exhaust the budget.
- "bootconfig: Reject unexpected data after null character and
cleanups" (Masami Hiramatsu):
Reject bootconfig data after an embedded null while accepting
trailing padding. Validate the full input in the userspace tool, and
retain internal tree sanity checks there rather than in the kernel.
- "kbuild: GCOV cleanups following the prefer-atomic fix" (Konstantin
Khorenko):
Use the actual GCOV build flags in the prefer-atomic compiler probe.
Move flag setup into scripts/Makefile.gcov and skip the probe when
GCOV is disabled.
- "ocfs2: xattr bucket validation fixes" (Joseph Qi):
Accept valid OCFS2 xattr buckets spanning 512-byte blocks, avoiding
false read-only remounts. Reject misaligned buckets and return errors
instead of panicking on corrupt defrag metadata.
- "ocfs2: fix metadata reservation for xattr value trees" (Joseph Qi):
Reserve enough OCFS2 metadata for new and relocated xattr value
trees. Fix shortfalls which could cause ENOSPC, dropped entries and
leaked clusters on fragmented filesystems; remove duplicate
reservations.
- "Introduce arch_do_panic" (Mete Durlu):
Add arch_do_panic() and move s390 and sparc handling out of generic
panic code. The architecture-specific actions now follow the
end-of-panic marker.
- "checkpatch: userspace improvements" (Petr Vorel):
Improve checkpatch's userspace detection and suppress kernel-only
checks. Recognize standard inttypes macros and add explicit
--userspace and --no-userspace overrides.
- "ocfs2: validate dr_fs_generation of dir index root blocks" (Joseph
Qi):
Check OCFS2 directory index generations and leaf block numbers.
Reject stale or misplaced blocks instead of using metadata left by a
previously formatted filesystem.
- "selftests/membarrier: Skip an unregistered memory barrier test on
Musl" (Chris Gellermann):
Skip the unregistered membarrier test when libc has already
registered the command. Musl does this during thread creation,
causing false failures in the multithreaded test.
- "init: fix array boundary bugs in boot parameter parsing"
(Wilson Felipe Pereira):
Fix boot argument boundary cases which leave stale arguments in
init's command line or panic when replacing the final environment
variable. Clear the last slot and check duplicates before the limit.
Summary of the 87 singletons in this merge:
The OCFS2 singletons fix deadlocks between xattr updates and inline
truncation, and during inode reclaim. Corrupt quota and extent
metadata could also crash the kernel or corrupt memory; these paths
now check the layout before using it. Other fixes close use-after-free
and publication races around cluster reconfiguration, node failure,
quota remount errors and shared refcount trees. New files now retain
security labels from all stacked LSMs instead of silently dropping all
but one. Valid xattr insertions also stop tripping FORTIFY on builds
with counted arrays.
Elsewhere, a lost wakeup could leave hardware-monitoring reads stuck
in unkillable sleep and block further I2C traffic. That is fixed, as
are kernel memory leaks in POSIX message-queue notifications and tmpfs
casefold setup which could grow until memory is exhausted. A driver
removal race which could crash the kernel is fixed too. The
hibernation command through reboot() also stops taking the same mutex
twice, avoiding a deadlock before suspension.
On the boot side, bzip2 decoding now accepts valid full blocks emitted
by lbzip2, fixing failures to boot those compressed kernels. Malformed
bzip2 streams can no longer overflow the decode buffer, and LZ4
initrds are checked before the decoder can read past their end. FAT
rejects geometry which overflows the data-area calculation on 32-bit
systems, and rolls back entries when inode setup fails, preventing
failed creations from reappearing after remount. The x86 XOR and RAID6
AVX paths also avoid imposing a slowdown on subsequent SSE code.
* tag 'mm-nonmm-stable-2026-10-25-14-57' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (134 commits)
scripts/gdb: fix stack depot out-of-bounds diagnostic
scripts/gdb: fix radix tree lookup pointer handling
ocfs2/cluster: remove zero-address nodes from IP tree on teardown
ocfs2: snapshot the dentry name while tracing file release
ocfs2: snapshot dentry names in file and setattr traces
ocfs2/dlm: drain join handlers before releasing module text
MAINTAINERS: update Markus's e-mail address
mailmap: add entries for Andrew Jones
stackdepot: stop preallocating after the final pool
ocfs2: drain quorum work before removing the cluster
watchdog: force console verbose when reporting lockups
profiling: fix sampling step on 64-bit big-endian systems
kernel/reboot: avoid potential deadlocks during suspension
compat: avoid double rounding in times()
ocfs2: pin owning modules for open debugfs files
ocfs2/cluster: initialize heartbeat regions before publishing them
selftests/proc: proc-pid-vm: fix unused variable warning on non-x86
ocfs2: pin refcount trees before dropping the lookup lock
ocfs2: validate the xattr tree root of an indexed xattr block
ocfs2/dlm: hold the lookup reference across convert AST enqueue
...
|
|
A tracing program can call scx_bpf_sub_grant() and scx_bpf_sub_revoke()
through scx_kfunc_ids_any, which is registered on the tracing kfunc hook.
Before returning, both invoke ops.sub_caps_updated() of the sub-schedulers
whose caps changed, so those ops run in the tracing program's context. No
SCX op expects to run from an arbitrary tracing context.
With an rq lock held, this leads to the same deadlock that b07a7450f6ef
("sched_ext: Reject rq-lock-taking kfuncs in tracing programs") fixed for
direct calls:
1. A tp_btf/sched_switch program fires from __schedule() with the rq lock
held. scx_locked_rq() only tracks rq locks held across SCX callbacks, so
it returns NULL there.
2. The program calls scx_bpf_sub_grant() on a child, which invokes the
child's ops.sub_caps_updated() before returning.
3. That op calls scx_bpf_cidperf_set() on this cpu, which sees no locked rq
and takes the rq lock the cpu already holds.
Add both kfuncs to scx_kfunc_ids_no_tracing and rewrite the set's comment to
cover both reasons for membership: taking an rq lock and running SCX ops.
Fixes: 5f2a9a4c2e6c ("sched_ext: Add coalescing sub_caps_updated() notifier for sub-schedulers")
Signed-off-by: Tejun Heo <tj@kernel.org>
|
|
An earlier patch sends a bypassed task to rescue when its host lacks
SCX_CAP_BASE on the cpu. The task's rescue time is charged to its owner, the
bypassing sub, which is usually exiting. The overload pick skips exiting
subs and ejects an unrelated sub or nobody. The host runs the task and lacks
BASE, but carries none of the charge.
Charge the sched running the rescuee, the host while the owner bypasses.
SCX_EV_SUB_RESCUE already counts on that sched and the stall watchdog blames
it.
Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
Since an earlier patch sends a bypassed task to rescue when its host lacks
SCX_CAP_BASE, a task scx_fail_parent() punted to its bypassing parent at
NONE can be a rescuee.
When a cgroup migration into a root-owned cgroup re-homes it while running,
nothing resets it: the DEQUEUE_SAVE dequeue and put_prev_task_scx() keep a
running task's slice, and __scx_disable_and_exit_task() skips a NONE task.
rq->scx.rescue.curr keeps pointing at the task, now on the root. An RT task,
for example, can keep the task from serving its grant until the last sub is
freed. From then on scx_rescuee() returns NULL and the rescue never ends.
scx_rescue_timerfn() keeps accessing the task after it is freed.
Move the reset out of scx_disable_task() into scx_reset_task_fields() and
call it on every path that takes a task out of its sched's control, the exit
of a NONE task included.
Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
While dispatch is in progress, rq_is_open() returns true and dsq_inc_nr()
schedules no reenqueue for an IMMED task inserted on an empty local DSQ. The
check at the end of dispatch_one() schedules the reenqueue only when more
than one task is queued, so a dispatch that keeps prev leaves a lone IMMED
task behind:
1. a parent's ops.dispatch() sub-dispatches into a child, which consumes a
task with SCX_ENQ_IMMED onto the empty local DSQ
2. the parent extends prev's slice and dispatch keeps prev
3. the local DSQ holds one task, so no reenqueue is scheduled
The IMMED task waits on the local DSQ behind prev.
Schedule the reenqueue when dispatch keeps prev and an IMMED task is queued,
as even a lone IMMED task can't run right away behind prev.
v2: The description and dispatch_one()'s comment are reworded.
Fixes: 98d709cba319 ("sched_ext: Implement SCX_ENQ_IMMED")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
A task already queued on a bypass DSQ stalls when the host loses
SCX_CAP_BASE on that cpu, as the cpu no longer consumes the DSQ. Sending a
bypassed task to rescue at enqueue time covers only a task enqueued after
the loss, and the balancer no longer donates to a capless cpu.
Have the bypass load balancer drain such a DSQ: a cpu where the host lacks
BASE is a donor with a target of zero, and the donees take its tasks under
their ordinary fill.
A drained task no donee can take is reenqueued at its home: to rescue when
the host lacks BASE there and rescue is enabled, and to the home's bypass
DSQ otherwise.
A balancing donor leaves a task that can't move to its donee where it is.
Reenqueueing it at home would put a task homed on the donor cpu at the tail
of its own DSQ, and a blocked task that proxy execution migrated there,
which every remote cpu refuses, would go to the tail on every pass.
Without rescue, a task homed on the drained cpu goes back onto the drained
DSQ, once per pass, and waits for the host to regain BASE, the bypass to
end, or the stall watchdog to eject the host. Its reenqueues are not counted
against the reenqueue limit, as its scheduler is bypassing.
A BASE loss kicks the balancer so that the drain doesn't wait for the next
interval. With a zero interval the kick is the only pass, and a task that
lands on the drained DSQ after it waits the same way.
v2: The description drops an in-flight affinity change from the causes of a
refused balancer move.
v3:
- The description says that a stuck task's reenqueues are not counted
against the reenqueue limit (sashiko).
- The description names the stall watchdog as the way out for a task stuck
without rescue, and the tasks a zero interval leaves on a drained DSQ.
- bypass_lb_reenq_home()'s kerneldoc and the description give the placement
with rescue disabled and for a task that must run at home, and the other
two comments on that placement point to the kerneldoc.
v4:
- Skip a task the donee would refuse on a balancing donor instead of
reenqueueing it at home (Andrea).
- bypass_lb_reenq_home()'s kerneldoc is corrected on a migration-disabled
task homed on a donee and on the local DSQ admission.
- The description is condensed.
Fixes: 147d1885f390 ("sched_ext: Add the SCX_CAP_ENQ_IMMED cap")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
The following patch uses the bypass load balancer to drain a cpu's bypass
DSQ when the host loses SCX_CAP_BASE there. The host no longer consumes that
DSQ, so the drain has to move every task on it. The balancer skips the ones
homed on another cpu.
An earlier pass moved such a task in from its home cpu's DSQ, and the donor
cpu has not consumed it yet. A task's DSQ membership may only change under
the task's rq lock, which the balancer does not hold.
Take such a task off the donor DSQ with unlink_dsq_and_switch_rq_lock(), the
handshake a remote consumer uses. It drops the DSQ lock and switches to the
task's rq lock. A dequeue that gets in between takes the task, and the
balancer moves on. The insert then runs under the task's home rq lock, as it
already does for a donor-homed task.
A task whose donee is its own cpu can always run there.
task_can_run_on_remote_rq() warns when asked about a task's own cpu, so the
balancer skips the test for such a task.
The walk holds the donor rq lock, which keeps the donor cpu's dispatch and
its enqueuers away from the donor DSQ. A move of a task homed elsewhere
switches to the task's rq lock, and both can reach the DSQ meanwhile. The
parked cursor keeps the walk's place through a consume, and the sequence
bound keeps a task enqueued meanwhile out of the pass.
v2: The description says "the following patch", not "patches", uses the
balancer to drain a DSQ.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
Prepare the bypass load balancer for moving tasks homed on other cpus. Such
a move takes the task's own rq lock through the remote consumer's handshake,
which drops the donor DSQ lock on the way, so its insert runs with the donor
DSQ unlocked and under the task's rq lock. The balancer's own move holds the
donor DSQ lock throughout, with the donee DSQ lock nested inside.
Switch the donor-homed move to the same scheme so that one body can serve
both: factor it out of bypass_lb_cpu() into bypass_lb_move(), which takes
the task off the donor DSQ and inserts with the DSQ unlocked. A task that
can't run on the donee goes back to the head of the donor DSQ instead of
being skipped in place.
The walk parks its cursor behind the task before each move and resumes from
it afterwards. It is bounded to the tasks queued when it began, by their DSQ
sequence numbers, so a task it put back is not visited again in the same
pass.
Before, the walk held both the donor rq lock and the donor DSQ lock across
every move. The donor DSQ can be very long and contended, a walk competing
with the cpu's dispatch and the enqueues for the DSQ lock would crawl, and
the walk is what relieves that contention by moving tasks off the DSQ. The
walk now cycles the DSQ lock around each insert but still holds the rq lock
throughout, and that alone keeps dispatch and enqueues away. The DSQ lock is
uncontended when retaken, so cycling it is cheap.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
host lacks SCX_CAP_BASE
The bypass load balancer spreads a host's bypassed tasks over the cpus of a
node by moving them between its per-cpu bypass DSQs. It counts every cpu of
the node as capacity and takes any cpu below the target as a donee. A cpu's
dispatch only descends into schedulers holding SCX_CAP_BASE on it, so a
bypass DSQ on a cpu where the host lacks BASE is never consumed. A task
moved there sits until the host regains the cap or the bypass window ends.
Count only cpus where the host holds BASE as capacity and pick donees from
them. The target rises to spread the node's tasks over the cpus that can run
them, and a capless cpu receives nothing. A capless cpu with tasks is still
an ordinary donor, offloaded down to the donor target when its queue grows
past the threshold; draining it is left to the following patches.
Fixes: 147d1885f390 ("sched_ext: Add the SCX_CAP_ENQ_IMMED cap")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
A bypassed task stalls on a cpu where its host lacks SCX_CAP_BASE. The host,
the nearest ancestor that is not bypassing, runs a bypassing sub-scheduler's
tasks through its per-cpu bypass DSQs. A cpu's dispatch only descends into
schedulers holding BASE on it, so the host's bypass DSQ on such a cpu is
never consumed until the host regains the cap.
Resolve a bypassed task against its host at enqueue time. If the host lacks
BASE on the cpu, send the task to rescue on that cpu, which runs it at
bounded bandwidth without the host's dispatch having to reach it.
The enqueue does not look above the host for an ancestor that holds BASE.
That ancestor would run the task on cpus it never granted the host.
With rescue disabled the task goes to the host's bypass DSQ as before and
stalls there until the host regains BASE or the bypass ends, or until the
stall watchdog ejects the host, which it blames for a task stuck on its DSQ.
A task that must run on the cpu, migration-disabled or with a migration
pending, is admitted to the local DSQ instead, as any insert of such a task
is.
Bypass also used to end a rescue, sending a preempted rescuee to the bypass
DSQ with every other preempted task. Now that rescue is how a bypassed task
runs while its host lacks BASE, keep a preempted rescuee on the local DSQ
under bypass too. dispatch_one() also keeps such a rescuee running, as an
earlier patch's bypass expiry leaves a kernel slice alone. The grant stays
bounded by served time.
A task scx_fail_parent() punted to its bypassing parent at NONE can be a
rescuee too. switched_from_scx() ends its slice as it does for an enabled
task, so that a NONE rescuee leaving sched_ext while running doesn't leave
rq->scx.rescue.curr pointing at it.
While at it, the rescue test in __scx_resolve_local_dsq() goes through the
new scx_rescue_enabled() helper.
v2:
- The description notes that dispatch_one() keeps a bypassed rescuee too,
through the new patch before this one (sashiko).
- scx_rescue_keep()'s kerneldoc drops dispatch_one() declining a rescuee
with slice left, which the new patch removes.
v3:
- With rescue disabled, a task that must run on the cpu is admitted to the
local DSQ instead of waiting on the host's bypass DSQ.
- The description names the stall watchdog as the way out for a task stuck
without rescue.
v4:
- switched_from_scx() ends the slice of a NONE task, which can now be a
rescuee (sashiko).
- scx_punt_task()'s kerneldoc no longer says that switched_from_scx() skips
a NONE task.
- scx_bypass()'s kerneldoc and the ops.sub_ecaps_updated() doc say where a
bypassed task can be besides the bypass DSQs.
Fixes: 147d1885f390 ("sched_ext: Add the SCX_CAP_ENQ_IMMED cap")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
The next patch keeps a bypassed rescuee on the local DSQ and needs
dispatch_one() to keep it running too. dispatch_one()'s keep test refuses
any prev whose scheduler is bypassing, so another local task could run ahead
of the rescuee, even an escalated SCX_TASK_PROTECTED one. The test exists
because a cpu-form scheduler can write p->scx.slice directly from BPF, so
the slice can't be trusted under bypass.
The refusal doesn't hold anyway. Since 025b1bd41965 ("sched_ext: Implement
hierarchical bypass mode"), the host's ops.dispatch() runs while prev's
scheduler bypasses, and the post-dispatch test in scx_dispatch_sched() keeps
a queued prev with slice left without checking bypass. A task dispatched in
between waits behind prev until the next bypass tick zeroes the slice, at
most a tick.
Expire the slice before the keep test instead when prev's scheduler is
bypassing, as the bypass tick would. Neither test keeps prev on a zeroed
slice, so prev rotates. A kernel slice survives the expiry and keeps prev.
Only a sub-scheduler's task can hold one, and sub-schedulers are cid-form,
which rejects direct slice writes, so it can be trusted.
Drop the stale pointer to scx_disable_workfn(), update scx_bypass()'s
kerneldoc, and add a TODO noting that the bypass slice handling can be
cleaned up once the cpu form is deprecated and the kfuncs refuse slice
raises from a bypassing scheduler.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
local DSQ
put_prev_task_scx() enqueues a still-runnable task when the pick has settled
on idle. The insert can land the task on the cpu's own local DSQ, and then
the cpu must not go idle over it. 7a9df7667963 ("sched_ext: Fix stall when a
task enqueued with SCX_ENQ_LAST lands back on its CPU's local DSQ")
reschedules the idle task in that case, but only after the SCX_ENQ_LAST
insert.
The plain insert, taken when the task can't stay on the cpu, can land on the
local DSQ too: rescue can admit a task there when its scheduler lacks
SCX_CAP_BASE on the cpu. Without the resched the cpu idles with the task
queued until an unrelated wakeup or the rescue timer's escalation.
Reschedule after either insert. In the following patches, the plain insert
becomes the common path for bypassed tasks under rescue.
v2: The description's forward reference to the plain insert becoming the
common path no longer names the following patch.
Fixes: 5fd501744b10 ("sched_ext: Add bandwidth-limited rescue execution for stranded tasks")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
A rescuee can keep running after its grant is served, on a cpu its scheduler
has lost. The write rules deny a slice extension only where the scheduler
lacks SCX_CAP_BASE on the cpu, and a scheduler holding BASE but not ENQ can
have a task under rescue there:
1. the scheduler extends the rescuee's slice
2. BASE is revoked. The cap-revoke eviction skips the rescuee.
3. the grant is served and scx_rescue_charge() ends the rescue
4. the task runs out the extension on the cpu its scheduler lost, outside
the rescue's bandwidth
A scheduler shortens or zeroes a rescuee's slice to preempt it, and
scx_rescue_keep() resumes the grant afterwards. An extension serves no such
purpose.
Refuse extending a kernel slice in both scheduler write paths, the direct
write in scx_bpf_task_set_slice() and the out-of-band application, counted
in SCX_EV_SLICE_DENIED like an extension without BASE. Shortening stays
allowed.
scx_rescue_keep()'s restore of the remainder raises the slice from zero.
Like the kernel's other slice writes, it goes through scx_set_task_slice()
and not the scheduler write paths.
v2:
- scx_rescue_admit()'s kerneldoc no longer mentions a served-time snapshot.
- SCX_EV_SLICE_DENIED's comment lists every refusal it counts.
- scx_bpf_task_set_slice()'s kerneldoc says that shortening is refused on a
protected slice.
Fixes: 5fd501744b10 ("sched_ext: Add bandwidth-limited rescue execution for stranded tasks")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
In the following patches, bypassed tasks run under rescue when their host
lacks SCX_CAP_BASE on the cpu. Under bypass the kernel zeroes the running
task's slice at every tick, as it does not trust the scheduler's slices.
A rescue grant has to survive that, or it is served a tick at a time with
the rescuee sent to the tail of the local DSQ each time. The generic slice
handling can't tell a kernel grant from a scheduler's slice.
Mark the grant as a kernel slice with SCX_TASK_KERNEL_SLICE, set at rescue
admission and cleared when the slice ends. The generic slice handling leaves
it alone. The default refill does nothing for it, and the three zeroings go
through the new expire_task_slice(), which skips it:
- the bypass tick's zeroing of the running task
- the eviction of a running task that can't stay on the cpu when bypass
lifts
- the cap-revoke eviction of a running task
Schedulers keep their control. Preemption and scx_bpf_task_set_slice() still
apply, and the rescue stays bounded by served time.
The sites that recognized the rescuee by identity, the reenqueue scan and
put_prev_task_scx()'s reinsert, test the flag instead.
scx_task_has_kernel_slice() hides the test behind the scx_has_subs() static
key, as only a sub-scheduler's task can carry the flag. The flag's entries
join the enum tables in tools/sched_ext.
No kernel slice reaches a zeroing until bypassed tasks run under rescue. The
cap-revoke eviction skipped the rescuee already, and bypass entry ends an
open rescue, so the bypass tick and the eviction at bypass lift see none.
The refill is the one behavior change. The pick refills a zero slice only
for a task that can stay on the cpu, so it concerns a rescuee whose
scheduler holds BASE there. A scheduler may zero such a queued rescuee's
slice, and the next pick used to refill it to the default.
Now the rescuee runs on the zero slice until the next tick. There
put_prev_task_scx() restores the remainder when another task is runnable.
When nothing else is runnable, dispatch_one() keeps the rescuee as the last
task. The pick then ends its zero slice, which ends the rescue, and refills
the default. A scheduler with SCX_OPS_ENQ_LAST skips the keep-last test.
scx_rescue_keep() restores the remainder there instead and the rescue goes
on.
v2:
- scx_rescue_admit()'s kerneldoc no longer mentions a served-time snapshot.
- The description covers a scheduler with SCX_OPS_ENQ_LAST, whose rescuee
dispatch_one() keeps on the remainder.
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
A scheduler preempts a running rescuee like any other task by zeroing its
slice. The rescue grant survives the preemption. put_prev_task_scx()
restores the unserved remainder through scx_rescue_keep() and queues the
rescuee at the tail of the local DSQ, behind whatever the scheduler wanted
to run first.
That breaks down when the scheduler zeroes the slice but dispatches nothing.
dispatch_one() keeps the previous task running only with slice left or, when
nothing else is runnable, with its scheduler holding SCX_CAP_BASE on the cpu
and not taking the last task through SCX_OPS_ENQ_LAST. A rescuee fails both
once its slice is zeroed if its scheduler lacks BASE there or takes the last
task:
1. ops.tick() or scx_bpf_task_set_slice() zeroes the rescuee's slice
2. the pick finds nothing else to run and settles on idle
3. put_prev_task_scx() restores the remainder and queues the rescuee on the
local DSQ
4. nothing reschedules the cpu, as the pick has settled and __schedule()
clears need_resched right after it
The cpu idles with the rescuee queued until an unrelated wakeup or the
rescue timer's escalation.
Restore the remainder from dispatch_one() too. When nothing else is
runnable, its last keep now keeps the rescuee running on the remainder, so
the pick never settles on idle with a rescuee runnable. put_prev_task_scx()
keeps handling a displaced rescuee, where another task won the pick.
A rescuee that passes the existing keep-last test is kept there instead and
continues as an ordinary task, its rescue ended with the zeroed slice. That
is fine as its scheduler holds SCX_CAP_BASE on the cpu.
The new keep also covers bypassed tasks run under rescue in the following
patches.
v2: The dispatch_one() comment and description now say that a rescuee whose
scheduler holds SCX_CAP_BASE on the cpu passes the keep-last test and
continues as an ordinary task (sashiko).
Fixes: 5fd501744b10 ("sched_ext: Add bandwidth-limited rescue execution for stranded tasks")
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
|
put_prev_task_scx() restores a preempted rescuee's unserved slice through
scx_rescue_keep() before it calls ops.stopping(). A rescuee whose slice the
scheduler zeroes there is left to the reinsert instead:
1. ops.stopping() zeroes the rescuee's slice
2. the reinsert condition special-cases the rescuee and keeps it on the
local DSQ with a zero slice
3. the pick's default refill supplies a slice, or the task runs on a zero
slice until the tick when its scheduler lacks BASE
Move the keep after the stopping call so that scx_rescue_keep() restores the
remainder whether a preemption or ops.stopping() zeroed the slice, and drop
the special case from the reinsert condition as a rescuee reaching it always
has a slice. A rescuee whose slice ops.stopping() zeroed now goes to the
tail of the local DSQ with its remainder restored, behind what the scheduler
wanted to run first, instead of staying at the head on a zero slice.
The slice end in the same block moves with it, which changes nothing for a
protected task. Its slice is the unserved remainder, and scx_rescue_charge()
ends the protection in the same update_curr_scx() call that zeroes the
slice, so the block never sees a protected task in either position.
v2: The description no longer says that a protected task's protection
outlasts ops.stopping().
Signed-off-by: Tejun Heo <tj@kernel.org>
Reviewed-by: Andrea Righi <arighi@nvidia.com>
|
