| Age | Commit message (Collapse) | Author |
|---|
|
Extract .BTF.inline via the following logic:
- Traverse all types reachable from LOC_PARAM, LOC_PROTO, LOCSEC
entries and mark them as having LOC color.
- Traverse all types reachable from the remaining types
and mark them as having MAIN or SHARED.
- Create a new base BTF object and copy all MAIN/SHARED types there.
- Create a new .BTF.inline object with base set to the new base BTF
object, and copy all LOC types there.
This way .BTF.inline can refer to types and strings in the new base,
and strings used only by .BTF.inline remain in it.
Additionally, all BTF_KIND_FUNC types that map to ELF symbols with the
same name are forced to be in the base BTF. Same for types referenced
from BTF id sets.
Regular types referenced only from .BTF.inline remain in the base BTF,
and sizes are 5 and 9.5Mb for vmlinux and vmlinux.inline respectively.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261009112324.1442319-2-alan.maguire@oracle.com
|
|
As the riscv64 JIT now supports instruction array (gotox jump tables),
enable the corresponding bpf selftest for riscv64. The guard is 64-bit
only because the RV32 JIT does not implement gotox.
Signed-off-by: Chen Pei <cp0613@linux.alibaba.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260928071845.2432-5-cp0613@linux.alibaba.com
|
|
The riscv64 JIT now supports gotox, so enable the gotox liveness test in
compute_live_registers.c on riscv64 as well.
The guard is 64-bit only because the RV32 JIT does not implement gotox.
Signed-off-by: Chen Pei <cp0613@linux.alibaba.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260928071845.2432-4-cp0613@linux.alibaba.com
|
|
The riscv64 JIT now supports the gotox instruction and jump tables, so
run the tests in verifier_gotox.c on riscv64 too, mirroring what was done
for arm64 and powerpc.
The guard is 64-bit only because the RV32 JIT does not implement gotox.
Signed-off-by: Chen Pei <cp0613@linux.alibaba.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Reviewed-by: Björn Töpel <bjorn@kernel.org>
Acked-by: Björn Töpel <bjorn@kernel.org>
Link: https://patch.msgid.link/20260928071845.2432-3-cp0613@linux.alibaba.com
|
|
Both probe_user and stacktrace_build_id fail on riscv64 due to riscv64's
trap-based kprobe handling.
For probe_user, kprobes on riscv64 are handled via traps where the entry
path invokes irqentry_nmi_enter(). Because bpf_probe_write_user()
rejects execution in NMI context, the test fails. Upstream discussions
regarding NMI semantics on riscv are ongoing [0] and will be tracked.
For stacktrace_build_id, under the kprobe trap context with interrupts
disabled, bpf_get_stackid() obtains a valid build_id, but the queued
irq_work is deferred. As a result, subsequent bpf_get_stack() fails to
acquire mmap_read_trylock() and falls back to BPF_STACK_BUILD_ID_IP,
causing an inconsistency between the two results.
Since stacktrace_build_id is already included in the generic DENYLIST,
only add probe_user to DENYLIST.riscv64.
Signed-off-by: Pu Lehui <pulehui@huawei.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/linux-riscv/20230702025708.784106-1-guoren@kernel.org [0]
Link: https://patch.msgid.link/20260930083522.1914145-5-pulehui@huaweicloud.com
|
|
When running vmtest.sh on riscv64, the kernel panic at boot due to an
illegal instruction. This is because '-cpu rv64' only enables a baseline
set of riscv extensions, whereas the kernel or toolchain requires newer
extensions. Switch to '-cpu max' so qemu enables all supported
extensions, and bump the required qemu version accordingly.
Additionally, qemu's pmu emulation for riscv64 currently has issues,
causing perf_event_open-related tests to fail. Temporarily disable the
sscofpmf extension to bypass these failures for now, and will try to
investigate and address it later.
While at it, update the libbpf/ci branch in INDEX_URL from 'master' to
'main'.
Signed-off-by: Pu Lehui <pulehui@huawei.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260930083522.1914145-4-pulehui@huaweicloud.com
|
|
The setget_sockopt selftest fails on kernels with HZ=250 (such as the
default on riscv64), specifically in the TCP_BPF_DELACK_MAX and
TCP_BPF_RTO_MIN sub-tests.
The test sets a socket option value in microseconds via bpf_setsockopt()
and immediately reads it back via bpf_getsockopt() to verify that it
matches. However, the kernel converts the value to jiffies internally
for storage and converts it back to microseconds upon retrieval. When
HZ=250 (4000 us per jiffy), 30000 us cannot be divided evenly:
usecs_to_jiffies(30000) -> 8 jiffies
jiffies_to_usecs(8) -> 32000 us (!= 30000 us)
This mismatch triggers an assertion failure.
Change the test value from 30000 to 40000. 40000 us divides evenly
across common kernel timer frequencies (HZ=100, HZ=250, and HZ=1000),
avoiding quantization errors and making the test robust across different
architectures.
Signed-off-by: Pu Lehui <pulehui@huawei.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260930083522.1914145-3-pulehui@huaweicloud.com
|
|
Enable callx bpf selftests for RV64 now that the RV64 JIT supports
indirect calls to subprogs.
Signed-off-by: Pu Lehui <pulehui@huawei.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260930064928.1909085-3-pulehui@huaweicloud.com
|
|
* kvm-arm64/rd-assignment-fixes:
: \
: Fixes for failed GICv3 RD assignment to vcpu rollback.
:
: Patches by Karl Mehltretter.
: /
KVM: arm64: selftests: Test VGICv3 redistributor region retry
KVM: arm64: selftests: Pass guest code to vm_gic_create_with_vcpus()
KVM: arm64: vgic-v3: Roll back assignments from the new region
KVM: arm64: vgic-v3: Undo assignment on iodev registration failure
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
* kvm-arm64/immutable-idregs-7.4:
: \
: Make ID registers immutable as soon as a vcpu has had a chance of
: running, preventing userspace from altering them from under the guest's
: feet. Patches courtesy of Mark Brown, selftest improvements by Fuad Tabba.
: /
KVM: arm64: selftests: Check ID regs are immutable after a failed run
KVM: arm64: Block ID register changes after we rely on the values
KVM: arm64: Finalize guest-wide sysregs prior to per-vCPU sysregs
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
* kvm-arm64/misc-7.4:
: \
: Misc KVM/arm64 fixes for 7.4:
:
: - Correctly report SMCCC_VERSION and SMCCC_ARCH_FEATURES as implemented
:
: - Don't register a page table dumper for pKVM VMs, as S2 is not
: accessible when in protected mode
:
: - Drop PAGE_SIZE alignment requirement for hyp trace event sections
:
: - Tighten PMUv3 being disabled for the guest even if vcpus were created
: with one.
:
: - Fix timer emulation when the HW doesn't support offsetting of the
: counter.
: /
KVM: arm64: selftests: Test a timer set past the counter's wrap
KVM: arm64: nv: Read a guest hypervisor's CNTV_CVAL_EL0 from memory on x1e
KVM: arm64: timers: Compute an offset-applied CVAL from the current count
KVM: arm64: Check ID_AA64DFR0_EL1.PMUVer in pmu_visibility()
KVM: arm64: Handle ID_AA64DFR0_EL1.PMUVer == NI in __kvm_pmu_event_mask()
KVM: arm64: Remove PAGE_SIZE alignment for hyp event ELF sections
KVM: arm64: Disable stage-2 ptdump of pKVM
KVM: arm64: selftests: Check the mandatory SMCCC_ARCH_FEATURES queries
KVM: arm64: Report SMCCC_VERSION and SMCCC_ARCH_FEATURES as implemented
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
* kvm-arm64/pre-faulting:
: \
: Enable stage-2 pre-faulting in the canonical IPA space, implementing the
: KVM_PRE_FAULT_MEMORY API. Patches courtesy of Lorenzo Stoakes, and based
: on an initial work by Jack Thomson.
: /
KVM: selftests: Add nested pre-fault test for arm64
KVM: selftests: Add option for different backing in pre-fault tests
KVM: selftests: Enable pre_fault_memory_test for arm64
Documentation: KVM: document arm64 KVM_PRE_FAULT_MEMORY
KVM: arm64: Implement KVM_PRE_FAULT_MEMORY
KVM: arm64: Pass walk flags to kvm_pgtable_get_leaf()
KVM: arm64: Propagate EHWPOISON in kvm_s2_fault_pin_pfn()
KVM: arm64: Size the stage-2 memcache from the fault MMU
KVM: arm64: Propagate and use kvm_s2_fault_result on S2 fault
KVM: arm64: Propagate and use mmu in s2fd when handling guest aborts
KVM: arm64: Propagate and use esr in s2fd when handling guest aborts
KVM: arm64: Use ESR helpers in guest abort handling
arm64: Add ESR fault helpers
KVM: Allow architectures to disallow pre-fault
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
* kvm-arm64/s2-rmap: (31 commits)
: \
: Add a per-shadow stage-2 reverse map tracking infrastructure,
: considerably reducing the overhead of MMU notifiers when running L2
: guests. Also fixes issues with the shadow S2 MMU debugfs infrastrtcture.
:
: Patches courtesy of Wei-Lin Chang.
:
: This branch drags the kvmarm-fixes-7.3-1 tag, as this series depends
: on some of the fixes.
: /
KVM: arm64: ptdump: Fix shadow ptdump sleep-in-atomic-context problem
KVM: arm64: ptdump: Check the page tables aren't freed when accessing
KVM: arm64: Refactor kvm_unmap_gfn_range() with common variables
KVM: arm64: nv: Drop kvm_s2_mmu pointer from kvm_guest_s2_mapping
KVM: arm64: nv: Avoid full shadow stage-2 unmap
KVM: arm64: nv: Track guest stage-2 mapping removal
KVM: arm64: nv: Track guest stage-2 mapping creation
KVM: arm64: nv: Introduce guest stage-2 tracking structures
KVM: arm64: Use a variable for the canonical IPA in kvm_s2_fault_map()
KVM: arm64: Fix AArch32 DBGBXVR<n> handling
KVM: arm64: Fix FGT mapping for HFGITR_EL2.nGCSEPP
KVM: selftests: fix steal_time for arm64 with host page size > 4K
KVM: arm64: Don't WARN on an unknown VM ioctl in protected mode
KVM: arm64: nv: Delay freeing of shadow S2 structures until VM destruction
KVM: arm64: nv: Fix life cycle of the nested_mmus array
KVM: arm64: Check every private mapping is hyp-owned at pKVM init
KVM: arm64: Move the private VA allocation cursor to __io_map_next
KVM: arm64: Match hyp text by physical address in fix_host_ownership()
KVM: arm64: Transfer the hyp stack pages out of the host stage-2
KVM: arm64: selftests: Test empty SMCCC filter range at base 0
...
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
* kvm-arm64/gicv5-7.4: (52 commits)
: \
: GICv5 IRS support, courtesy of Sascha Bischoff. From the cover letter:
:
: "This series builds on the initial vGICv5 support and adds support
: for the GICv5 IRS, as described by the GICv5 (EAC0) specification.
: With this, a GICv5 guest is no longer restricted to PPIs, and can make
: use of SPIs and LPIs as well."
: /
KVM: arm64: vgic-v5: Correctly handle host ISTE __le32 conversion
KVM: arm64: vgic-v5: Tidy-up programming of vpe descriptor address
KVM: arm64: vgic-v5: Correctly reset h_lpi_ist to NULL
KVM: arm64: vgic-v5: Drop __iomem attribute from {vmd,vpet}_base
KVM: selftests: Add VGICv5 sparse vCPU IDs test
KVM: selftests: Add VGICv5 IST save/restore coverage
KVM: selftests: Add VGICv5 LPI delivery tests
KVM: selftests: Add VGICv5 SPI injection tests
KVM: selftests: Add VGICv5 CPU sysreg attribute tests
KVM: selftests: Add VGICv5 USERSPACE_PPIS tests
KVM: selftests: Add VGICv5 IST attribute tests
KVM: selftests: Add VGICv5 IRS_REGS attribute tests
KVM: selftests: Add VGICv5 NR_IRQS attribute tests
KVM: selftests: Add VGICv5 IRS address attribute tests
Documentation: KVM: Add the VGICv5 IRS save/restore sequences
Documentation: KVM: Add docs for KVM_DEV_ARM_VGIC_GRP_IST
Documentation: KVM: Add KVM_DEV_ARM_VGIC_GRP_IRS_REGS to VGICv5 docs
Documentation: KVM: Document KVM_DEV_ARM_VGIC_GRP_CPU_SYSREGS for VGICv5
KVM: arm64: gic-v5: Implement save/restore mechanisms for ISTs
KVM: arm64: gic-v5: Add VGICv5 IST save/restore UAPI
...
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
* kvm-arm64/selftests-7.4:
: \
: Various fixes for selftests:
:
: - Render sea_to_user less braindead, though the utility of the test
: itself is still debatable
:
: - Correctly skip testing FEAT_EASE when the feature isn't available
:
: - Check for consumption of a pended SError
: /
KVM: arm64: selftests: Check the pending SError state in external_aborts
KVM: arm64: selftests: Skip sea_to_user when EINJ places no poison
KVM: arm64: selftests: Skip sea_to_user without 1GB hugepages
KVM: arm64: selftests: Fix EINJ handling in sea_to_user
KVM: arm64: selftests: Fix VM leak in early return
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
To temporarily resolve the conflict between
eb13a1ff271b ("random: vDSO: avoid call to memset() when zeroing reserved parameter")
and
f85147b291e3 ("kbuild: move the toolchain checks into scripts/Kconfig.toolchain")
for -next. Linus will handle this during the merge window, as he prefers
to handle these conflicts himself.
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
|
|
https://gitlab.freedesktop.org/drm/kernel into v7.4/vfio/next
pci/vgaarb rework for drm/vfio
This reworks the vgaarb API to be a bit more sane.
Signed-off-by: Alex Williamson <alex@shazbot.org>
|
|
The test is roughly divided into two stages, and the sequence
is as follows:
I) Setup
1. Attach two BPF programs to a cgroup
2. Establish a TCP connection (@client <-> @child) within the cgroup
3. Enable BPF_TCP_OPS_FLAG_RCVQ on @child via setsockopt()
II) RPC frame exchange in various patterns
4. Send a partial RPC descriptor from @client to @child
5. Verify that epoll does NOT wake up @child
6. Send the remaining data of the RPC frame
7. Verify that epoll finally wakes up @child
During setup, two BPF programs are attached to simulate
a real-world scenario; one is bpf_tcp_ops and the other is
CGROUP_SOCKOPT.
While the bpf_tcp_ops prog handles the dynamic adjustment of
sk->sk_rcvlowat, the CGROUP_SOCKOPT prog is used to enable
the TCP AutoLOWAT feature via userspace setsockopt() using
pseudo options:
#define SOL_BPF 0xdeadbeef
#define BPF_TCP_AUTOLOWAT 0x8badf00d
setsockopt(fd, SOL_BPF, BPF_TCP_AUTOLOWAT, &(int){1}, sizeof(int));
This reflects a common production use case where an application
decides to start parsing RPC frames only at a certain point in
the stream (e.g., after HTTP Upgrade), rather than immediately
after TCP 3WHS (->passive_established(), etc).
When BPF_TCP_AUTOLOWAT is enabled, the BPF prog sets
BPF_TCP_OPS_FLAG_RCVQ and initializes sk_local_storage
for two sequence numbers to manage its state.
Then, for the RPC frame exchange, this test uses a simple format
defined as follows:
0 8 16 24 32
+--------+--------+-------+--------+ `.
| header size | |
+--------+--------+-------+--------+ > RPC descriptor (8 bytes)
| payload size | |
+--------+--------+-------+--------+ .'
~ header ~
+--------+--------+-------+--------+
~ payload ~
+--------+--------+-------+--------+
Every time a new skb is enqueued to sk->sk_receive_queue, the
bpf_tcp_ops prog parses it and updates these sequence numbers:
rpc_desc_seq : the SEQ # of the start of the RPC descriptor
rpc_end_seq : the SEQ # of the end of the RPC frame
=> rpc_desc_seq + 8 + header size + payload size
Assume we receive two RPC descriptors in the following pattern:
1. When we receive skb-1, only part of the RPC descriptor is parsed.
rpc_desc_seq is set to the first byte while rpc_end_seq is
unknown. Thus, sk->sk_rcvlowat is set to the size of the RPC
descriptor (8 bytes).
<- skb-1 -> <---- skb-2 ----> <------ skb-3 ----->
+-----------+.................+....................+......
| RPC desc 1 | header + payload | RPC desc 2 | ...
+-----------+.................+....................+......
^ ^-.
`- rpc_desc_seq `- sk->sk_rcvlowat
2. Next, we receive skb-2, which completes the first RPC descriptor.
Now rpc_end_seq is known, so sk->sk_rcvlowat is advanced to it.
<- skb-1 -> <---- skb-2 ----> <------ skb-3 ----->
+-----------+-----------------+....................+......
| RPC desc 1 | header + payload | RPC desc 2 | ...
+-----------+-----------------+....................+......
^ ^
'- rpc_desc_seq '- rpc_end_seq
& sk->sk_rcvlowat
3. Once we receive skb-3, which contains the next full RPC descriptor,
rpc_desc_seq is advanced and rpc_end_seq is updated according
to the size of RPC frame 2.
Note that sk->sk_rcvlowat is NOT updated to the new rpc_end_seq
yet. This ensures that the application is woken up to read the
already complete RPC frame 1.
<- skb-1 -> <---- skb-2 ----> <------ skb-3 ----->
+-----------+-----------------+--------------------+......
| RPC desc 1 | header + payload | RPC desc 2 | ... |
+-----------+-----------------+--------------------+......
^ ^
rpc_desc_seq -----------' rpc_end_seq ----...-'
& sk->sk_rcvlowat
This sequence corresponds to the 4th test case in rpc_test_cases[],
and we can see helpful output if we "#define DEBUG":
# cat /sys/kernel/tracing/trace_pipe | \
awk '{ if ($0 ~ /AF_/) sub(/^.*AF_/, "AF_"); print $0 }' & \
BGPID=$!; ./test_progs -t tcp_autolowat; kill -9 -$BGPID
...
AF_INET6 rpc_test_cases[3]: Start parsing skb: seq: 0, end_seq: 1, len: 1, rpc_desc_seq: 0, rpc_end_seq: 0, rpc_buff_len: 0
AF_INET6 rpc_test_cases[3]: Copied 1 bytes: rpc_desc_buff_len: 1
AF_INET6 rpc_test_cases[3]: Setting rcvlowat: tp->copied_seq: 0, rpc_desc_seq: 0, rpc_end_seq: 0, rpc_desc_buff_len: 1
AF_INET6 rpc_test_cases[3]: Set rcvlowat: expected: 8, actual: 8
AF_INET6 rpc_test_cases[3]: Start parsing skb: seq: 1, end_seq: 8, len: 7, rpc_desc_seq: 0, rpc_end_seq: 0, rpc_buff_len: 1
AF_INET6 rpc_test_cases[3]: Copied full descriptor: rpc_desc_seq: 0, rpc_end_seq: 258, header_len: 100, payload_len: 150
AF_INET6 rpc_test_cases[3]: No more descriptor: rpc_end_seq: 258, end_seq: 8
AF_INET6 rpc_test_cases[3]: Setting rcvlowat: tp->copied_seq: 0, rpc_desc_seq: 0, rpc_end_seq: 258, rpc_desc_buff_len: 8
AF_INET6 rpc_test_cases[3]: Set rcvlowat: expected: 258, actual: 258
...
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Reviewed-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://patch.msgid.link/20261008031604.256498-11-kuniyu@google.com
|
|
The next patch guards bpf_tcp_ops callbacks in the fast
path with per-socket flags.
Let's use bpf_tcp_ops_set_flags() to enable flags for
parse_hdr, write_hdr_opt, and hdr_opt_len.
Note that BPF_TCP_OPS_FLAG_PARSE_HDR_OPT_ALL and
BPF_TCP_OPS_FLAG_PARSE_HDR_OPT_UNKNOWN are used on the
server and client, respectively, for better coverage.
Signed-off-by: Kuniyuki Iwashima <kuniyu@google.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Reviewed-by: Amery Hung <ameryhung@gmail.com>
Acked-by: Stanislav Fomichev <sdf@fomichev.me>
Link: https://patch.msgid.link/20261008031604.256498-4-kuniyu@google.com
|
|
Add BPF_F_TEST_SKB_CHECKSUM_COMPLETE tests for tc programs that move the
skb out of CHECKSUM_COMPLETE with bpf_skb_change_tail():
- change_tail_trim trims the last 4 bytes of pkt_v4. They hold the
non-zero tcp.urg_ptr, so the stale skb->csum no longer matches the
packet.
- change_tail_grow_write appends 4 zero bytes, which on their own do
not change the sum, and then writes a byte with BPF_F_RECOMPUTE_CSUM.
That write can no longer update skb->csum.
Both return -EBADMSG without the preceding test_run fix and pass with
it.
Also add store_no_recompute as a negative test. It writes a byte
without BPF_F_RECOMPUTE_CSUM, so the skb stays CHECKSUM_COMPLETE with a
stale skb->csum, and test_run must keep returning -EBADMSG. This checks
that the validation still runs for CHECKSUM_COMPLETE skbs. The passing
case, a CHECKSUM_COMPLETE skb whose checksum is updated correctly, is
already covered by test_skb_pkt_end.
Signed-off-by: Maxim Skokov <skokovmaksimevg@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261009174755.22026-3-skokovmaksimevg@gmail.com
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs
Pull vfs fixes from Christian Brauner:
"This contains fixes for the current development cycle.
All of them came out of a review of the mount code that started with a
bug report. The review modeled the corner cases of mount propagation,
unmounting and mount reference counting and turned up a lot of bugs.
Most of them years old. Most fixes come with a selftest.
- Rework connected mounts.
A mount that is unmounted together with its parent can stay
attached to the parent to keep its mountpoint covered. That happens
when the mountpoint is removed with rmdir(), unlink() or rename(),
when a detached tree is dissolved, and for locked mounts in any
umount that isn't synchronous, including the teardown of their
mount namespace. The parent then owns the child and drops it on its
own final mntput(). So any reference from the child's superblock
back to one of its ancestors becomes a cycle that is never freed.
A loop device backed by an image on a tmpfs and mounted on that
same tmpfs is enough. Remove the directory the tmpfs is mounted on
from the host, let the container's mount namespace exit, and the
loop device, the tmpfs and the filesystem on the loop device are
leaked for good. The same works with autofs, zram, ecryptfs,
binfmt_misc, fuse passthrough, zloop, a mass storage gadget and md,
and the selftests have reproducers for them. This has been possible
since v4.1. It is also why "put_mnt_ns(): leave mounts connected"
was reverted in -rc5. Keeping every mount of a dying mount
namespace connected made these cycles trivial to create.
Every unmounted mount is now detached from its parent. Where the
mountpoint has to stay covered the mount leaves a cover on the
parent instead, allocated together with the mount. A lookup on the
unmounted parent that hits a cover finds an empty immutable
directory or file on the private nullfs instance. Nothing leads
from a cover to another mount, so no unmounted mount owns another
one and no cycle can form.
This is visible to userspace. A formerly connected mount can no
longer be reached through its unmounted parent and ".." inside it
leads nowhere, as for every other lazily unmounted mount.
With that the private nullfs instance becomes reachable from
userspace, so it now refuses mounts on top, is mounted read-only,
and refuses fsnotify marks and file locks. Its inodes are shared by
every holder and a watch or a lock would otherwise reach across
users. may_decode_fh() now also decides its subtree check under a
single mount_lock hold, as a racing umount could otherwise let it
decode into what a locked child covered.
- umount:
* Don't silently unmount busy mounts.
Since v4.13 propagate_umount() takes down propagated copies of
the victim with children as long as each child is an overmount or
another copy of the victim, but propagate_mount_busy() only ever
checked copies without children or with just an overmount. A
container that moved a tree beneath its copy of a host mount lost
that tree from under its open file descriptor to a plain umount()
on the host. propagate_mount_busy() now applies the same rules,
walking each chain of copies once.
* Don't let a migrating task hide its reference from umount().
mnt_get_count() sums the per-cpu counters under mount_lock but
the mntget() and mntput() fast paths don't take it. A task that
takes a reference on a cpu the sum has already passed and drops
it after migrating to one the sum hasn't reached yet hides the
reference it held to begin with, and umount() succeeds with the
file still open. Gets and puts now live in separate per-cpu
counters and all puts are summed before all gets with a full
barrier in between, the way srcu_readers_active_idx_check() does
it. mntget() is unchanged and mntput() gains an smp_wmb().
* Check each submount for references right before unmounting it.
shrink_submounts() and mark_mounts_for_expiry() checked all their
victims up front. Unmounting the first could move a busy
overmount to where the next victim's propagated copy is looked up
and it was then unmounted without a check.
* Never expire a locked mount.
A shrinkable mount moved beneath a locked mount with
MOVE_MOUNT_BENEATH takes over the lock, and umount() of an
unlocked ancestor expired it and revealed what it covered. That
umount() now fails with EBUSY as it does for any other locked
child. A lazy umount still takes the whole tree.
- Overmounts and locked mounts:
* Unhash a dentry before detaching the mounts on it.
unlink(), rmdir() and rename() detach the mounts on the victim
but only d_delete() it once its inode is unlocked, a window that
includes an expedited RCU grace period. In between, a lookup from
a mount namespace in which the dentry is a mountpoint found it
hashed, positive and uncovered. Drop the dentry first, as
d_invalidate() already does.
* Don't reveal overmounted entries in refwalk.
A refwalk that had grabbed the dentry before the unlink never
rechecked it the way rcuwalk does with d_seq and mount_lock.
Without any artificial widening three walkers read the covered
file 27 times in a minute. step_into() now fails an unhashed
dentry marked DCACHE_CANT_MOUNT with -ESTALE and the walk is
retried.
* Keep covered mounts covered in OPEN_TREE_NAMESPACE.
Creating such a mount namespace only takes a user namespace and
the copy followed bind mount rules: no children without
AT_RECURSIVE and no unbindable mounts with it. An unprivileged
user could see what mounts covered in the source, such as the
parts of /proc and /sys that container runtimes mask. If the
caller doesn't own the source mount namespace a non-recursive
copy of a mount with something mounted below the requested
directory is now refused and a recursive copy includes unbindable
mounts, the way unshare() copies.
* Keep the lock on a mount that a propagated copy is moved beneath.
MNT_LOCKED moved to any mount that ended up beneath a locked
mount, propagated copies included. A host mount and umount on a
directory covered by a locked mount in a less privileged mount
namespace left that cover unlocked for the namespace's owner to
remove. Only mounts the caller places beneath take over the lock
now.
* Handle mount locking for automounts correctly. Which copies to
lock was decided by the mount namespace of the task that
triggered the automount. A task in a user namespace that
triggered one on a host mount through a file descriptor got the
host's own automount locked while its own copy stayed unlocked
and could have nosuid, nodev and noexec cleared. Use the owner of
the mount namespace the mount lands in.
- Use-after-free and crashes:
* Refuse an automount below a mount that is in no namespace.
The private clones overlayfs uses for its layers have the
MNT_NS_INTERNAL error pointer as their namespace, which
finish_automount() let through and count_mounts() dereferenced. A
fanotify filesystem mark on an overlayfs lower layer hands out
file descriptors on such a clone. With debugfs as the lower layer
opening "tracing" oopses with namespace_sem held for writing and
every mount operation on the system blocks from then on.
* Reset the old parent's ->overmount in mnt_change_mountpoint().
When propagate_umount() moved an overmount off a mount that a
file descriptor kept alive, MOVE_MOUNT_BENEATH through that
descriptor later followed the stale pointer into the freed
overmount.
* statmount() with STATMOUNT_BY_FD and pivot_root() read the parent
of a mount that may be unmounted and only held by a file
descriptor, while the parent's final mntput() can free it.
statmount() now reads it under mount_lock and pivot_root() first
checks that both mounts are in the caller's mount namespace.
* Queue a mount only once for mount notifications. A mount
reparented by one umount_tree() and taken down by the next under
the same namespace_sem hold, as in shrink_submounts(), was queued
twice. That cut the mounts queued in between out of notify_list
while it still pointed at them, and once they were freed every
later mount operation walked freed memory.
* Don't let a pseudo dentry become the root of a mount. A bind
mount of a bpf token file did that with a DCACHE_NORCU dentry,
which is freed without an RCU grace period while lockless path
walks may still look at it. Refuse to clone such a mount.
* Don't inherit MNT_UMOUNT in clone_mnt(). A bind mount of a lazily
unmounted nsfs or pidfs mount through its file descriptor started
out flagged as unmounted. Among other things __detach_mounts()
then dropped the namespace's reference on it, the mount outlived
its namespace and mount_setattr() through the descriptor read the
freed namespace. A recursive bind mount of such a mount also
copied the unmounted stack still attached to it. That now fails
with EINVAL, copying the mount itself still works.
* Remove the fsnotify marks of a mount namespace in free_mnt_ns()
instead of the RCU callback that frees the namespace, where
taking the group mutexes meant sleeping in softirq context.
- Propagation and copies:
* Keep a copied mount unbindable. Since v6.17 clone_mnt() didn't
copy the unbindable flag, so every mount namespace created with
CLONE_NEWNS had bindable copies of all unbindable mounts. This
had been fixed once before.
* Refuse MOVE_MOUNT_SET_GROUP on an unbindable mount. It made the
mount an unbindable slave, a state nothing else can produce, or
silently dropped the unbindable flag. CRIU applies MS_UNBINDABLE
after restoring sharing and isn't affected.
* Check a recursive bind mount for mount namespace loops.
Recursively bind mounting a tree from another mount namespace
could put a mount of a namespace's file inside that same
namespace, which then pins itself and all its mounts. Repeating
it leaks without limit, the reproducer took Shmem from 380 kB to
65916 kB. The copy is now checked with check_for_nsfs_mounts()
before it is grafted, as move_mount() does.
* Look at the topmost mount for a mount namespace file.
attach_recursive_mnt() never looked at the topmost mount of the
source's chain of overmounts. If that was the chain's only mount
namespace file an existing mount at a propagated destination got
buried below the root of the nsfs file where no path walk reaches
it.
* Don't put a mountpoint on a dentry that's being removed.
attach_recursive_mnt() makes a mountpoint of the source's root
without its inode lock, so a racing rmdir() of that directory
could leave a mount on it that nothing ever detaches.
d_set_mounted() now checks cant_mount() as well.
- nullfs:
* Take no inode lock for readdir of an immutable directory.
The root of every empty mount namespace is the same nullfs
directory and iterate_dir() held its i_rwsem across
->iterate_shared(). A reader whose buffer faults on a FUSE mount
of its own holds it for as long as its server wants, and with an
exclusive locker queued behind it every lookup that misses the
dcache, every create and every mount in that directory waits. One
user of an empty mount namespace stalls all others. Directories
with the new FOP_IMMUTABLE flag skip the lock.
* Refuse to reconfigure internal superblocks through fspick(),
MS_REMOUNT or the read-only remount that a synchronous umount()
of the root does. For nullfs only root in the initial user
namespace could do it, but the superblock is shared by every
mount namespace and the flags showed up in statfs() for all of
them.
* Don't update the access time on nullfs and refuse F_SET_RW_HINT
on an immutable inode.
- unshare: Free an nsproxy that was never installed with
nsproxy_free() when set_cred_ucounts() fails. put_nsproxy() dropped
active references that were never taken, which triggered a warning
and hid the caller's own namespaces from listns().
- Smaller changes: mount_setattr() checks the target before it walks
the tree to allocate peer group ids, unshare() puts the old
fs_struct before the old namespaces, dissolve_on_fput() drops the
file's reference to the tree itself, disconnect_mount() is
simplified and the documentation of the propagated unmount rule is
brought up to date"
* tag 'vfs-7.3-rc7.fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/vfs/vfs: (58 commits)
namespace: simplify disconnect_mount()
selftests/filesystems: test covered mounts
namespace: rework connected mounts
nullfs: add an empty immutable regular file
selftests/filesystems: check that reading the root of an empty mount namespace stalls nobody
selftests/filesystems: add a helper that holds a readdir in a page fault
readdir: take no inode lock on an immutable directory
nullfs: refuse file locks
fsnotify: let a filesystem refuse marks on its objects
namespace: nothing is mounted on or written through knullfs
namespace: keep the private nullfs instance in knullfs
fhandle: decide the subtree check under mount_lock
selftests/filesystems: check that an automount below an overlay layer is refused
selftests/filesystems: check the atime of the empty mount namespace root
selftests/filesystems: check that a lock lands on the right mount and stays
namespace: keep the lock on a mount that a propagated copy is moved beneath
namespace: never expire a locked mount
nullfs: don't update the access time
namespace: handle mount locking for automounts correctly
namespace: refuse an automount below a mount that is in no namespace
...
|
|
- Computed expression shapes: register and spilled-counter recurrences,
substitution at latches, agreeing and conflicting backedge updates,
and ALU, extension, byte-swap, load/store and atomic expressions.
- Iteration bounds and header-visit counts for:
- pre-condition loops using JEQ, JLT, JLE, JGE and JGT;
- post-condition loops using JLT, JLE, JGE, JGT and JNE;
- signed pre- and post-conditions using JSLT, JSLE, JSGT and JSGE;
- increasing and decreasing counters, nonzero initial values,
immediate and invariant-register bounds, and immediate exits;
- additional exits that reduce the minimum iteration count or
tighten the upper bound;
- rounded bounds, counter wraparound, count overflow, extreme
steps and JNE divisibility.
- Widening, backedge clamping and state-pruning convergence,
including multiple induction variables used for map-value accesses
and nonconstant entry values requiring a common alignment.
Check precision propagation to initial values and bounds.
- Nested and sibling loops, shared headers, exits to another loop
header, exits across multiple nesting levels, and side entries
into loop nests.
Check conservative summaries for irreducible children while still
widening supported reducible descendants, value preservation on
inner-loop exits, and detection of nonterminating outer loops.
- Invalidation of stack-slot expressions after indirect stack writes
and global calls. Widening across bpf_probe_read_kernel() calls
in static subprograms.
- Stack spill widths and slot states, including narrow spills, zero,
unallocated and STACK_MISC slots, and counters spilled and filled
using store-release/load-acquire.
- Avoid widening registers used to address stack spills, fills and
dynptr constructor arguments, including dependencies through nested
loops. Conversely, allow widening addresses of byte-sized stores.
- Conditional assignments joining scalar values and stack pointers,
including preservation of a common base and gcd-derived step.
- Pointer widening and comparisons with compatible memory origins,
fallback for incompatible origins and nullable pointers, and
invalidation of stale packet pointer relationships.
- Conservative fallback for unsupported recurrences, unavailable
counter spills and zero computed header counts.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261009-scev-minimal-rebase-v3-51-bcba9bc53ac9@gmail.com
|
|
Coverage:
- straight-line code with an ldimm64 in the middle;
- an asymmetric if-then-else diamond (unequal-depth idoms_intersect());
- a simple loop, a loop with an if-else body, and nested loops;
- a loop header with two back edges (three predecessors);
- an irreducible CFG (fixpoint convergence);
- immediate dominators across subprogram boundaries, including a callee
whose entry instruction is itself a loop header;
- a self-loop;
- an indirect jump (gotox) with a two-entry jump table.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261009-scev-minimal-rebase-v3-50-bcba9bc53ac9@gmail.com
|
|
Test cases covering the following branches in bpf_compute_loops():
- Case B: simple backedge creating a loop (loop_single)
- Case B: two independent loops (loop_two_independent)
- Case B + D: nested loops where the inner header's loop_header points
to the outer header (loop_nested)
- Case C: diamond CFG with no loops (fwd_edges_no_loop)
- Case D: sibling inner loops within one outer loop
(loop_nested_siblings)
- Three levels of loop nesting (loop_three_levels)
- Loop with an if-else body containing forward branches
(loop_with_if_else)
- Case E: An irreducible loop (loop_irreducible)
- A self-loop (loop_self)
- A test with sibling loops nested in outer loops
(loop_nested_siblings_common_ancestors)
- A test case with outer loop's backedge originating
from an inner loop (loop_outer_backedge_from_inner).
Cases B, C, D and E are described in loops.c:compute_loops_in_subprog().
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://patch.msgid.link/20261009-scev-minimal-rebase-v3-49-bcba9bc53ac9@gmail.com
|
|
Verifier tests for accessing arrays at a varying offset through a
PTR_TO_BTF_ID pointer, e.g. a[i].b where 'i' is a register with a
varying value:
- bpf_obj_new() is used to obtain a PTR_TO_BTF_ID to a program-defined
struct
- bpf_get_current_task_btf() is used to obtain a PTR_TO_BTF_ID from
the kernel BTF.
The cases cover:
- in-bounds access to a field of an array of structs;
- out-of-bounds access where the maximal offset runs past the array;
- a scalar (long) array with a matching stride;
- a byte array (step 1);
- a 4-byte read whose maximal offset spills past the array end;
- an unaligned step that is not a whole number of elements;
- both dimensions of a 2D array;
- an array of pointers within a program-allocated object, where reading a
pointer member yields a SCALAR_VALUE but the varying offset is still
validated against the array bounds (in-bounds and out-of-bounds);
- a varying access into an array embedded in a kernel BTF type
(task_struct's comm[]), covering the non-allocated, trusted-pointer path
(in-bounds and out-of-bounds);
- an array nested below a struct member at a non-zero offset, exercising
the offset telescoping done while walking across a struct boundary
(in-bounds and out-of-bounds);
- a trailing flexible array member, whose unbounded tail makes a varying
access have no upper bound to exceed. A program-defined struct cannot be
used here because bpf_obj_new() forbids flexible array members, so a
kernel type (struct vring_used, with its 'ring[]' tail) is used
(in-bounds, and an unaligned step that is not a whole number of
elements).
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261009-scev-minimal-rebase-v3-48-bcba9bc53ac9@gmail.com
|
|
Tests for the base/step reasoning in states.c:range_within():
- step_prune_hit_multiple: prune a step-4 range contained in a cached
step-2 range when the scalar is used to address the stack.
- step_prune_miss_non_multiple: do not let a cached step-2 range hide a
step-3 path reaching division by zero.
- step_prune_hit_const_on_line: prune constant 6 against a cached
base-0, step-3 range.
- step_prune_miss_const_off_line: do not prune constant 7 against that
range even though its bounds and tnum admit the value.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261009-scev-minimal-rebase-v3-47-bcba9bc53ac9@gmail.com
|
|
- step_arith{32,64}_no_overflow: non-overflowing +, *, <<
operations maintain step and base.
- step_neg_value_range: Preserve the signed range of {-3, 0, 3, 6}
when intersecting with a non-power-of-two step across zero.
- step_arith{32,64}_overflow: Reset the stride and preserve valid
bounds when a {32,64}-bit operation wraps.
- step_mov32_truncate: Reset the stride when MOV32 discards nonzero
upper bits from positive values.
- step_mov32_negative: Reset the stride when MOV32 zero-extends
the low 32 bits of negative signed values.
- step_movsx: Reset the stride when sign extension changes the
register value.
- step_movsx_bounds_on_line: Keep bounds and stride when sign
extension does not change the register value.
- step_bounds_resync_after_arith: Snap bounds refined from var_off
back onto the line.
- step_linked_regs_add64: Translate both bounds and base when
propagating a comparison's refinement through a scalar-ID delta.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261009-scev-minimal-rebase-v3-46-bcba9bc53ac9@gmail.com
|
|
A static callee contributes the registers its code reads at the entry,
a global callee its declared arguments.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261009-scev-minimal-rebase-v3-45-bcba9bc53ac9@gmail.com
|
|
Check that passing a stack pointer to a no-op global callee produces:
- use, def and may_def marks for a fixed-size memory argument.
- No stack effects for an untrusted BTF pointer argument.
This verifies that call effects follow the declared argument types
rather than the callee's body.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261009-scev-minimal-rebase-v3-44-bcba9bc53ac9@gmail.com
|
|
Tests for helper arguments whose access size or read/write effects
depend on the map:
- helper_map_output_known_size checks that for a known map access
MEM_WRITE | MEM_UNINIT produces a must_write mark.
- helper_map_output_merged_sizes checks that when maps with different
value sizes merge, MEM_WRITE | MEM_UNINIT produces use and may_def
marks over the maximum value size, without a def mark.
- helper_bloom_peek_input checks that for bpf_map_peek_elem()
on a bloom filter the buffer is treated as an input.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261009-scev-minimal-rebase-v3-43-bcba9bc53ac9@gmail.com
|
|
Check stack write tracking for atomic operations:
- Restored by 64-bit load-acquire, xchg, fetch_add and cmpxchg.
- Spilled by store-release and restored by a plain load.
- Restored from a caller's stack slot by xchg in a callee.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261009-scev-minimal-rebase-v3-42-bcba9bc53ac9@gmail.com
|
|
Liveness analysis should produce may_def marks for read-modify-write
atomic operations:
- lock *(u64 *)(r10 -8) += r1
- r1 = atomic64_fetch_add((u64 *)(r10 -8), r1)
- r1 = atomic64_xchg((u64 *)(r10 -8), r1)
- r0 = atomic64_cmpxchg((u64 *)(r10 -8), r0, r1)
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261009-scev-minimal-rebase-v3-41-bcba9bc53ac9@gmail.com
|
|
Branches to cover:
- per-insn writes (record_stack_access_off):
* off_cnt==1, fully covered slots -> def + may_def (equal)
* partial coverage -> may_def only (no def)
* off_cnt 2-4, precise offsets -> may_def only, per offset
- whole-frame writes:
* off_cnt==0 (offset lost) -> may_def SPIS_ALL
* ARG_IMPRECISE (frame lost) -> may_def SPIS_ALL per frame
- merge_instances():
* both passes touch a frame -> may_def union
* one pass leaves a frame untouched -> may_def preserved
- merge_may_write(): a callee's writes into ancestor frames are
summarized as may_def at the caller's call instruction
- analyze_subprog() -> may_def stack slot range
when callback is unknown.
Tests exercising them:
- may_write_two_precise_offsets - off_cnt 2-4 precise multi-offset
- imprecise_frame_write - cross-frame ARG_IMPRECISE write,
plus its caller-side summarization
- merge_preserves_may_write - map-value vs stack pointer at two
callsites: union is preserved when
one pass does not touch the frame
- must_write_not_same_slot - off_cnt==0 whole-frame may_def
- two_byte_write_no_kill - partial write, may_def without def
- kfunc_iter_stack_liveness - full-slot def == may_def
- helper_output_unknown_size - output size unknown to liveness,
may_def without use or def
- caller_stack_write,
conditional_stx_in_subprog,
shared_instance_must_write_overwrite
- callee writes summarized as may_def
at the call site
- bpf_loop_two_callbacks - call to unknown callback marks full
stack range as may_def.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261009-scev-minimal-rebase-v3-40-bcba9bc53ac9@gmail.com
|
|
Validate changes from:
"bpf: allow subrange relations for PTR_TO_STACK in regsafe()".
Add a test case that checks that a widened stack pointer with range
[-72, -16] is considered a superset of a stack pointer with range
[-64, -16].
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261009-scev-minimal-rebase-v3-39-bcba9bc53ac9@gmail.com
|
|
The bpf_iter/unix and skc_to_unix_sock tests index sun_path but bound
that index by sizeof(struct sockaddr_un). This allows indices 108 and
109 into the 108-byte array. With SCEV loop widening, the variable-offset
BTF access check rejects the resulting range.
Bound the index by sizeof(sun_path) in both tests. Runtime behavior for
valid UNIX addresses is unchanged.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261009-scev-minimal-rebase-v3-38-bcba9bc53ac9@gmail.com
|
|
Same as __msg, but expects the match on the line immediately after
the last match, not just any later line.
E.g.:
__msg_next("a")
__msg_next("c")
This would match consecutive output "a\n" "c\n",
but would not match output "a\n" "b\n" "c\n".
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261009-scev-minimal-rebase-v3-37-bcba9bc53ac9@gmail.com
|
|
Loop-bound computation and register widening
===========================================
During the main verification pass, use SCEV expressions and register
values at the entry to the loop to estimate the number of iterations
the loop might execute. Use this number to widen loop induction variables.
For example:
0: r7 = 5;
1: r6 = 0;
2: while (r6 < 3) { // latch
3: if (r6 == 1)
4: r7 = 10;
5: r6 += 1;
6: }
8: use(r7);
- Scalar evolution expressions (SCEVs) at the latch are:
- r6 = (linear r6 1)
- r7 = (any r7 10)
- For this pre-condition loop, this corresponds to the linear inequality
`r6_entry + n < 3`. With r6_entry = 0, the inequality is `n < 3`.
- Hence infer that r6's range at line 2 is [0,3], including the final
failed test. The body starts with r6 in [0,2].
- r7's (any r7 10) expression means that its initial value 5 from line 0
is joined with 10 from line 4: it is widened to [5,10] within
and after the loop.
Currently such inference is supported for:
- Reducible loops with one backedge and complete backedge/exit lists.
- A linear latch dominating the backedge, with one successor exiting
the loop. Additional exits are allowed; they can shorten execution.
- 64-bit arithmetic and comparisons whose continuation condition
normalizes to signed/unsigned < or <=, or !=.
- Initial counter, latch offset, slope and bound evaluable as constants
at loop entry, yielding a finite, representable iteration bound.
- Values with an eligible type at loop entry, e.g. SCALAR_VALUE or
PTR_TO_STACK, but not PTR_TO_CTX.
In order for widening to proceed, every register live at the loop
entry has to have a SCEV that can be widened. Currently, the following
expressions are supported:
- (linear <self> <step>): entry value + iteration# * constant step.
- (reg <self>): loop invariant; keep its entry value unchanged.
- (any ...): join the entry value with immediates or invariant registers;
types must match, and immediates require SCALAR_VALUE.
Main verification loop integration
==================================
Before proceeding with the usual instruction processing, do_check()
takes the following steps at loop entries (H):
- Checks whether the loop can be widened and obtains its
iteration bound (scev.c:bpf_compute_loop_iters()).
- If it can, saves an unwidened checkpoint E0, which will be used as
a base for deriving widened values throughout this invocation of
the loop.
- Widens registers in the current state using E0 and SCEV expressions
computed for H (scev.c:bpf_widen_scev_regs()).
- Pushes a per-frame loop-stack record to the current verifier state.
For terminating loops, the record stores E0 and the iteration bounds.
- Lets is_state_visited() create a new checkpoint W at the same
header, now containing the widened values, and proceeds with
normal checking.
- On an exit edge, pops the loop record and verifies the exit
path normally.
is_state_visited() treats a loop proven to terminate like an
iterator-based loop: RANGE_WITHIN comparison attempts to establish
convergence against a checkpoint whose exploration is still
in progress.
For example:
r0 = 0;
H: r0 += 1;
if (r0 != 3) goto H;
exit;
first H: prove H_count = 3; save E0(r0 = 0)
widen r0 to [0,2]; push the loop record
save W(r0 = [0,2])
body: r0 becomes [1,3]; fork at the conditional
exit: r0 = 3; pop the loop record and check exit
backedge: r0 = [1,2]; clamp, then match with W and prune
Clamping
--------
Independent register ranges lose correlations between induction
variables. For example:
r6 = 0; r7 = 0;
do { r6 += 1; r7 += 2; } while (r6 < 3);
use(r7);
header: r6 in [0,2], r7 in {0,2,4}
body: r6 in [1,3], r7 in {2,4,6}
backedge: r6 in [1,2], r7 still in {2,4,6}
The latch narrows r6 but does not propagate this refinement to r7.
To account for this, before comparing states on a backedge,
scev.c:bpf_clamp_scev_regs() recomputes r7's range from E0 and the
saved iteration bound, then intersects the current range {2,4,6}
with the recomputed {0,2,4}, arriving at {2,4}.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261009-scev-minimal-rebase-v3-34-bcba9bc53ac9@gmail.com
|
|
Tests for multiple analyses performed by the verifier need the
verifier log to contain analysis results alongside the program
disassembly. This patch moves the log-level-2 program dump from
bpf_compute_live_registers() to a standalone function called from
bpf_check(), in order to provide a common logging function for such
analyses (and thus avoid printing program disassembly multiple times).
Rename the dump header from "Live regs before insn:" to
"Program dump (scc? insn#: live_regs_before):" and print available
source-line information before the corresponding instructions.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://patch.msgid.link/20261009-scev-minimal-rebase-v3-22-bcba9bc53ac9@gmail.com
|
|
This computes exact liveness for the arguments of a call to a
subprogram instead of assuming that r1..r5 are read:
- For static subprograms compute use registers at the
bpf_compute_live_registers() fixpoint phase.
- For global subprograms use BTF information.
Add a separate pass doing try_btf_prepare_func_args() in order for
bpf_subprog_info->arg_slot_cnt to be available during
bpf_compute_live_registers().
The conservative liveness marks hindered SCEV widening logic on a set
of production programs.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261009-scev-minimal-rebase-v3-11-bcba9bc53ac9@gmail.com
|
|
Track stack slots that instructions may modify, in order to account
for such writes when constructing SCEV expressions.
Consume the may_write flag in arg_access_info separately from reads and
definite writes. Include partially covered slots and all candidate
offsets; imprecise writes mark whole candidate frames. Possible writes
do not kill liveness.
Merge analyzed instances as:
may_write(dst) |= may_write(src)
must_write(dst) &= must_write(src)
For each ancestor frame f, summarize callee writes at the callsite:
may_write(callsite, f) |= OR_{i in callee} may_write(i, f)
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261009-scev-minimal-rebase-v3-4-bcba9bc53ac9@gmail.com
|
|
Return kfunc stack effects through arg_access_info while preserving the
existing BTF size lookup and read/def classification. Retain __sz/__szk
handling and conservatively mark pointer arguments as possible writes.
Remove the temporary signed-size adapter from liveness. Add liveness
assertions for privileged and unprivileged __uninit outputs.
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261009-scev-minimal-rebase-v3-3-bcba9bc53ac9@gmail.com
|
|
The term "KHO scratch" is vague and overloaded.
For one, it does not accurately describe what the memory is for. From
KHO's perspective, it is memory passed by the previous kernel that is
guaranteed to have no allocations and hence it is safe to allocate from.
For another, the term is overloaded. KHO also knows to discover other
ranges of memory that don't have any preservations and are safe to
allocate from. This is done by kho_extend_scratch(). These areas, while
called "scratch", are completely different from the chunk of memory
passed by the previous kernel for early boot allocations.
Rename "KHO scratch" to "KHO boot memory", or "KHO bootmem" in short.
Update all function names, variable names, comments, and documentation
to use this.
Call the discovered areas "noprsrv", matching what memblock would see
them as. Rename kho_extend_scratch() to reflect this.
This patch largely has no functional changes. The only functional
changes are renames of debugfs files from "scratch_phys" and
"scratch_len" to "bootmem_phys" and "bootmem_len" respectively.
Signed-off-by: Pratyush Yadav (Google) <pratyush@kernel.org>
Link: https://patch.msgid.link/20260922041321.233986-4-pratyush@kernel.org
Signed-off-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
|
|
sparc64 now selects HAVE_ARCH_SECCOMP_FILTER, so teach seccomp_bpf
how to read and write its registers: the syscall number lives in %g1
and the return value in %o0. The NT_PRSTATUS regset appends the
register window read back from the tracee's stack and has no exported
layout, so use the sparc-specific PTRACE_GETREGS64/PTRACE_SETREGS64
requests instead, which transfer the uapi struct pt_regs directly.
Errors are signaled by the carry bit in tstate with a positive errno
value in %o0, so provide a SYSCALL_RET_SET that maintains both, and
mark the arch SYSCALL_RET_SET_ON_PTRACE_EXIT since a return value
poked at entry would be overwritten by the syscall skip path, as on
powerpc.
Passes 95 of 95 on an UltraSPARC T4-1 (16 skipped for missing
optional features such as uprobes).
Link: https://github.com/sparclinux/issues/issues/11
Signed-off-by: Stian Halseth <stian@itx.no>
Reviewed-by: Kees Cook <kees@kernel.org>
Reviewed-by: Andreas Larsson <andreas@gaisler.com>
Tested-by: Andreas Larsson <andreas@gaisler.com>
Link: https://lore.kernel.org/r/20260902072745.3412940-3-stian@itx.no
Signed-off-by: Andreas Larsson <andreas@gaisler.com>
|
|
ublk_fault_inject_queue_io() queues the delay of each I/O as an
IORING_OP_TIMEOUT with a completion count of 1. Such a timeout also
completes as soon as any other CQE is posted on the ring.
With more than one I/O in flight, the completion of one I/O ends the
delay of the others, and ublk_fault_inject_tgt_io_done() reports every
early completion as "unexpected cqe res 0".
Pass a count of 0 so that only the expiry of the timer completes the
timeout.
fio 4k random reads for 10 s at queue depth 4 on a fault_inject device
with --delay_us 1000000:
before after expected
I/Os 2263170 40 40
mean latency 16.3 us 1.000 s 1 s
"unexpected cqe res 0" 2263169 0 0
Fixes: 81586652bb1f ("selftests: ublk: add generic_06 for covering fault inject")
Signed-off-by: Qiliang Yuan <odys.yuan@gmail.com>
Reviewed-by: Ming Lei <tom.leiming@gmail.com>
Link: https://patch.msgid.link/20261008-bug-ublk-selftest-fault-inject-delay-v1-2-a96af7ead06e@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
The fault_inject target delays each I/O with an IORING_OP_TIMEOUT, and
generic_06 relies on the delay to kill the server while an I/O is still
outstanding.
The timespec of the timeout is a local variable of
ublk_fault_inject_queue_io(). The SQE only records its address, and
io_uring reads it when the SQE is submitted, after the function has
returned. The timeout gets whatever the stack holds by then and expires
almost at once, so --delay_us has no effect, and generic_06 fails
because dd completes before the server is killed.
Store the timespec in the per-device fi_opts, and split the delay into
seconds and nanoseconds so that delays of a second or more give a valid
timespec.
fio 4k random reads for 10 s at queue depth 1 on a fault_inject device
with --delay_us 1000000:
before after expected
I/Os 446927 10 10
mean latency 18.3 us 1.000 s 1 s
Fixes: 81586652bb1f ("selftests: ublk: add generic_06 for covering fault inject")
Signed-off-by: Qiliang Yuan <odys.yuan@gmail.com>
Reviewed-by: Ming Lei <tom.leiming@gmail.com>
Link: https://patch.msgid.link/20261008-bug-ublk-selftest-fault-inject-delay-v1-1-a96af7ead06e@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
|
|
Add a test that checks for underflows when a parent unmaps the page first.
Also check that when the child exits the reserve count is correct.
Link: https://lore.kernel.org/all/alEJkwn5VlTTH_ZX@bender.morinfr.org/
Link: https://lore.kernel.org/aqgUdbtumaO8RiIb@bender.morinfr.org
Signed-off-by: Guillaume Morin <guillaume@morinfr.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: Breno Leitao <leitao@debian.org>
Reviewed-by: Mike Rapoport <rppt@kernel.org>
Tested-by: Breno Leitao <leitao@debian.org>
|
|
Since commit 0389c305ef56 (“selftests/mm: skip soft-dirty tests when
CONFIG_MEM_SOFT_DIRTY is disabled”), the soft-dirty test is skipped when
soft-dirty is not supported.
There is therefore no reason to exclude the test from being built on
arm64. Remove the arm64-specific exclusion.
Link: https://lore.kernel.org/20260911210611.4001419-1-yeoreum.yun@arm.com
Signed-off-by: Yeoreum Yun <yeoreum.yun@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Suggested-by: David Hildenbrand (Arm) <david@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Acked-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Tested-by: Zenghui Yu (Huawei) <zenghui.yu@linux.dev>
Cc: Alice Ryhl <aliceryhl@google.com>
Cc: Andrew Ballance <andrewjballance@gmail.com>
Cc: Christopher Li <sparse@chrisli.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Yury Norov (NVIDIA) <yury.norov@gmail.com>
|
|
DEFINE_MTREE() uses MTREE_INIT(), which initializes the tree's embedded
lock with a static spinlock initializer. If the tree is an automatic
local object, lockdep rejects its address as a non-static class key and
disables locking validation on the first lock acquisition.
Apply ASSERT_STATIC_STORAGE() to DEFINE_MTREE() to catch automatic local
definitions at compile time. For example:
void example(void)
{
DEFINE_MTREE(mt);
mtree_destroy(&mt);
}
GCC reports:
error: initializer element is not constant
name##_storage_check = &(name)
^
note: in expansion of macro 'ASSERT_STATIC_STORAGE'
ASSERT_STATIC_STORAGE(name)
note: in expansion of macro 'DEFINE_MTREE'
DEFINE_MTREE(mt);
File-scope definitions and static local trees remain valid. Automatic
local trees must instead use mt_init() or mt_init_flags(). Direct uses of
MTREE_INIT() and MTREE_INIT_EXT() are unchanged.
Replace the local DEFINE_MTREE() in the interval-tree span test with a
plain declaration; the test already initializes the tree with
mt_init_flags() before use. Convert the three local Maple Trees in the
userspace radix-tree tests to mt_init().
Validated file-scope and static local definitions with a kernel object
build, and confirmed that an automatic local definition fails to compile.
The Maple Tree test and region allocation benchmark objects also build
with lockdep enabled.
Link: https://lore.kernel.org/20260911221444.1523311-4-ynorov@nvidia.com
Signed-off-by: Yury Norov <ynorov@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Cc: Alice Ryhl <aliceryhl@google.com>
Cc: Andrew Ballance <andrewjballance@gmail.com>
Cc: Christopher Li <sparse@chrisli.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
|
|
DEFINE_IDA() uses IDA_INIT(), which initializes the embedded XArray lock
with a static spinlock initializer. For an automatic local IDA, lockdep
cannot use the lock address as a persistent class key and reports "INFO:
trying to register non-static key" before disabling itself.
Apply ASSERT_STATIC_STORAGE() to DEFINE_IDA() so that this misuse is
rejected at compile time. For example:
void example(void)
{
DEFINE_IDA(ida);
ida_destroy(&ida);
}
GCC reports:
error: initializer element is not constant
name##_storage_check = &(name)
^
note: in expansion of macro 'ASSERT_STATIC_STORAGE'
ASSERT_STATIC_STORAGE(name)
note: in expansion of macro 'DEFINE_IDA'
DEFINE_IDA(ida);
File-scope definitions and static DEFINE_IDA() within a function remain
valid. Automatic local IDAs must instead be initialized with ida_init().
Direct uses of IDA_INIT() are not covered by this declaration check.
Convert the five automatic local IDAs in the userspace radix-tree tests to
ida_init() so they satisfy the new requirement.
Validated file-scope and static local definitions with a kernel object
build, and confirmed that an automatic local definition fails to compile.
Link: https://lore.kernel.org/20260911221444.1523311-3-ynorov@nvidia.com
Signed-off-by: Yury Norov <ynorov@nvidia.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Cc: Alice Ryhl <aliceryhl@google.com>
Cc: Andrew Ballance <andrewjballance@gmail.com>
Cc: Christopher Li <sparse@chrisli.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
|
|
The XFS setup for ./khugepaged all:file checks that the kernel supports
XFS but not that mkfs.xfs is installed. When CONFIG_XFS_FS=y and xfsprogs
is missing, mkfs.xfs and mount both fail, but
SPLIT_HUGE_PAGE_TEST_XFS_PATH is assigned from mktemp -d and stays set.
The test then runs against plain tmpfs and six subtests fail.
The script already has a skip path for this test, but it never runs
because the path variable is always set. Assign
SPLIT_HUGE_PAGE_TEST_XFS_PATH only once mkfs.xfs and mount have both
succeeded, and remove the image and directory otherwise.
Link: https://lore.kernel.org/20260912202903.16157-1-jaeyeon.lee.dev@gmail.com
Signed-off-by: Jaeyeon Lee <jaeyeon.lee.dev@gmail.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Suggested-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Reviewed-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Assisted-by: LLM
Cc: Shuah Khan <shuah@kernel.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Michal Hocko <mhocko@suse.com>
|
