| Age | Commit message (Collapse) | Author |
|---|
|
Pull bpf fixes from Alexei Starovoitov:
- Remove BPF_F_ARENA_SCALAR flag and .arena.data section in libbpf that
were added for rust-bpf in this merge window. We found a cleaner
approach that does it on compiler side instead of kernel/libbpf. No
released kernel has them (Alexei Starovoitov)
- Fix bpf_send_signal_task() returning -EPERM when it is called from a
kernel thread (Aditya Sharma)
- Various fixes (Alexei Starovoitov):
- Use emit_rsb_call() to call bpf_unwind_target() in x86 JIT to fix
objtool warning during the build
- Fix scalar evolution: clear delta of a widened register, don't
widen a pointer to a buffer, don't clamp a register whose type
changed in the loop, and reset the range of a clamped packet
pointer
- Finish the insn in the state that maybe_fork_scalars() splits
off, so that pruning doesn't drop it at the head of a counted
loop
- Preserve the registers that carry stack arguments across
arch_bpf_timed_may_goto() on x86-64, arm64 and riscv64
- Don't re-sort subprog_info[] for every new subprogram
- Fix flaky map_kptr and rcu_reentry selftests
- Use memmove() in bpf_skb_load_bytes(), bpf_skb_load_bytes_relative()
and bpf_xdp_load_bytes(), since the destination can be a slice of the
same packet (Ayushman Rout)
- Fixes and follow-ups for scalar evolution: reject pointer operands in
32-bit recurrences and latches, don't intersect with the current
register range on clamp to avoid a false 'verifier bug', avoid false
infinite loop reports, and recognize a 32-bit copy of the induction
variable (Eduard Zingerman)
- Fix out-of-bounds write to btf->types[] for split BTF and propagate
btf_add_type() errors in btf_check_all_metas() (Feng Yang)
- Fix use-after-free in bpf_timer callback that fires after its inner
map was freed (Mani Tofigh)
- Fix x86 JIT stopping before jumps converge: size backward jumps by
the insns they jump over (Mykyta Yatsenko)
- Fix use-after-free of struct_ops trampoline: dissociate struct_ops
progs before the grace period of the map (Qiliang Yuan)
- Validate stack argument reads against the caller state to fix
out-of-bounds read in the verifier, and reject callbacks with stack
arguments (Weiming Shi)
- Tie borrowed iterator results and pointers loaded from trusted fields
to the lifetime of their source, so that iterator next, destroy and
reference release invalidate them (Xu Yunxiang)
* tag 'bpf-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf: (48 commits)
bpf: Don't re-sort subprog_info[] for every new subprogram
bpf: Fix out-of-bounds btf->types write for split BTF
selftests/bpf: Test stack arguments across may_goto
bpf: Keep stack arguments in arch_bpf_timed_may_goto()
selftests/bpf: Test maybe_fork_scalars() at the head of a counted loop
bpf: Finish the insn in the state that maybe_fork_scalars() splits off
selftests/bpf: Test the clamp of pointers in scalar evolution
bpf: Reset the range of a packet pointer that scalar evolution clamps
bpf: Don't clamp a register whose type changed in the loop
selftests/bpf: Test a loop that moves a pointer to a buffer
bpf: Don't widen a pointer to a buffer in scalar evolution
selftests/bpf: Avoid a page fault in rcu_reentry while the handler is armed
selftests/bpf: Wait for the last maps of map_kptr to be freed
selftests/bpf: Test that a widened register is not linked
bpf: Clear delta of a register that scalar evolution widens
bpf, x86: Remove r0_on_stack from fetching atomics in arena
bpf: Remove BPF_F_ARENA_SCALAR
selftests/bpf: Remove tests for arena access through numbers
libbpf: Keep format strings of bpf_printk() in .rodata
libbpf: Remove .arena.data
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/docs/linux
Pull more documentation updates from Jonathan Corbet:
"A handful of late-arriving changes, most of which were ready ahead of
the merge window:
- More Portuguese translation updates
- Some docs-build and self-test fixes
- Bring documentation for the user-space access primitives
in include/linux/uaccess.h into the docs build.
This revealed a number of malformed kerneldoc comments,
which are fixed in a subsequent tweak to uaccess.h"
* tag 'docs-7.4-2' of git://git.kernel.org/pub/scm/linux/kernel/git/docs/linux:
uaccess: fix some erroneous kerneldoc comments
docs/translations/pt_BR: admin-guide: add efi-stub.rst
docs/translations/pt_BR: admin-guide: add kernel-parameters.rst
docs/translations/pt_BR: admin-guide: add bootconfig.rst
docs/translations/pt_BR: admin-guide: add perf-security.rst
docs/translations/pt_BR: admin-guide: add LSM/index.rst
docs/translations/pt_BR: admin-guide: add hw-vuln/index.rst
docs/translations/pt_BR: admin-guide: add cputopology.rst
docs/translations/pt_BR: admin-guide: add sysctl/index.rst
docs/translations/pt_BR: admin-guide: add sysfs-rules.rst
docs/translations/pt_BR: admin-guide: add features.rst
docs: pt_BR: process: translate userspace debugging guide
docs: kerneldoc.py: preserve headings in kernel-doc output
docs: xforms_lists: Handle DECLARE_KHOSER_PTR() struct members
docs: test_doc_build: don't select urllib3 yanked version
docs: test_doc_build: address pylint warnings
docs: test_doc_build: update it to support newer Sphinx versions
docs: test_doc_build: better handle python version
docs: admin-guide: Fix typo in kernel-parameters.txt
docs: core-api: include documentation from linux/uaccess.h
|
|
The format strings of bpf_printk() moved from .rodata to .rodata.str for
one reason. .arena.data moved .rodata into arena, and helpers cannot read
strings from arena.
.arena.data is removed. Move the strings back to .rodata. As a result, an
object that has its own .rodata has one map less.
This reverts commit 373e1223ff00
("libbpf: Keep format strings of bpf_printk() in .rodata.str"). libbpf
1.8.0 has the strings in .rodata.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261027190314.1440199-4-alexei.starovoitov@gmail.com
|
|
Rust has no address spaces. Thus a global variable cannot have the __arena
attribute. The .arena.data section was a workaround. It told libbpf to do
two things:
- Move .data, .bss and .rodata into arena.
- Load the progs with BPF_F_ARENA_SCALAR.
The compiler must select the location of a variable, not libbpf. libbpf
already puts the .addr_space.1 section in arena. The __arena variables of C
are in this section, and LLVM can use it for Rust too. Until LLVM does
that, the global data of Rust progs stays in array maps.
This reverts commit 8ab58ed42b08 ("libbpf: Keep global data in arena when
the object has .arena.data"). libbpf 1.8.0 does not have the section.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261027190314.1440199-3-alexei.starovoitov@gmail.com
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next
Pull bpf updates from Alexei Starovoitov:
"Major changes:
- Introduce scalar evolution (SCEV) analysis in the verifier.
It infers how registers and stack slots change in a loop and lets
the verifier check bounded loops with induction variables as ranges
instead of walking every iteration. The number of processed insns
drops ten fold in programs with loops (Eduard Zingerman)
- Add 'callx' instruction: indirect call of a static BPF subprogram.
The address is taken with ld_imm64 or loaded from read-only data.
Supported by x86-64, arm64 and riscv64 JITs (Alexei Starovoitov, Pu
Lehui)
- Add unwind_goto and unwind_exit instructions, so that Rust panic or
C++ exception can run destructors of every frame it passes through.
The normal path is free of unwind code. Supported by x86-64 and
arm64 JITs (Alexei Starovoitov)
- Support programs compiled by rust-bpf: treat load and store through
a number as arena access with BPF_F_ARENA_SCALAR, accept rust names
in BTF (Alexei Starovoitov)
- Raise BPF program stack size from 512 bytes to 2 KiB on x86-64 and
arm64 (Kumar Kartikeya Dwivedi)
- Support struct, union and __int128 up to 16 bytes passed and
returned by value in BPF subprograms and kfuncs, and fix x86-64 and
arm64 trampolines for such arguments (Yonghong Song)
- Introduce a common way to attach struct_ops to a cgroup via
bpf_link. bpf_tcp_ops is the first user and mirrors TCP sock_ops
callbacks (Martin KaFai Lau, Amery Hung)
- Describe inlined functions in BTF. New kinds tell where the inline
sites are and how to fetch the arguments there.
CONFIG_DEBUG_INFO_BTF_INLINE keeps it apart from the main BTF in
.BTF.inline section or in btf_vmlinux_inline.ko (Alan Maguire)
- Fix many memory safety, lifetime and locking bugs in BPF core,
verifier, iterators, local storage, dynptr, devmap, cpumap and
seg6local (Josef Bacik, Alexei Starovoitov)
- Diff stat is dominated by selftests and by removal of
lib/test_bpf.c. Its 16k lines of instruction tests were converted
to BPF selftests that BPF CI runs on every architecture it covers
(Daniel Borkmann)
Other features and fixes:
- Add KASAN checks to memory accesses of JITed programs on x86-64
(Alexis Lothoré)
- Complete unification of helper, kfunc and subprog argument checks
in the verifier (Amery Hung)
- Add bpf_copy_from_user_mm[_str]() kfuncs (Anastasios Papagiannis)
- Various features and fixes (Daniel Borkmann):
- Add .bpf keyring for program signature verification, modelled
after dm-verity keyring, and support ML-DSA in signing tools
- Emit shorter and faster code in x86-64 JIT for patterns that
LLVM generates
- Fix various issues in gotox handling
- Let BPF LSM set xattrs at inode creation with bpf_inode_init_xattr()
(David Windsor, Daniel Borkmann)
- Use sleepable page allocation in sleepable arena paths. Fix races
between arena free, allocation and user page fault (Emil Tsalapatis)
- Switch BPF crypto from lskcipher to AES-CBC and AES-ECB libraries
(Eric Biggers)
- Add bpf_proactive_reclaim() kfunc (Hui Zhu)
- Render decl_tags in libbpf btf_dump and add tests for bpftool C
dump (Ihor Solodrai)
- Speed up tracing_multi attach on x86 (Jiri Olsa)
- Various features and fixes (Kumar Kartikeya Dwivedi):
- Add BPF_PROG_STREAM_OPEN command to read BPF streams through a
file descriptor with poll support
- Let kernel structs publish arena pointers with __arena type tag
- Fix __uninit kfunc output buffers and fastcall rewrite with
indirect stack accesses
- Refuse CO-RE relocations for signed programs
- Add enqueue_rcvq and dequeue_rcvq hooks to bpf_tcp_ops to adjust
sk_rcvlowat and avoid unnecessary wakeups (Kuniyuki Iwashima)
- Add bpf_ksock_sendto() kfunc and allow ksock connect and send from
bpf_wq (Mahe Tardy)
- Fix racy plt target update in arm64 bpf_arch_text_poke()
(Matthew Wood)
- Restructure BPF selftests Makefile as a layered build
(Mykola Lysenko)
- Speed up hash map lookups for u32 and u64 keys (Mykyta Yatsenko)
- Add signed div/mod, movsx, bswap and gotol to MIPS JITs, signed
div/mod to parisc JITs (Nicholas Dudar, Johan Almbladh)
- Fix stack overflow in build ID parser on 32-bit (Nickolai
Zeldovich)
- Add stack arguments, signed arena loads and gotox to riscv64 JIT.
Pu Lehui is now co-maintainer of riscv BPF JITs (Pu Lehui, Chen
Pei, Feng Jiang)
- Add bpf_call_rcu() and bpf_call_rcu_tasks_trace() kfuncs. Support
fetching and/or/xor atomics in arena on x86-64 (Puranjay Mohan)
- Fix flaky selftests, selftests with clang 23 and bpf-gcc
(Qiliang Yuan, KaFai Wan, Vineet Gupta)
- Fix builds broken by fatal resolve_btfids warnings, BTF generation
on ARC, and define arch_cmpxchg_local() on arc, hexagon and
openrisc (Sasha Levin)
- Reduce memory use of hash maps by up to 50% per element
(T.J. Mercier)
- Mark pointer fields of sched_ext ops arguments as trusted (Tejun
Heo)
- Remove legacy bpf_asm, bpf_dbg and bpf_jit_disasm tools (Xia Yan)
- Fix index overflow in queue and stack maps, arena range tree after
allocation failure, tail call loop after freplace detach, lost
signal in bpf_send_signal() (Yuan Chen)
- Many more fixes and cleanups"
* tag 'bpf-next-7.4' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next: (660 commits)
selftests/bpf: Time tc_edt from its first packet with data
selftests/bpf: Test unwind_exit of an exception callback
selftests/bpf: Tests for 32-bit scalar evolution bounds computation
bpf: Reject unwind_goto and unwind_exit in offloaded progs
bpf: Support 32-bit linear scalar evolution
bpf: Make unwind_exit of an exception callback exit in JITs
selftests/bpf: Test forbidden bpf_ksock_sendto() LSM attach
selftests/bpf: Test ksock sendto()
bpf: Add bpf_ksock_sendto() kfunc
selftests/bpf: Test unwinding from R0: R2 subprog
selftests/bpf: Add a C++ prog with exceptions
selftests/bpf: Add tests for unwind_goto and unwind_exit
bpf, arm64: Support unwind_goto and unwind_exit
bpf, x86: Support unwind_goto and unwind_exit
bpf: Add unwind_goto and unwind_exit insns
tools/bpf: Remove legacy BPF tools
bpf: Annotate data races on bpf_lru_node type
selftests/bpf: Add arena allocation race tests
bpf: Fix arena race between page free and alloc leading to incoherency
bpf: Revalidate faulted arena pages before installing user mappings
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
Pull MM updates from Andrew Morton:
- "selftests/mm: improve khugepaged coverage" (Kiryl Shutsemau):
Give khugepaged a proper workout. Extend the tests to all supported
mTHP orders, check exactly which windows collapsed, and make scan
completion deterministic. Race collapse against faults, pinning, fork
and remap; verify that the contents survive.
Add coverage for whole-page-table zaps and memory pressure too.
- "selftests/mm: separate GUP microbenchmarking from functional testing"
(Sarthak Sharma):
Untangle GUP benchmarking from correctness testing. Move the
benchmark into tools/mm, add a standalone harness-based GUP selftest,
and move shared helpers into tools/lib/mm. Give each tool one job
instead of mixing timing measurements with correctness checks.
Cover the mapping, pinning and batch-size combinations explicitly.
- "mm: Switch device DAX to section-based vmemmap optimization" (Muchun
Song):
Apply section-based vmemmap optimization to device DAX. Rework the
backing metadata layout to save memory while preserving the handling
of device memory hotplug and its different alignment and lifetime
requirements.
- "mm: Introduce section-based vmemmap optimization for HugeTLB"
(Muchun Song):
Cut the memory consumed by HugeTLB's struct page array with
section-based vmemmap optimization. Arrange the metadata so large
huge pages need fewer backing pages, while keeping initialization,
teardown and the various hugepage configurations straight.
- "mm/huge_memory: clean up and decouple the anon and file split
helpers" (Kairui Song):
Separate anonymous and file-backed folio splitting, which have rather
different locking and bookkeeping requirements. Move the work into
separate helpers, drop dead checks and tidy swap-cache handling. Fix
error paths, including one that could leave a folio frozen.
Keep the swap cache cases separate from file-cache splitting.
- "mm: Unconditional per-VMA locks and cleanups" (Dave Hansen):
Make per-VMA locking unconditional and clean up the old conditional
code around it. This removes configuration-dependent plumbing and
lets callers depend on one locking model rather than carrying both
cases along.
- "memcg: remove the v1 soft limit" (Shakeel Butt):
Remove the old cgroup v1 memory soft-limit machinery. This has long
since been superseded by the newer memory-control interfaces; hauling
around a separate reclaim scheme for it no longer buys us much.
- "read proc/pid/smaps_rollup under per-vma lock" (Suren Baghdasaryan):
Read /proc/PID/smaps_rollup using per-VMA locks where possible,
rather than holding mmap_lock across the entire walk. Retain the
contention fallback and add tearing tests. This considerably reduces
latency when another thread is busy changing its mappings.
- "mm/damon: introduce data access-as-a-data attribute" (SJ Park):
Teach DAMON to expose memory access observations as data attributes,
making them available to the probe and filtering machinery. Extend
the interfaces, documentation and tests so the new information can be
used rather than merely collected.
- "Remove PG_private by using page/folio->private checks instead" (Zi
Yan):
Get rid of PG_private and use the folio's private pointer to say
whether private data is attached. Convert the filesystem users,
including erofs, f2fs and md, and clean up the old accessors and
flags. There are a few special cases, notably swapcache and hugetlb,
which need explicit care.
- "mm: make userland page table freeing RCU-safe" (Lorenzo Stoakes):
Make freeing userspace page tables safe against RCU readers. Sort out
the page-table lifetime and locking rules so walkers cannot stumble
over freed tables while mappings are being torn down.
- "make unused huge shrinker memcg aware" (Qi Zheng):
Make shmem's unused-huge-folio shrinker aware of memory cgroups, so
reclaim can find those folios under the right memcg. Wire up the
per-memcg tracking and its lifetime rules, and give the shrinklist
its own lockdep class to avoid spurious dependency reports.
- "mm: Drop pxd_ERROR()" (Anshuman Khandual):
Remove the old pxd_ERROR() page-table checks and their
architecture-specific baggage. Simplify the common page-table
interfaces and callers now that these checks no longer serve a
useful purpose.
- "memcg: group struct fields by access pattern" (Shakeel Butt):
Rearrange memcg's hot and cold fields according to how they're
actually accessed. Keep frequently updated counters and their
neighboring state from fighting over cachelines and reduce needless
cacheline traffic.
- "mm/damon/core: fix the size charged for a filter-trimmed region"
(Donggeun Yoo):
Fix DAMOS quota accounting when a filter trims a monitoring region.
Charge only the part left after filtering, not the original region,
and add KUnit coverage. Previously the excluded memory could eat into
the quota and make schemes run more slowly than requested.
- "mm/damon: add kunit and selftests for probes and probe weights" (SJ
Park):
Add more tests for DAMON probes and their weights, including
validation and the results reported to userspace. This is mostly
about catching regressions in the newer probe infrastructure before
they become harder to diagnose.
- "mm: memcontrol: constify the read side of the read side of the memcg
API" (Tal Zussman):
Mark the read-only parts of the memcg interface const, from folio and
lruvec accessors to counters and swap helpers. Most of this is
mechanical, but it makes the contracts clearer and lets the compiler
catch accidental writes.
- "mm/damon/vaddr: support {prep,apply}_probes" (SJ Park):
Support the new prepare/apply probe operations in DAMON's
virtual-address backend. This brings vaddr monitoring into line with
the probe infrastructure and makes the added probe types useful there
too.
- "mm/mglru: clean up folio counters and flag usage" (Kairui Song):
Simplify MGLRU's folio counters and flag handling. Remove redundant
accounting paths and make the state transitions easier to follow
without changing the intended reclaim policy.
- "arch, mm/execmem: resolve confusion about set_direct_map_valid_noflush()"
(Mike Rapoport (Microsoft)):
Clarify and straighten out the set_direct_map_valid_noflush()
interface across architectures and execmem. Make the requested
mapping changes and the callers' expectations agree, rather than
relying on subtly different architecture behavior.
- "mm: zswap: free cold writeback folios promptly" (Alexandre Ghiti):
Free zswap writeback folios as soon as their disk write completes.
They are cold temporary buffers, so putting them on the LRU only
makes reclaim rediscover them later. Extend swap-cache dropbehind and
let zswap keep these folios off the LRU altogether.
- "mm: gup: cleanup gup_fast call chain" (Aristeu Rozanski):
Clean up the GUP-fast call chain and simplify how the fast-path
helpers pass their arguments and results around. Keep the page-pin
behavior the same while making the call sequence easier to maintain.
- "mm: optimize zone-device memmap initialization" (Li Zhe):
Speed up initialization of zone-device struct pages. Avoid redundant
work when setting up large device-memory ranges, where touching every
descriptor can get expensive.
- "mm/zswap: shrink zswap_entry via a pool id" (Jianyue Wu):
Reduce the size of zswap_entry by storing a compact pool identifier
instead of a full pointer. Keep the pool lookup and lifetime handling
correct while saving per-entry metadata.
- "mm/damon: support access monitoring of hugetlb-backed memory"
(Krishna Iyer):
Let DAMON monitor hugetlb-backed memory too. Extend the relevant
access checks and add coverage so those mappings no longer fall
through a hole in the monitoring machinery.
- "mm/execmem: fixes and cleanups for the ROX cache" (Mike Rapoport):
Tidy execmem's read-only-executable cache and fix its handling of the
awkward allocation and release cases. The aim is to make the cache's
ownership and permission transitions more predictable.
- "zsmallc: remove old object read API" (Sergey Senozhatsky):
Drop zsmalloc's old object-read API and move its remaining users to
the supported access mechanism. This removes a duplicate interface
which we no longer need.
- "mm/mempolicy: stop copying state in the interleave paths" (Gregory
Price):
Stop copying mempolicy state in the interleave paths when a reference
to the existing policy will do. This avoids extra work and makes the
ownership rules less obscure.
- "mm/mglru: speed up inc_min_seq() and fix cold/hot inversions" (Barry
Song):
Make MGLRU's generation advancement cheaper and fix cases where cold
and hot folios end up in the wrong order. Adjust the counter and list
handling so reclaim doesn't keep passing over the better eviction
candidates.
- "mm: Fix MF_DELAYED handling on memory failure" (Lisa Wang):
Treat MF_DELAYED as a partially completed memory-failure operation,
not an outright failure. Propagate that result correctly through
shmem and the generic handler, fix the extra-reference accounting,
and add tests for the resulting SIGBUS behavior.
- "tools/mm/page_owner_sort: fix --sort, add module filter, improve
usage" (Ye Liu):
Fix page_owner_sort's --sort handling, add module-based filtering and
make the usage output more helpful. This makes the tool less
frustrating when tracking down who is holding on to pages.
- "mm/damon: move damos filter range arguments validation to core" (SJ
Park):
Move DAMOS filter range validation into the core, where all callers
can share it. Also close the sysfs read-twice races which could pass
inconsistent ranges, and add tests for invalid input.
- "mm/damon: misc cleanups" (Zenghui Yu):
Clean up a collection of small DAMON rough edges in the core,
interfaces and tests. No large behavior change here, just simpler
code and fewer opportunities for confusion.
- "mm/damon: cleanup code, add test cases, and update guidances in
docs" (SJ Park):
Clean up DAMON's core and interfaces, add missing test cases and
bring the documentation into line with the code. These changes should
make the newer facilities easier to understand and maintain.
- "mm/damon: introduce hugepage_size probe filter" (SJ Park):
Add a DAMON probe filter for hugepage size. Let users select a range
of folio sizes through sysfs, validate the range and document the new
interface, so monitoring can concentrate on memory backed by the
desired hugepage sizes.
- "selftests/mm: pagemap_ioctl test fixes and cleanups" (Zenghui Yu):
Fix some pagemap ioctl test assumptions and clean up their setup and
reporting. Make failures point to the operation that went wrong
rather than to test harness bookkeeping.
- "mm: avoid large folio splits when swap is unavailable" (Johannes
Weiner):
Avoid splitting large folios merely to attempt swap when no swap
space is available. There is little point paying for an expensive
split when reclaim cannot actually swap the resulting pages out.
- "mm: workingset: fix the shadow node budget under MGLRU" (Hui Zhu):
Fix the workingset shadow-node budget with MGLRU enabled. Keep the
shadow entries within the intended limits so refault tracking does
not quietly consume more metadata than it should.
- "selftests/mm: TAP output and global-state fixes" (Song Hu):
Fix selftests which produce misleading TAP output or fail to restore
global settings after a run. A test ought to leave the system as it
found it, particularly when a whole suite follows behind.
- "mm: remove page_swap_entry()" (Tal Zussman):
Remove page_swap_entry() and convert its remaining users to the folio
and swap-entry helpers. Lose an old page-level interface and make the
swap paths a little more consistent.
- "mm: refactor zonelist constructors and iterators" (Gregory Price):
Refactor the zonelist construction and walk helpers to reduce
duplicated logic. Keep the allocation ordering rules intact while
making it clearer how zones are selected.
- "mm: kmemleak: batch the struct page scan" (Breno Leitao):
Scan adjacent struct pages in batches instead of taking kmemleak_lock
once for every single page. This preserves scan coverage and the
lock-hold bound, while cutting overhead substantially on debug
kernels.
- "mm/damon: misc improvements in tests and documents" (Zenghui Yu):
Fix some DAMON selftests which leave kdamond running or fail under
Python safe-path mode, and improve the KUnit coverage and
documentation. Nothing especially glamorous, but these are worth
doing.
- "mm/damon: introduce probe_hits_wsum DAMOS core filter" (SJ Park):
Add a DAMOS core filter using weighted probe-hit counts. This gives
schemes another way to select regions based on what the monitoring
probes observed.
- "mm/mglru: fix ineffective memory protection for non-kswapd reclaim"
(Ridong Chen):
Fix MGLRU's memory protection handling for reclaim outside kswapd.
The protection settings weren't having the intended effect in these
paths, leaving protected memory more exposed than requested.
- "mm/damon: improve readability, clarity and test coverage" (SJ Park):
Simplify a handful of DAMON core helpers, clarify their behavior and
expand test coverage. There are no intended userspace-visible changes
in this batch.
- "mm: replace PF_KCOMPACTD/PF_KSWAPD with kthread_func()" (Kefeng
Wang):
Stop identifying reclaim and compaction workers through the PF_KSWAPD
and PF_KCOMPACTD task flags. Use their kthread functions instead and
remove the redundant flag plumbing.
- "mm/hugetlb: fix surplus accounting and availability checks during
demotion" (Longlong Xia):
Fix HugeTLB surplus accounting and availability checks when demoting
huge pages. Keep the counters and reservation decisions consistent as
one huge page turns into several smaller ones.
- "mm/damon: add kunit tests for probe_hits handling and probe params
validation" (Jason Angelov):
Add KUnit tests for DAMON probe-hit handling and invalid probe
parameters. These cover edge cases that were too easy to miss in
ordinary monitoring tests.
- "mm/damon: fix misc bugs in kunit, quota goals and sysfs refresh_ms"
(SJ Park):
Fix a few DAMON bugs in KUnit tests, quota-goal handling and the
sysfs refresh interval. Small fixes, but the incorrect results and
control behavior were real.
- "mm: zswap: optimize zswap invalidate and store" (Kefeng Wang):
Simplify zswap's invalidate and store paths to avoid avoidable
lookups and work. Keep the entry lifetime rules intact while making
those frequent operations cheaper.
- "mm: replace custom ratelimiting logic" (Pedro Falcato):
Replace hand-rolled MM rate limiting with the common kernel helpers.
There is no need to maintain another local version of the same
mechanism.
- "mm/vmalloc: minor cleanups" (Ye Liu):
Tidy some vmalloc initialization and reporting code, and pull
duplicated vmap-area insertion work into a helper. No functional
changes are intended.
- "mm/mglru: clean up isolate_folios for readability and clarity"
(Ridong Chen):
Clean up MGLRU's isolate_folios() so the decisions are easier to read
and verify. Keep reclaim behavior unchanged while removing
unnecessary twists in the control flow.
- "Catch automatic storage in IDA and Maple Tree definitions" (Yury
Norov):
Catch IDA and maple-tree definitions placed in automatic storage
where the lifetime rules are wrong or easy to get wrong. Add checks
so such mistakes are harder to slip through unnoticed.
- "mm, swap: some random fixes and cleanups" (Kemeng Shi):
Fix and simplify a few unrelated swap paths, removing redundant work
and tidying up error handling. A small collection of things which
were better fixed than left lying around.
- "mm/page_io: folio conversion cleanups" (Tal Zussman):
Finish some page-to-folio conversions in swap I/O, removing obsolete
wrappers and using folio operations consistently. The I/O behavior is
intended to remain the same.
- "mm/swap: skip empty clusters in the swapoff scan" (Youngjun Park):
Skip empty swap clusters during swapoff instead of walking slots
which have nothing to release. This avoids pointless work on sparsely
used swap devices.
- "mm: trace: decode architecture-specific VMA flags" (Meijing Zhao):
Teach the VMA trace output to decode architecture-specific flags,
rather than leaving them as unexplained bits. That should make
tracing a good deal more useful when debugging unusual mappings.
- "mm, swap: don't spin on a bad swap entry" (Breno Leitao):
Stop repeatedly retrying a swap entry which is already known to be
bad. Make the error path terminate cleanly instead of wasting CPU
chasing an entry that cannot be used.
- "mm/page_isolation: fix UBSAN shift-out-of-bounds in
isolate_single_pageblock" (Qi Xi):
Fix an out-of-range shift in isolate_single_pageblock() and tighten
its associated boundary handling. UBSAN found the issue; the result
should be more robust pageblock isolation.
- "mm: khugepaged: fix tracepoint UAF" (Vernon Yang):
Fix a use-after-free in khugepaged tracepoint handling. Make sure the
information recorded by tracing stays valid for as long as the trace
event needs it.
- "mm/damon: fix DAMOS bugs in core, paddr and vaddr" (SJ Park):
Fix several DAMOS problems across the core and physical- and
virtual-address backends. The changes close error cases and bring the
implementations back into agreement.
- "mm/hugetlb_cgroup: move the per-node usage along with the folio"
(Hongfu Li):
Move HugeTLB per-node usage accounting when folios migrate between
nodes or are reparented to another cgroup. Otherwise numa_stat can
retain stale charges and even underflow when the folio is freed.
- "docs/core-api: memory-allocation: add k[mz]alloc_obj() and clarify
kmalloc" (Mike Rapoport):
Document the typed k[mz]alloc_obj() allocation helpers and clear up
some kmalloc guidance. Make it easier for callers to choose the right
interface.
- "selftests/mm: Validate selections and scope memfd_secret setup"
(Tianyi Chen):
Check selftest selections before running and confine memfd_secret
setup to the cases that actually need it. Avoid confusing failures
caused by the test harness itself.
- "mm/damon: move zero damos quota target_value handling to the core
layer" (SJ Park):
Handle a zero DAMOS quota goal target in the core rather than making
each user of the interface invent its own rule. Keep quota decisions
consistent across callers.
- "mm/damon: Introduce a huge page collapsing mechanism using auto
tuning" (Asier Gutierrez):
Add an auto-tuned DAMON mechanism for collapsing huge pages. Use
monitoring feedback to decide when collapse is worthwhile rather than
forcing one fixed policy onto every workload.
- "kmemleak: fix stale documentation and raise the verbose default"
(Breno Leitao):
Bring the kmemleak documentation up to date and require a third
confirming scan before verbose kernels print a suspected leak. The
extra scan helps suppress transient false positives in noisy debug
builds.
- "mm/memory: improve insert_pages() error handling" (Avi Weiss):
Tighten insert_pages() error handling so partial failures are
reported and cleaned up consistently. The caller should not have to
guess how far the operation got.
- "mm/damon: allow NULL or unstarted damon_ctx parameter for
damon_call()" (SJ Park):
Make damon_call() handle NULL and not-yet-started contexts safely.
Simplify the rules for callers and avoid an indefinite wait on an
invalid state.
- "mm/damon: introduce pgidle_set probe filter type" (SJ Park):
Add a pgidle_set DAMON probe filter, giving monitoring users another
way to select pages by their idle-tracking state. Wire it through the
filter interfaces and tests.
- "mm/damon/core: remove unused helper functions" (Cheng-Han Wu):
Remove DAMON core helpers which no longer have callers. Keep the API
surface smaller rather than retaining dead code just in case.
- "mm: stop calling pmd_folio() on special PMDs" (Gregory Price):
Avoid calling pmd_folio() on special PMD mappings that have no struct
page. The old assumption could fault in procfs walkers; use the
normal-folio checks and handle those mappings properly.
- "mm: remove three unused helpers from mm.h" (Tal Zussman):
Remove three unused helpers from mm.h. Fewer obsolete interfaces for
the next person to stumble over.
- "mm/damon: improvements in efficiency, error handling, documents"
(Liew Rui Yan):
Skip needless DAMOS quota work once the quota is full, propagate a
missing sysfs error and tidy a couple of comments. The ignored error
could make a monitoring-start request appear successful when it
wasn't.
- "docs/mm/damon/design: add explanation of nr_snapshots" (Liew Rui
Yan):
Explain DAMON's nr_snapshots accounting in the design documentation
and straighten out the associated terminology. Make it clearer what
users should expect from that counter.
- "selftests/cgroup: fixes for test_zswap on single core VM" (Wilson
Felipe Pereira):
Fix two test_zswap failures on single-CPU VMs. Wait for the exiting
child's cgroup to become unpopulated before reconfiguring it, and
avoid another test assumption which breaks under constrained
scheduling.
- "docs/ksm: fix advisor documentation and comment" (Jinjiang Tu):
Correct the KSM advisor documentation and a misleading code comment.
Keep the description aligned with how the advisor actually works.
- "Fix bugs in HugeTLB allocation when mem_cgroup_charge_hugetlb()
fails" (Ackerley Tng):
Fix HugeTLB allocation cleanup when the memcg charge fails. Make sure
failure exits undo what was allocated rather than leaving stale
accounting or resources behind.
- "mm, memcg: fix memory.peak reset clobbering other fds' watermark"
(Ridong Chen):
Fix memory.peak reset handling so resetting one file descriptor
doesn't overwrite the watermark kept for another. Keep those per-open
measurements independent as intended.
- "mm: reject zone device folios in more folio walkers" (Gregory
Price):
Reject zone-device folios in page walkers that cannot safely handle
them. Extend the existing checks rather than assuming every folio has
ordinary system-memory backing.
- "percpu: remove code with no effect" (Sang-Heon Jeon):
Remove a few pieces of percpu code which no longer do anything. There
is no reason to keep maintaining dead paths.
* tag 'mm-stable-2026-10-19-21-33' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (539 commits)
mm: shmem: assign a lock class to the unused huge shrinklist
mm: vmscan: put rotation-missed folios at the LRU tail
mm: kmemleak: scan the struct page array in MAX_SCAN_SIZE batches
mm: kmemleak: move the struct page scan into a helper
mm/damon: document that a zero sample_interval is accepted
mm/damon: fix typos in comments
mm/damon/sysfs: propagate damon_call() error in turn_damon_on
mm/damon/core: skip quota score setup when the quota is full
mm/damon/tests/core-kunit: test the size charged for a filter-trimmed region
mm/damon/core: charge only the part of a region the filter left
mm: disallow raw PFN mappings of huge/shared zeropage
selftests/mm: zap whole PTE tables in the khugepaged race harness
selftests/mm: add memory-pressure threads to the khugepaged race harness
selftests/mm: race the collapse of windows with holes
selftests/mm: add khugepaged race harness
selftests/mm: check that one khugepaged pass collapses one window
selftests/mm: run every supported collapse order by default
selftests/mm: cover a shared-source collapse write race
selftests/mm: parameterize the mixed-source collapse case by source order
selftests/mm: add order-parameterized khugepaged collapse cases
...
|
|
KHO ABI structures declare their preservation pointers with
DECLARE_KHOSER_PTR(name, type), which expands to a union of the pointer
and its physical address. kernel-doc does not know the macro and does
not see the member: it warns that an unnamed member is not described
and that the description of the member is excess.
Translate DECLARE_KHOSER_PTR(name, type) into "type name", as is done
for DECLARE_KFIFO_PTR().
Signed-off-by: Pasha Tatashin <pasha.tatashin@soleen.com>
Message-ID: <20261015020313.946940-1-pasha.tatashin@soleen.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/kbuild/linux
Pull Kbuild / Kconfig updates from Nathan Chancellor:
"Kbuild:
- Suppress compiler flags in debug info for external module builds
- Use '-fmacro-prefix-map' in more places in the build
- Merge initial set of changes from Lorenzo's speed up series, along
with some fixes found through testing:
- Avoid allocating .modinfo in vmlinux
- Speed up kallsyms by indexing symbols by token and outputting
binary data to disk directly then including in assembly via
.incbin
- Don't sort $(NM) output when unnecessary
- Only emit relocations on the final vmlinx link
- Reimplement mksysmap in C
- Move many cc-option and ld-option calls into the Kconfig stage to
cache their results
- Use pigz for gzip compression when available
- Move gen_init_cpio and gen_initramfs.sh from usr/ to scripts/
- Add a TOML configuration to scripts/container and update
documentation accordingly
- Add a generic, opt-in mechanism for checking that headers can
compile standalone
- Update machine function splitting argument for LLVM 24 rename in
AutoFDO build
- Miscellaneous minor clean ups, fixes, and hardening, mostly
found/fixed via LLMs
Kconfig:
- Turn recursive ranges into an error
- Fix savedefconfig for defaults that were adjusted for being out of
range and revert a previously applied workaround
- Improvements and hardening around integer and hexadecimal values
- Emit a warning when selecting or implying a choice symbol, which
has no effect
- Emit a warning when KCONFIG_PROBABILITY is malformed
- Introduce def_string / def_int / def_hex to mirror the existing
def_bool / def_tristate to succinctly set a value and type at the
same time.
- Add cc-option-str and ld-option-str macros to act like Kbuild's
cc-option and ld-option, returning the string when the option is
supported, allowing many checks to be moved into Kconfig
- Minor usability improvements to Kconfig tests"
* tag 'kbuild-7.4-1' of git://git.kernel.org/pub/scm/linux/kernel/git/kbuild/linux: (59 commits)
kbuild: AutoFDO: Update split machine function argument for upstream rename
kbuild: Mark the initcall ordering recipe as recursive
kbuild: Mark the gzip recipes as recursive
localmodconfig: Explicitly report boolean configs during integrity check
scripts/make_fit: Ignore schema checks when decomposing DTBs
clang-tools: Import os for broken pipe handling
kconfig: use the value argument in sym_get_near_range_bound()
kconfig: fix infinite loop in conf_choice() when no choice value is visible
kconfig: cleanup stale comment about 'keys' array
kconfig: Improve ergonomics around disabling warnings with cc-option
scripts/Kconfig.toolchain: Drop compiler symbols from dependencies
checkkconfigsymbols.py: Parse Git status output line by line
gen_init_cpio: stop parsing on lines with a missing argument list
.gitignore: drop Module.markers
.gitignore: ignore pacman package signatures
kbuild: add header check facility as a manually run static analyzer
Documentation: dev-tools: refer to user ID rather than id
Documentation: dev-tools: update container.rst with config file
scripts: add TOML config to container tool
kconfig: tests: {old,save}defconfig: Forward dynamic keyword arguments
...
|
|
Pull documentation updates from Jonathan Corbet:
"It was a reasonably busy cycle for docs; changes include:
- Add the AGENTS.md symbolic link into the top-level directory in the
hopes that LLMs will read it and produce better work
- The removal of the Assisted-by requirement, as concluded at the
Maintainer Summit
- A new "testdocs" make target to run the documentation unit tests
- Lots of work on the Portuguese, and Chinese translations; there is
now maintenance for the traditional Chinese translations as well
...and the usual pile of minor fixes, documentation updates, etc"
* tag 'docs-7.4' of git://git.kernel.org/pub/scm/linux/kernel/git/docs/linux: (125 commits)
docs: add AGENTS.md as a symlink to README
docs: kernel-parameters: adjust reserve_mem build option
docs: kernel-parameters: don't mix cmdline options & build options
docs/ja_JP: translate submitting-patches.rst (review and report tags)
docs: remove Assisted-by references
Revert "docs: landlock: fix ENOMSG condition in create_ruleset kernel-doc"
docs: kdoc: parse context_lock_struct() as struct declaration
docs: timers: hpet: Document userspace API and ioctl commands
docs: timers: hpet: Modernize file references
Documentation: sched-preemption: Add a SPDX license identifier
docs: ABI: sysfs-driver-xdata: use literal blocks for console commands
docs: ABI: sysfs-class-firmware-attributes: use literal blocks for console commands
ABI: sysfs-bus-usb: use literal block for console commands
docs/ABI: testing: aspeed-uart-routing: use literal block for console commands
docs: kernel-parameters: fix a truncated sentence for preempt=lazy
sched/doc: add a preemption model overview
docs: ABI: TMR manager: use literal blocks for console I/O
Documentation/ABI: IFS: use literal blocks for commands
ABI: sysfs-bus-pci: use literal blocks for commands
docs: translations: pt_BR: add debugging index
...
|
|
A kernel struct member tagged __arena reaches a program built against
vmlinux.h as a pointer in the kernel address space with the tag as an
attribute, struct scx_cmask __attribute__((btf_type_tag("arena"))) *cmask,
so the program casts it to the arena address space before dereferencing
it, although the verifier already hands it an arena pointer.
The tag denotes a pointer into the BPF arena, address_space(1) to the
compiler, so render it as such: __attribute__((address_space(1)))
followed by the tag attribute as before, which keeps the dump faithful
to the BTF. A program then reads the member as an arena pointer and
hands it to anything declared with its own __arena without a cast.
Nothing else changes for consumers of the C dump. Skeletons strip the
modifiers of variable types and never carried the tag. Clang without
casts between address spaces still accepts the declaration and only a
program touching such a member would need a cast there, as it did for
every arena pointer before. GCC has no address spaces for BPF and warns,
under -Wattributes, that it ignores the attribute, which the selftests
build already silences.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261019132203.3034698-15-memxor@gmail.com
|
|
A program has an arena only when one of its insns refers to the arena
map. The kernel can hand a program arena pointers all the same: as
__arena struct_ops arguments, through __arena struct members and as
kfunc return values. The verifier rejects those in a program without an
arena, since it has nowhere to place them, and so a struct_ops callback
that touches nothing but its arguments fails to load until it refers to
the arena artificially, e.g. with asm volatile("" :: "r"(&arena)).
Bind the arena of an object to every program loaded from it. The kernel
binds the maps of a continuous fd_array (fd_array_cnt) like maps an insn
refers to, so pass the arena in slot 0, which is free: a kfunc insn with
offset 0 names vmlinux BTF rather than a slot. The module BTF slots that
follow stay as they are. A feature probe keeps older kernels, which
reject a non-zero fd_array_cnt, on the previous behavior.
An object has at most one arena, libbpf refuses a second one, so there
is no ambiguity about which arena a program gets. Programs that never
touch the arena see it among their maps, and pay for the arena base
setup in their JIT prologue until a later patch spares them that; the
arena now lives as long as any program of the object. With the count,
the kernel also resolves every module BTF slot of the array for every
program of the object, so each of them holds the modules the object
calls into, not only the programs that call them. Where the JIT is
disabled or the loader lacks CAP_PERFMON, every program of such an
object now fails to load, since an arena needs both, where before only
the programs using the arena did. bpf_program__clone() binds the arena
the same way, so a program loads through veristat as it does through
bpf_object__load(). An arena exclusive to another program and the
light skeleton loader are left alone.
An arena exclusive to the program itself is bound only where the kernel
checks an exclusive map against the digest of the final program. Earlier
kernels compared a map bound through the fd_array against a digest they
had not computed yet and refused the load for the very program the map
is exclusive to, so a second probe loads a program with a map exclusive
to it through fd_array_cnt; where that fails, the program keeps binding
its arena through its insns, as before.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261019132203.3034698-12-memxor@gmail.com
|
|
bpf_program__clone() hands the kernel the fd_array of the object along
with obj->fd_array_cnt when the caller supplies neither. That counter is
not a count the kernel can use: it is the index of the next free slot
of an array built for the sparse fd_array API, where module BTF fds
start at slot 1 and slot 0 stays empty, since a kfunc insn with offset
0 names vmlinux BTF rather than a slot. A non-zero fd_array_cnt turns
the array into a continuous one, which the kernel resolves slot by
slot, and slot 0 holds 0. Cloning a program from an object that calls a
module kfunc thus fails with "fd 0 is not pointing to valid bpf_map or
btf", for every program of the object, which is what veristat runs into
on such an object. bpf_object__load() passes the array without a count.
Do the same in bpf_program__clone(), leaving the count to callers who
pass an array of their own, and rename the counter to fd_array_used so
that it no longer reads like the uapi field.
Fixes: 970bd2dced35 ("libbpf: Introduce bpf_program__clone()")
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Mykyta Yatsenko <yatsenko@meta.com>
Link: https://patch.msgid.link/20261019132203.3034698-4-memxor@gmail.com
|
|
libbpf 1.8.0 has been released [1].
Update libbpf.map and libbpf_version.h to start v1.9 development cycle.
[1] https://github.com/libbpf/libbpf/releases/tag/v1.8.0
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
Link: https://lore.kernel.org/bpf/20261005183511.201045-1-ihor.solodrai@linux.dev
|
|
A khugepaged test has to tell "not collapsed" from "not scanned yet", and
nothing in the selftests can. wait_for_scan() in khugepaged.c comes
closest: it polls full_scans until the counter has advanced by two, since
the pass in progress may already have passed the test's mm. But it only
returns in time if scan_sleep_millisecs happens to be short, and it is
private to that one test.
Add khugepaged_full_pass() to hugepage_settings, built on the same
advance-by-two wait but driven through sysfs: a store to
scan_sleep_millisecs wakes the daemon, so the barrier completes whatever
the scan cadence. A store made while the daemon is scanning rather than
sleeping is lost, so the helper keeps storing until the pass lands.
One wake completes one pass only if pages_to_scan covers every mm on the
list, so callers need it large.
Settings pushes must not start passes of their own. A store to either
sleep knob wakes the daemon, so thp_write_settings() now writes a
khugepaged knob only when its value changes.
Link: https://lore.kernel.org/20260919002451.496763-11-kirill@shutemov.name
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Tested-by: Muhammad Usama Anjum <usama.anjum@arm.com>
Assisted-by: LLM
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Barry Song <baohua@kernel.org>
Cc: "David Hildenbrand (arm)" <david@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Hugh Dickens <hughd@google.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: kernel-team@meta.com
Cc: Lance Yang <lance.yang@linux.dev>
Cc: Leon Romanovsky <leon@kernel.org>
Cc: Liam Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport (Microsoft) <rppt@kernel.org>
Cc: Nico Pache (Red Hat) <nico.pache@linux.dev>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Usama Arif <usama.arif@linux.dev>
Cc: "Vlastimil Babka (SUSE)" <vbabka@kernel.org>
Cc: Zi Yan <ziy@nvidia.com>
|
|
The page cache caps folio order at MAX_PAGECACHE_ORDER, which is below the
PMD order on arm64 with 64K pages, where a PMD is 512M. A PMD-sized page
cache folio is impossible there, so the kernel refuses these collapses:
MADV_COLLAPSE answers -EINVAL and khugepaged passes over the range. Four
shmem cases ask for a PMD-sized folio anyway, fail, and the run bails out
in the middle.
Skip the shmem and file mem types where the cap is below the PMD order.
The cap is not shmem-specific: it applies to every file folio. Add
thp_file_supported_orders() to read the orders the page cache allows.
Anonymous collapse is unaffected: its orders are not capped this way.
Link: https://lore.kernel.org/20260919002451.496763-5-kirill@shutemov.name
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Reviewed-by: Baolin Wang <baolin.wang@linux.alibaba.com>
Tested-by: Muhammad Usama Anjum <usama.anjum@arm.com>
Assisted-by: LLM
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Barry Song <baohua@kernel.org>
Cc: "David Hildenbrand (arm)" <david@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Hugh Dickens <hughd@google.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: kernel-team@meta.com
Cc: Lance Yang <lance.yang@linux.dev>
Cc: Leon Romanovsky <leon@kernel.org>
Cc: Liam Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Nico Pache (Red Hat) <nico.pache@linux.dev>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shuah Khan (Samsung OSG) <shuah@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Usama Arif <usama.arif@linux.dev>
Cc: "Vlastimil Babka (SUSE)" <vbabka@kernel.org>
Cc: Zi Yan <ziy@nvidia.com>
|
|
Move hugepage_settings.[ch] from tools/testing/selftests/mm/ to
tools/lib/mm/ so the THP and HugeTLB helpers can be shared more easily
between selftests and other tools.
Keep the helpers exposed to mm selftests through vm_util.h where possible,
and use direct <mm/hugepage_settings.h> includes for files that do not
include vm_util.h. Adjust the selftests/mm build to compile the moved
implementation from its new location.
Remove the remaining kselftest dependency by including file_utils.h
directly and using EXIT_FAILURE in the signal handler.
Link: https://lore.kernel.org/20260918112234.195857-4-sarthak.sharma@arm.com
Signed-off-by: Sarthak Sharma <sarthak.sharma@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Tested-by: Muhammad Usama Anjum <usama.anjum@arm.com>
Cc: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Barry Song <baohua@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: John Hubbard <jhubbard@nvidia.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Lance Yang <lance.yang@linux.dev>
Cc: Leon Romanovsky <leon@kernel.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Mark Brown <broonie@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Nico Pache <npache@redhat.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Zi Yan <ziy@nvidia.com>
|
|
Move read_file(), write_file(), read_num(), write_num() and
write_num_ignore_einval() out of tools/testing/selftests/mm/vm_util.c into
a new shared helper under tools/lib/mm/.
These helpers are used by mm selftests today and will also be needed by
shared hugepage helpers in subsequent patches. Move them to a generic
location so they can be reused outside selftests as well.
Keep the helpers exposed to mm selftests through vm_util.h by including
the new shared header there, and link the new helper into the selftests/mm
build.
Update the explicit x86 protection_keys 32-bit and 64-bit build rules to
preserve prerequisite paths, now that file_utils.c is built from
tools/lib/mm.
Add tools/lib/mm/ to the MEMORY MANAGEMENT - MISC entry in MAINTAINERS.
Link: https://lore.kernel.org/20260918112234.195857-3-sarthak.sharma@arm.com
Signed-off-by: Sarthak Sharma <sarthak.sharma@arm.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Acked-by: Mike Rapoport (Microsoft) <rppt@kernel.org>
Acked-by: David Hildenbrand (Arm) <david@kernel.org>
Tested-by: Muhammad Usama Anjum <usama.anjum@arm.com>
Cc: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Barry Song <baohua@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>
Cc: John Hubbard <jhubbard@nvidia.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Lance Yang <lance.yang@linux.dev>
Cc: Leon Romanovsky <leon@kernel.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Lorenzo Stoakes <ljs@kernel.org>
Cc: Mark Brown <broonie@kernel.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Nico Pache <npache@redhat.com>
Cc: Peter Xu <peterx@redhat.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Zi Yan <ziy@nvidia.com>
|
|
We're about to extend af_unix sockets and coredump code with the ability
to hand out thread-specific pidfds. Add a few simple helpers that allow
to operate on multiple struct pids up to PIDTYPE_MAX with automatic
bounds checking.
Reviewed-by: Alexander Mikhalitsyn <aleksandr.mikhalitsyn@futurfusion.io>
Reviewed-by: Kuniyuki Iwashima <kuniyu@google.com>
Signed-off-by: Andrew Halaney <andrew@amutable.com>
Link: https://patch.msgid.link/20261005-work-unix-passpidfd-v4-1-350183b6e02a@amutable.com
Signed-off-by: Christian Brauner (Amutable) <brauner@kernel.org>
|
|
Make btf_relocate() optionally return the string-offset map it builds
while relocating a BTF relative to a distilled base. A subsequent split
BTF object can use that map to rebase references to base strings.
Also account for the accumulated base string offset when assigning a
new BTF base. This is required when the base is itself split.
Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261009112324.1442319-5-alan.maguire@oracle.com
|
|
bpf_printk(), BPF_SNPRINTF(), BPF_SEQ_PRINTF() and bpf_stream_printk()
copy the format into a static const array, which the compiler puts into
.rodata. When global data of the object is in arena .rodata is there
too and the helper doesn't take the format:
R1 type=scalar expected=fp, pkt, pkt_meta, map_key, map_value, mem,
ringbuf_mem, buf, trusted_ptr_, ctx
String literals are in .rodata.str1.1, which libbpf keeps in a read-only
map. No compiler flag moves a named array there. Put the arrays into
.rodata.str with a section attribute.
An object that uses the macros gets one more map, .rodata.str.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20261002124714.180012-14-alexei.starovoitov@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
All memory of a Rust program is arena, global data included. There are
no address spaces in Rust to say so. LLVM also drops bounds checks of
indexes into constant tables of core that it proved, so the verifier
can't check the access as access to map value.
The object asks for it with a section named .arena.data. What is in
the section doesn't matter:
#[used]
#[link_section = ".arena.data"]
static DATA_IN_ARENA: u8 = 0;
or in C:
char data_in_arena SEC(".arena.data");
There is nothing to tell to libbpf, so bpftool, veristat and other
loaders work with such objects as they are. When the object has the
section:
- .data, .bss and .rodata sections are appended to arena data after
__arena variables, at the alignment of the section. Like __arena
variables the data is at the end of arena.
- relocations of insns against the sections become relocations against
the arena map.
- array maps of the sections are not created.
- when the object has no arena map libbpf creates one that is as large
as the data. bpf_object__find_map_by_name(obj, "arena") finds it and
bpf_map__set_max_entries() makes room for allocations. __arena
variables still need the arena map to be declared.
Read-only sections that stay frozen array maps:
- .data.rel.ro and sections that have relocations in them. The kernel
recognizes pointers to functions in them by value for callx.
- .rodata.str*. They hold const strings for __str arguments of kfuncs.
A copy of them is in arena too.
The program dereferences pointers that it loads from data, so pointers
to data that are stored in data become addresses of arena, wherever the
pointer is. The arena map is created ahead of the other maps then, since
its address goes into their data. A pointer to .rodata.str* points to
the copy. A pointer to a section that is not in arena fails the load:
sec '.data': pointer to 'tbl' at offset 8 can't be resolved:
sec '.data.rel.ro' is not in arena
So does an arena that is pinned or reused. libbpf doesn't mmap it, so
its address is not known.
Programs of the object are loaded with BPF_F_ARENA_SCALAR, since they
access the data through plain numbers.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://lore.kernel.org/bpf/20261002124714.180012-13-alexei.starovoitov@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
bpf_object__relocate_core() allocates a CO-RE candidate cache before
processing relocation records. Two instruction offset validation
failures return -EINVAL directly instead of going through the common
cleanup path.
These checks were moved into bpf_object__relocate_core() when
bpf_core_apply_relo() was split, preserving direct returns that were
safe in the previous helper but now bypass cleanup.
Route both failures through the out path so the candidate cache and
target BTF override are released.
Signed-off-by: Luis Vieira <luisflavieira@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260929-libbpf-core-relo-cleanup-v1-1-c9cd0b5775a9@gmail.com
|
|
BTF_KIND_DECL_TAG is an attribute on a declaration: with kind_flag=0
the tag name is the argument of a btf_decl_tag(), and with kind_flag=1
it encodes a complete __attribute__.
When doing btf_dump, render both forms at every declaration btf_dump
emits - a record, a record member and a typedef:
struct foo { ... } __attribute__((bar));
struct foo { int a __attribute__((bar)); };
typedef struct { ... } __attribute__((bar)) foo_t;
A decl tag is a standalone type pointing at its target. Build an
index of decl tags in btf_dump_resize(). Keep it as a flat array
sorted by (target ID, tag ID). This allows for a stable emission order
in btf_dump_emit_decl_tags().
btf_dump_emit_decl_tags() binary searches for where the target's
entries would begin and walks tags while the target matches. A record
shares its target with its members, so the component_idx is matched
there.
A record attribute goes after the closing brace, where
__attribute__((packed)) already goes. A member attribute goes after
the bit-field width: clang rejects one between the declarator and the
':'. A typedef takes it after the declarator, so a typedef of an
anonymous record can carry two groups at once, one binding to the
record and one to the typedef.
This closes a generic gap in btf_dump, which silently dropped every
decl tag it was given.
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260929234608.48503-3-ihor.solodrai@linux.dev
|
|
btf_dump_mark_referenced() both walks every type added since the last
resize and decides what each one references. Move the walk out to
btf_dump_resize() and hand the function one type at a time, so a second
per-type job can share the same pass instead of adding another one.
No functional change.
Signed-off-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260929234608.48503-2-ihor.solodrai@linux.dev
|
|
kernel-doc fails to parse struct declarations wrapped in
context_lock_struct(), e.g. struct debugfs_cancellation:
include/linux/debugfs.h:245 ... error: Cannot parse struct or union!
Add a struct transform that expands context_lock_struct() to a plain
struct declaration, and apply the struct transforms to the whole
declaration before split_struct_proto(), matching what dump_var() and
dump_function() already do for their prototypes. The member-only
transform in dump_struct() is then redundant.
Add a regression test for the debugfs_cancellation declaration.
Fixes: 6e530e2e3119 ("debugfs: Make debugfs_cancellation a context lock struct")
Signed-off-by: Omokefe Emmanuel Onanaroghene <emmaonana18@gmail.com>
Tested-by: Randy Dunlap <rdunlap@infradead.org>
Reviewed-by: Aleksandr Loktionov <aleksandr.loktionov@intel.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20260925234054.128477-1-emmaonana18@gmail.com>
|
|
Clang puts an extern variable that ends up with a section, an extern
map in .maps or an arena global in .addr_space.<N>, into the BTF
DATASEC of that section, next to the object's own definitions and
with offset 0. Unlike .kconfig and .ksyms, that section is a real
allocated ELF section whenever the same object also defines a
variable in it. The static linker handles such externs incorrectly in
two ways.
The output symbol of an unresolved extern gets the destination
section's index and offset instead of staying SHN_UNDEF with a zero
value. For ephemeral sections both are 0, so this only shows up for
allocated sections, where the extern turns into a size-zero NOTYPE
definition. Linking the output again then fails with "conflicting
non-weak symbol" against a strong definition of that variable, or,
against a weak one, keeps the bogus symbol and resolves every
reference to offset 0 of the section. And if the section of the
object with the extern doesn't start at offset 0 of the output
section, the output symbol fails linker_sanity_check_elf_symtab() on
the next link with "invalid extern symbol" before any of that.
When the extern is resolved by a later object in the same link, the
DATASEC entry appended for the extern keeps its placeholder offset,
as linker_append_btf() only updates the size of an existing entry.
The ELF symbol and the BTF secinfo then disagree and skeleton
generation fails with:
Error: Something is wrong for .addr_space.1's variable #1: need offset 0, already at 8.
Keep unresolved externs SHN_UNDEF with a zero value in the output
symbol table, and once an extern is resolved take the offset of its
DATASEC entry from the ELF symbol, which already accounts for weak
vs. strong resolution.
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260926000243.2830819-1-andrii@kernel.org
|
|
The gzip step of a kernel build is very lengthy, especially for larger
builds such as allmodconfig.
gzip itself cannot be run in parallel, however an alternative tool exists
that can, providing the same feature set as gzip itself - pigz - which
works as a drop-in replacement.
On a 128-core Threadripper, gzip -9 of a 36 MiB x86-64 vmlinux.bin takes
1.6s, and with pigz it takes 0.09s, so the performance increase is
significant.
Therefore introduce the KPGZIP build variable which specifies the
compressor to use for the kernel image and default it to pigz if available,
or KGZIP otherwise (which itself defaults to gzip).
Modules, packaging and the embedded kernel configuration (CONFIG_IKCONFIG)
continue to use KGZIP which defaults to a single-threaded gzip instance.
Module compression is run in parallel by make, so pigz offers no benefit
there, packaging sits outside the build and the embedded configuration is
too small to matter.
pigz is executed using scripts/jobserver-exec, which reserves the job slots
make has free - necessary, as otherwise pigz would just grab unbounded
CPUs.
The kernel image compression step is always performed at the end of the
build so should typically have all job slots available.
pigz always provides byte-for-byte identical output across runs regardless
of thread count, but pigz and gzip are not guaranteed to generate the same
binary output, so reproducible builds must use the same tools consistently.
Also update Documentation/kbuild/kbuild.rst to describe KPGZIP and
Documentation/process/changes.rst to list the optional pigz dependency.
While there, add entries for the other compressor programs.
Finally, update scripts/ver_linux to include the pigz version there.
Every x86 build ends with the compression of vmlinux.bin, 36MB for
defconfig and over 200MB for allmodconfig, no-op builds are unchanged.
Whole build, 128-thread Threadripper 9980X, best of N runs:
before after delta
-------------------------------
x86 defconfig, touch mm/vma.c, gcc 7.7s 6.0s -1.7s (-22%)
x86 defconfig, touch mm/vma.c, clang 6.9s 5.4s -1.5s (-21%)
x86 defconfig, clean, gcc 28.2s 26.3s -1.8s (-6%)
x86 defconfig, clean, clang 28.7s 27.2s -1.5s (-5%)
x86 allmodconfig, touch mm/vma.c, gcc 23.7s 15.4s -8.3s (-35%)
x86 allmodconfig, touch mm/vma.c, clang 22.5s 15.6s -7.0s (-31%)
x86 allmodconfig, clean, gcc 294.1s 278.6s -15.5s (-5%)
x86 allmodconfig, clean, clang 283.9s 266.8s -17.1s (-6%)
Suggested-by: Kees Cook <kees@kernel.org>
Link: https://zlib.net/pigz/
Assisted-by: LLM
Reviewed-by: Manuel Ebner <manuelebnerli@mailbox.org>
Reviewed-by: Kees Cook <kees@kernel.org>
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Link: https://patch.msgid.link/20260923-build-speedup-v4-22-73128809a4a4@kernel.org
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
|
|
Expose the BPF_PROG_STREAM_OPEN command through a low-level libbpf wrapper.
The options structure carries stream open flags, including non-blocking
mode, while leaving room for future extensions.
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://patch.msgid.link/20260925045536.1480933-4-memxor@gmail.com
|
|
Cross-merge BPF and other fixes after downstream PR.
Conflicts:
kernel/bpf/helpers.c
tools/testing/selftests/bpf/prog_tests/cb_refs.c
tools/testing/selftests/bpf/prog_tests/verifier.c
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
|
|
Add support for new kinds to libbpf. BTF_KIND_LOC_PARAM and
BTF_KIND_LOC_PROTO are dedup-able so add support for their
deduplication, whereas since BTF_KIND_LOCSEC contains a unique
offset it is not. LOC_PARAM is considered a primary type
since it contains no external references; LOC_PROTO is a
reference type consisting of LOC_PARAM references so they
are handled in the primary and reference dedup phases
respectively.
For BTF field iteration, BTF_KIND_LOCSEC needs 2 m_offs[] values
for the associated KIND_FUNC and KIND_LOC_PROTO type ids in
each LOCSEC entry.
Add APIs to add location param, location prototypes and location
sections and btf_is_* tests, data accessors for each.
For BTF distillation we add location info to split BTF.
Signed-off-by: Alan Maguire <alan.maguire@oracle.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260924111428.75957-3-alan.maguire@oracle.com
|
|
Light skeleton doesn't create maps. The loader prog does at run time.
Teach gen_loader to create a copy of .rodata map with offsets of
functions for a prog, see create_func_ptr_map().
- Such maps are not maps of the object and are not in the loader ctx.
Reserve slots in fd_array after the maps of the object for them and
close their fds when the loader is done or fails. The progs hold them.
Which progs have callx is not known when the loader is initialized,
so reserve for every prog if .text has callx and for the progs that
have callx otherwise.
- Add bpf_gen__func_ptr_map_create() that emits map create, update,
freeze and returns the index in fd_array for ld_imm64
BPF_PSEUDO_MAP_IDX_VALUE.
- User space can set .rodata before loading the skeleton, see
initial_value in bpf_map_desc. Copy it into the copy of the map as
well and store the offsets of functions on top, since user space
doesn't have them.
- Light skeleton can be generated for a target of another endianness.
Store the offsets in the byte order of the object.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260924031042.1690890-15-alexei.starovoitov@gmail.com
|
|
PIC code keeps constants with pointers, e.g. vtables, in .data.rel.ro
instead of .rodata. Nothing writes there after relocation.
libbpf treats every .data* section as writable, while the kernel looks
for pointers to functions in frozen read-only maps only.
Make .data.rel.ro and .data.rel.ro.* maps read-only and frozen like
.rodata and collect pointers to functions there.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260924031042.1690890-14-alexei.starovoitov@gmail.com
|
|
The kernel recognizes a pointer to a function in frozen read-only array
map by value: byte offset of a static function in the program.
Make it work for tables of functions, struct ops, vtables in .rodata:
static const struct shape_ops square_ops = { 1, area, 10, perimeter };
...
return ops->area(x) * ops->scale;
is compiled into
r3 = *(u64 *)(r1 + 8)
callx r3
where 'area' in .rodata is R_BPF_64_ABS64 relocation against .text.
- Collect relocations in .rodata* that are aligned pointers to
functions. Relocations in data were ignored. The rest still are.
- Append all static functions that the section points to, since any
of them can be called, if the prog refers to the section and has
callx. callx cannot call global functions. Don't pull them in. Such
pointer is NULL.
- Functions have different offsets in different progs, so create a copy
of the map per prog and store the offsets there. The kernel replaces
them with addresses at load and the prog must be the only user of
the map. The map of the section itself is left as-is.
- Progs without callx cannot call them and the kernel doesn't look for
pointers in their data. They keep using the map of the section and
nothing is appended to them.
- The kernel treats any aligned u64 that matches an offset of a static
function as a pointer. Warn if data has one that isn't.
Light skeleton is not supported yet.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260924031042.1690890-13-alexei.starovoitov@gmail.com
|
|
A pointer to a static function in data, e.g.
static int (* const handlers[])(void *ctx) = { foo, bar };
is R_BPF_64_ABS64 relocation against .text section symbol with the offset
of the function stored in place. The linker rejects it:
relocation against STT_SECTION in non-exec section is not supported!
so an object with a table of static functions cannot go through
'bpftool gen object', which all selftests do.
Functions move by the offset of input .text in the output section.
Add it to the pointer. Swap bytes if the object is of foreign endianness,
since data sections are kept in the byte order of the object.
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20260924031042.1690890-12-alexei.starovoitov@gmail.com
|
|
elf_sym_iter_new() calculates the number of symbols by dividing the
symbol table data size by sh_entsize. A malformed ELF file with a zero
sh_entsize causes a division by zero.
Reject symbol table sections with a zero entry size before calculating
the number of symbols.
Signed-off-by: Luis Vieira <luisflavieira@gmail.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260916-libbpf-elf-entsize-fix-v1-1-cf181da33eab@gmail.com
|
|
bpftool rounds memory-mapped data map sizes to the host page size when
generating a light skeleton. The generated code therefore uses a 64K
mapping size when bpftool runs on a 64K-page host, even if the skeleton
runs on a 4K-page target. The target rejects the oversized map mmap(),
causing failure of loading the light skeleton.
When try to run 64K-page selftests on 4K-page VM, the error message does
not provide the reason about page size.
test_atomics:PASS:atomics skeleton open 0 nsec
test_atomics:FAIL:atomics skeleton load unexpected error: -12 (errno 22)
#15 atomics:FAIL
Pass the original map value size and max entries to the generated code and
round mmap size to the runtime page size in the user-space light
skeleton helpers. This keeps generated light skeletons independent of the
build host page size.
Fixes: d510296d331a ("bpftool: Use syscall/loader program in "prog load" and "gen skeleton" command.")
Signed-off-by: Leon Hwang <leon.hwang@linux.dev>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Quentin Monnet <qmo@kernel.org>
Link: https://lore.kernel.org/bpf/20260915161450.96249-1-leon.hwang@linux.dev
|
|
Add btf_module_allowlist and btf_module_allowlist_cnt fields to
bpf_object_open_opts to limit which kernel module BTFs libbpf is
allowed to load.
When the option is not specified, the existing behavior remains
unchanged. An explicitly specified empty allowlist prevents libbpf from
consulting any kernel module BTFs.
The allowlist limits which kernel module BTFs libbpf will consult
wherever module BTF might be needed.
Suggested-by: Andrii Nakryiko <andrii@kernel.org>
Signed-off-by: Fuyu Zhao <zhaofuyu@vivo.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260915124104.77287-2-zhaofuyu@vivo.com
|
|
Add bpf_map__attach_cgroup_opts() to attach a struct_ops map to a cgroup
through a BPF link.
Also extend struct bpf_prog_query_opts with a type_id field so a
BPF_STRUCT_OPS query on a cgroup can select the struct_ops type to
enumerate.
Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
Signed-off-by: Amery Hung <ameryhung@gmail.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://patch.msgid.link/20260917200542.3689605-14-ameryhung@gmail.com
|
|
CO-RE relocation of an ldimm64 instruction operates on two instruction
slots. A malformed BPF ELF can end a function after the first slot and
attach a CO-RE relocation to it. libbpf allocates the instruction array
according to the function symbol size, so the shared relocation code would
then access beyond the allocation.
Reject a terminal ldimm64 in libbpf's relocation loop, where the program
length is available, before resolving or applying the relocation. Both
resolved and unresolved relocations validate the absent second slot, and
unresolved relocation poisoning would additionally write past the array.
The in-kernel caller is protected by the verifier's early instruction-stream
check before it applies CO-RE relocations.
Fixes: eacaaed784e2 ("libbpf: Implement enum value-based CO-RE relocations")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/20260914140852.03DA21F0089B@smtp.kernel.org
Link: https://patch.msgid.link/20260917233222.2542500-11-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
CO-RE relocation records can name any instruction offset. When a
relocation cannot be resolved, bpf_core_patch_insn() currently poisons its
target before checking whether that instruction is a valid relocation
target. Malformed metadata can therefore replace jumps, calls, exits,
register-source arithmetic, or non-immediate loads instead of failing at
the relocation step.
Handle poisoning only after the instruction has passed the same class and
operand-form checks used for a resolved relocation. Route invalid forms
through the existing diagnostic and return a hard error. Keep poisoning
supported instructions, including both halves of a plain ldimm64, so an
unresolved relocation in dead code remains valid.
Extend bpf_core_poison_insn() to poison both halves of ldimm64, and return
its status directly from each validated instruction case. This avoids
routing the success path through a common label and leaves the helper free
to report errors.
The shared relocation code applies this restriction to both libbpf and
in-kernel CO-RE.
Fixes: d7a252708dbc ("libbpf: Improve handling of failed CO-RE relocations")
Reported-by: Nicholas Carlini <npc@anthropic.com>
Suggested-by: Nicholas Carlini <npc@anthropic.com>
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Link: https://patch.msgid.link/20260917233222.2542500-7-memxor@gmail.com
Signed-off-by: Eduard Zingerman <eddyz87@gmail.com>
|
|
A local struct_ops type can contain a bitfield absent from the
corresponding kernel BTF type. bpf_map__init_kern_struct_ops() checks
whether data for an absent local member is zero.
For a bitfield, member->offset contains the bit offset and field width.
bpf_map__init_kern_struct_ops() used the encoded value to calculate the
member data pointer before rejecting the bitfield. The zero-data check
could therefore read outside st_ops->data and crash libbpf.
Reject local bitfields before calculating the member data pointer. Keep
the existing rejection for bitfields in the kernel type.
Fixes: c911fc61a7ce ("libbpf: Skip zeroed or null fields if not found in the kernel type.")
Signed-off-by: Mingpei CAO <caomingpei@gmail.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Reviewed-by: Amery Hung <ameryhung@gmail.com>
Link: https://lore.kernel.org/bpf/20260915162252.473044-2-caomingpei@gmail.com
|
|
bpf_program__set_log_buf() checks the existing prog->log_size instead
of the incoming log_size argument when enforcing the UINT_MAX limit.
As a result, an oversized value can be accepted.
The setter also accepts a non-NULL log_buf with a zero log_size, while
bpf_prog_load() rejects mismatched log buffer and size values.
Validate the log buffer and size pair consistently with bpf_prog_load(),
and check the supplied log_size against UINT_MAX.
Signed-off-by: Luis Vieira <luisflavieira@gmail.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Mykyta Yatsenko <yatsenko@meta.com>
Link: https://lore.kernel.org/bpf/20260915-libbpf-log-buf-fix-v2-1-2feca3fa4842@gmail.com
|
|
When changing BPF program flags, applications often need to just add or
remove a single flag, without touching the existing ones. When using the
bpf_program__set_flags() function, this requires reading the existing
flags and doing bitwise manipulations before resetting the result, which
is slightly awkward to do, and easy to forget.
Add two new convenience helpers, bpf_program__add_flags() and
bpf_program__clear_flags(), which wraps bpf_program__set_flags() in the
bitwise operations required to modify flags without clobbering the
existing ones.
Suggested-by: Andrii Nakryiko <andrii.nakryiko@gmail.com>
Signed-off-by: Toke Høiland-Jørgensen <toke@redhat.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Reviewed-by: Jiayuan Chen <jiayuan.chen@linux.dev>
Acked-by: Eduard Zingerman <eddyz87@gmail.com>
Acked-by: Ihor Solodrai <ihor.solodrai@linux.dev>
Link: https://lore.kernel.org/bpf/20260912084109.432834-1-toke@redhat.com
|
|
bpftool's -d option is documented to enable bpf_trace_printk() messages
from the generated syscall loader when used with -L as specified in
commit d510296d331a ("bpftool: Use syscall/loader program in "prog load"
and "gen skeleton" command.")
However commit b59e4ce8bcaa ("bpftool: Switch bpf_object__load_xattr()
to bpf_object__load()") changed bpftool to use bpf_object__load() which
call the internal bpf_object_load with extra_log_level to 0 instead of
bpf_object__load_xattr with the user request log_level.
All the plumbing was still there to generate the bpf_trace_printk()
instructions from the generator but was now unreachable because
bpf_gen__init() was called with extra_log_level to 0, leaving
gen->log_level at 0.
This uses the obj->log_level field introduced in commit e0e3ea888c69
("libbpf: Allow passing user log setting through bpf_object_open_opts")
set from reading verifier_logs in do_skeleton(). This preserves both
object-level and explicit load-time logging settings.
Fixes: b59e4ce8bcaa ("bpftool: Switch bpf_object__load_xattr() to bpf_object__load()")
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Mahe Tardy <mahe.tardy@gmail.com>
Link: https://lore.kernel.org/r/20260907170536.9996-1-mahe.tardy@gmail.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
|
|
init_arena_map_data() used the build-host page size while it opened an
object. This could reject skeleton generation when the build host and
target use different page sizes.
Validate the size during load, before libbpf calculates the arena data
offset. This uses the running kernel page size and rejects an
undersized map before relocation.
This problem manifested when cross compiling BPF selftests for
arm64 (64K page) on x86 (4K page). In this setup skeleton generation
fails.
Signed-off-by: Mykyta Yatsenko <yatsenko@meta.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Reviewed-by: Emil Tsalapatis <emil@etsalapatis.com>
Link: https://lore.kernel.org/bpf/20260908-libbpf_arena_thing-v1-1-0092c0e0d91b@meta.com
|
|
btf_dedup_identical_types() reads both array descriptors from t1,
skipping comparisons of their referenced types. This can incorrectly
merge distinct structs and corrupt CO-RE relocation metadata.
Read the second descriptor from t2.
Fixes: 62e23f183839 ("libbpf: Improve BTF dedup handling of "identical" BTF types")
Closes: https://lore.kernel.org/bpf/CABzjXVz3iBuump-pXFkefZ5v+2uu4fG61zqRyWD4xmz3PuBycw@mail.gmail.com/
Signed-off-by: Mingpei CAO <caomingpei@gmail.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260908164920.108074-2-caomingpei@gmail.com
|
|
Fix typos in comments, reported by scripts/checkpatch.pl using the
misspelling list in scripts/spelling.txt. Only touches comments, no code
changes.
Assisted-by: Cursor:claude-opus-5
Signed-off-by: Hemanth Selam <hemanth.selam@gmail.com>
Reviewed-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Message-ID: <20260904113257.13594-3-hemanth.selam@gmail.com>
|
|
Cross-merge BPF and other fixes after downstream PR.
Conflicts:
kernel/bpf/backtrack.c
include/linux/bpf_verifier.h
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
|
|
Currently get_debian_kernel_version() expects the version to be in the
form of x.y.z but the Debian kernels uploaded to experimental are not
always LTS kernels. Like the current Debian kernel in experimental
is 7.2~rc7-1~exp1 and uname -v reported "Debian 7.2~rc7-1~exp1".
Signed-off-by: Sudip Mukherjee <sudipm.mukherjee@gmail.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260831134537.1227914-1-sudipm.mukherjee@gmail.com
|
|
The libbpf section definition modifiers for XDP frags support and
sleepable programs stores the flags bits only in the private section
definition cookie from object open to load time. This has the
unfortunate consequence that API consumers cannot see (or manipulate)
the flag between object open and program load.
In particular, libxdp has special handling of frags-enabled programs to
make them compatible with the dispatcher. This doesn't work on XDP
programs that enable frags through the 'xdp.frags' section definition
because the flag is not visible through bpf_program__flags()[0].
Fix this by changing how libbpf loads the program flags from section
definitions: instead of using the private section definition cookie, add
a setup function to the default section definitions that stores the
flags for sleepable and XDP frags programs in the prog_flags field of
struct bpf_program.
Exposing the flags this way means that any use of
bpf_program__set_flags() will override the flags unless the caller takes
care of updating flags in a non-destructive way. This is unavoidable
with the set-only API, and any user setting flags unconditionally is
already broken in the sense that they will also override any other
current and future flags.
In addition, prog_flags survives program type changes through
bpf_program__set_type(). It is the responsibility of the caller to
ensure the flags are cleared if they are incompatible with the new
program type.
[0] https://github.com/xdp-project/xdp-tools/issues/587
Fixes: 082c4bfba4f7 ("libbpf: Add SEC name for xdp frags programs")
Signed-off-by: Toke Høiland-Jørgensen <toke@redhat.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/bpf/20260901-libbpf-frags-flags-v3-3-4eb6f14968b0@redhat.com
|
