| Age | Commit message (Collapse) | Author |
|---|
|
git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next
Pull bpf updates from Alexei Starovoitov:
"Major changes:
- Introduce scalar evolution (SCEV) analysis in the verifier.
It infers how registers and stack slots change in a loop and lets
the verifier check bounded loops with induction variables as ranges
instead of walking every iteration. The number of processed insns
drops ten fold in programs with loops (Eduard Zingerman)
- Add 'callx' instruction: indirect call of a static BPF subprogram.
The address is taken with ld_imm64 or loaded from read-only data.
Supported by x86-64, arm64 and riscv64 JITs (Alexei Starovoitov, Pu
Lehui)
- Add unwind_goto and unwind_exit instructions, so that Rust panic or
C++ exception can run destructors of every frame it passes through.
The normal path is free of unwind code. Supported by x86-64 and
arm64 JITs (Alexei Starovoitov)
- Support programs compiled by rust-bpf: treat load and store through
a number as arena access with BPF_F_ARENA_SCALAR, accept rust names
in BTF (Alexei Starovoitov)
- Raise BPF program stack size from 512 bytes to 2 KiB on x86-64 and
arm64 (Kumar Kartikeya Dwivedi)
- Support struct, union and __int128 up to 16 bytes passed and
returned by value in BPF subprograms and kfuncs, and fix x86-64 and
arm64 trampolines for such arguments (Yonghong Song)
- Introduce a common way to attach struct_ops to a cgroup via
bpf_link. bpf_tcp_ops is the first user and mirrors TCP sock_ops
callbacks (Martin KaFai Lau, Amery Hung)
- Describe inlined functions in BTF. New kinds tell where the inline
sites are and how to fetch the arguments there.
CONFIG_DEBUG_INFO_BTF_INLINE keeps it apart from the main BTF in
.BTF.inline section or in btf_vmlinux_inline.ko (Alan Maguire)
- Fix many memory safety, lifetime and locking bugs in BPF core,
verifier, iterators, local storage, dynptr, devmap, cpumap and
seg6local (Josef Bacik, Alexei Starovoitov)
- Diff stat is dominated by selftests and by removal of
lib/test_bpf.c. Its 16k lines of instruction tests were converted
to BPF selftests that BPF CI runs on every architecture it covers
(Daniel Borkmann)
Other features and fixes:
- Add KASAN checks to memory accesses of JITed programs on x86-64
(Alexis Lothoré)
- Complete unification of helper, kfunc and subprog argument checks
in the verifier (Amery Hung)
- Add bpf_copy_from_user_mm[_str]() kfuncs (Anastasios Papagiannis)
- Various features and fixes (Daniel Borkmann):
- Add .bpf keyring for program signature verification, modelled
after dm-verity keyring, and support ML-DSA in signing tools
- Emit shorter and faster code in x86-64 JIT for patterns that
LLVM generates
- Fix various issues in gotox handling
- Let BPF LSM set xattrs at inode creation with bpf_inode_init_xattr()
(David Windsor, Daniel Borkmann)
- Use sleepable page allocation in sleepable arena paths. Fix races
between arena free, allocation and user page fault (Emil Tsalapatis)
- Switch BPF crypto from lskcipher to AES-CBC and AES-ECB libraries
(Eric Biggers)
- Add bpf_proactive_reclaim() kfunc (Hui Zhu)
- Render decl_tags in libbpf btf_dump and add tests for bpftool C
dump (Ihor Solodrai)
- Speed up tracing_multi attach on x86 (Jiri Olsa)
- Various features and fixes (Kumar Kartikeya Dwivedi):
- Add BPF_PROG_STREAM_OPEN command to read BPF streams through a
file descriptor with poll support
- Let kernel structs publish arena pointers with __arena type tag
- Fix __uninit kfunc output buffers and fastcall rewrite with
indirect stack accesses
- Refuse CO-RE relocations for signed programs
- Add enqueue_rcvq and dequeue_rcvq hooks to bpf_tcp_ops to adjust
sk_rcvlowat and avoid unnecessary wakeups (Kuniyuki Iwashima)
- Add bpf_ksock_sendto() kfunc and allow ksock connect and send from
bpf_wq (Mahe Tardy)
- Fix racy plt target update in arm64 bpf_arch_text_poke()
(Matthew Wood)
- Restructure BPF selftests Makefile as a layered build
(Mykola Lysenko)
- Speed up hash map lookups for u32 and u64 keys (Mykyta Yatsenko)
- Add signed div/mod, movsx, bswap and gotol to MIPS JITs, signed
div/mod to parisc JITs (Nicholas Dudar, Johan Almbladh)
- Fix stack overflow in build ID parser on 32-bit (Nickolai
Zeldovich)
- Add stack arguments, signed arena loads and gotox to riscv64 JIT.
Pu Lehui is now co-maintainer of riscv BPF JITs (Pu Lehui, Chen
Pei, Feng Jiang)
- Add bpf_call_rcu() and bpf_call_rcu_tasks_trace() kfuncs. Support
fetching and/or/xor atomics in arena on x86-64 (Puranjay Mohan)
- Fix flaky selftests, selftests with clang 23 and bpf-gcc
(Qiliang Yuan, KaFai Wan, Vineet Gupta)
- Fix builds broken by fatal resolve_btfids warnings, BTF generation
on ARC, and define arch_cmpxchg_local() on arc, hexagon and
openrisc (Sasha Levin)
- Reduce memory use of hash maps by up to 50% per element
(T.J. Mercier)
- Mark pointer fields of sched_ext ops arguments as trusted (Tejun
Heo)
- Remove legacy bpf_asm, bpf_dbg and bpf_jit_disasm tools (Xia Yan)
- Fix index overflow in queue and stack maps, arena range tree after
allocation failure, tail call loop after freplace detach, lost
signal in bpf_send_signal() (Yuan Chen)
- Many more fixes and cleanups"
* tag 'bpf-next-7.4' of git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next: (660 commits)
selftests/bpf: Time tc_edt from its first packet with data
selftests/bpf: Test unwind_exit of an exception callback
selftests/bpf: Tests for 32-bit scalar evolution bounds computation
bpf: Reject unwind_goto and unwind_exit in offloaded progs
bpf: Support 32-bit linear scalar evolution
bpf: Make unwind_exit of an exception callback exit in JITs
selftests/bpf: Test forbidden bpf_ksock_sendto() LSM attach
selftests/bpf: Test ksock sendto()
bpf: Add bpf_ksock_sendto() kfunc
selftests/bpf: Test unwinding from R0: R2 subprog
selftests/bpf: Add a C++ prog with exceptions
selftests/bpf: Add tests for unwind_goto and unwind_exit
bpf, arm64: Support unwind_goto and unwind_exit
bpf, x86: Support unwind_goto and unwind_exit
bpf: Add unwind_goto and unwind_exit insns
tools/bpf: Remove legacy BPF tools
bpf: Annotate data races on bpf_lru_node type
selftests/bpf: Add arena allocation race tests
bpf: Fix arena race between page free and alloc leading to incoherency
bpf: Revalidate faulted arena pages before installing user mappings
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
Pull probes updates from Masami Hiramatsu:
"Asynchronous optprobe optimizer and cleanups:
- Make optprobe optimizer multi-generational and asynchronous
Rework the optimizer as a state machine to avoid calling
time-consuming synchronize_rcu_tasks() while holding kprobe_mutex,
text_mutex, and cpus_read_lock(). Uses call_rcu_tasks() and a fixed
ring of generation slots instead, so locks are only held during the
brief instruction- patching window
- Use DEFINE_DEBUGFS_ATTRIBUTE for the kprobes enabled knob
Replace a hand-rolled debugfs read/write pair (dating from 2007)
with the proper DEFINE_DEBUGFS_ATTRIBUTE macro, removing dead code
and the stale XXX comment
Code quality and documentation cleanups:
- Add const qualifier to new_argv allocation type in trace_probe.c,
preparing for the type-aware kmalloc_obj allocator conversion
- Clarify btf_find_struct_member() container parameter
Rename the misleading @member_type parameter to @container_type and
add proper kernel-doc documentation for all parameters and return
values
Boot tracing test framework and testcases:
- Add a generic boot tracing test framework under
tools/testing/selftests/ftrace/boottime/
The test runner builds a minimal initramfs with busybox, boots
QEMU, and runs tracefs checker scripts, supporting bootconfig,
kernel command-line options, and multi-boot crash/reboot scenarios
- Add boot-time tracing testcases
Verify dynamic kprobe, synthetic events, eprobe, fprobe, tprobe,
and trace instance creation via bootconfig
- Add kernel cmdline tracing testcases
Verify ftrace, trace_event, trace_buf_size, trace_options,
trace_clock, and trace_instance kernel parameters
- Add persistent ring buffer testcases:
Verify memory reservation via reserve_mem= and trace data retention
across guest crash/reboot, as well as backup instance creation and
boot log preservation
- Change non-unique test symbol from name_show to init_once in
kprobe_non_uniq_symbol.tc for more reliable cross-architecture
testing"
* tag 'probes-v7.4' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace:
ftrace/selftests: Change non-unique test symbol to init_once
tracing/probes: Clarify btf_find_struct_member() container parameter and add kernel-doc
kprobes: Make optprobe optimizer multi-generational and asynchronous
tracing/probes: Add const to new_argv allocation type
kprobes: use DEFINE_DEBUGFS_ATTRIBUTE for the enabled knob
selftests/ftrace: Add persistent ring buffer testcases
selftests/ftrace: Add kernel cmdline tracing testcases
selftests/ftrace: Add boot-time tracing testcases
selftests/ftrace: Add generic boot tracing test framework
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
Pull ring-buffer updates from Steven Rostedt:
- Add Vincent Donnefort as ring-buffer maintainer
Vincent has been doing a lot of updates to the ftrace ring buffer
lately. I'm going to start delegating updates for the ring buffer to
him.
- Use RING_BUFFER_MIN_NR_PAGES instead of open coding the number 2
Instead of using the number 2 as the minimum number of pages that a
ring buffer may have, create and use a macro
RING_BUFFER_MIN_NR_PAGES.
- Clean up the resize_disabled_checks
There's duplication of code to check for resize disabled. Create a
helper function rb_resize_disabled() and use that. Not only does it
put the code in one place, it also removed a redundant check.
- Remove unnecessary cpumask_test_cpu checks
The loop for_each_buffer_cpu(buffer, cpu) only loops over CPUs in the
buffer->cpumask. There's no reason to test if the CPU is set in that
same loop.
- Take cpus_read_lock() when changing the subbuf order
A CPU coming on line for the first time allocates its CPU buffer. But
if this happens while the subbuffer is being resized, it can cause an
inconsistent state of subbufer sizes in the buffer. Add a
cpus_read_lock() to ring_buffer_subbuf_order_set() to prevent this
race.
* tag 'trace-ringbuffer-v7.4' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace:
ring-buffer: Take cpus_read_lock() when changing the subbuf order
ring-buffer: Remove unnecessary cpumask_test_cpu
ring-buffer: Clean up resize_disabled checks
ring-buffer: Unify ring buffer minimum page calculations
ring-buffer: Add a MAINTAINERS category specific for the tracing ring-buffer
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
Pull ftrace updates from Steven Rostedt:
- Have the stacktrace option work with func-args
The func_stack_trace option has it's own ftrace handler and so does
the func-args option. When they are both set, neither would run. The
function tracer itself would error out.
Allow the stacktrace and function args to work together.
- Avoid BTF lookup in atomic context
The ftrace_dump() can call function graph tracing output from an
atomic context (like an NMI watchdog going off). The BTF lookup is
not safe to call from atomic context. Have btf_find_func_proto()
return if in atomic context.
- Build trace_btf.c when CONFIG_DEBUG_INFO_BTF is enabled
New code is depending on trace_btf.c but that only gets built when
BPF_SYSCALL is also enabled. As the new code doesn't depend on that,
have trace_btf.c get built with just DEBUG_INFO_BTF set.
- Have function graph return values have BTF processing
Have the return values printed by the function graph tracer use BTF
to print the values in better format. For example, if it's a char,
short, signed or unsigned, print according to the value defined by
BTF.
- Add a "max" field to the function profiler stats
The function profiler shows all the functions run in the kernel, the
number of times they are called, and the total time they took up.
Also add a "max" field which holds the longest time a function ran
for.
- Fix memory leak in module with all skipped functions
On module load, the ftrace entries are recorded to keep track of what
functions can be traced or not. Some functions may be skipped for
various reasons (some defined by the architecture). If all functions
in the module are skipped, its data page is not freed and can leak.
Catch this case and free the empty page.
- Allow architectures to reject functions from being traced
Allow architectures to skip a function address by returning zero from
ftrace_call_adjust().
- Initialize modname and offset in print_rec()
The variables modname and offset are initialized by
kallsyms_lookup(). If that fails, the values are not set, but the
code continues to print other information. The modname and offset are
used later, but if the kallsyms_lookup() failed, they will not be
set.
If a module is being loaded and unloaded at the same time as reading
the available_filter_functions file, it can cause a fault due to the
lookup failing and modname and offset being used uninitialized.
* tag 'trace-ftrace-v7.4' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace:
ftrace: Initialize modname and offset in print_rec()
ftrace: Let ftrace_call_adjust() reject a patch site
ftrace: Do not leak a module's empty page group
ftrace: Add max field to function profiler stats
tracing: Update funcgraph-retval documentation
fgraph: Enhance funcgraph-retval with BTF-based type-aware output
ftrace: Build trace_btf.c when CONFIG_DEBUG_INFO_BTF is enabled
tracing: Avoid BTF lookup in atomic context
ftrace: Have stacktrace option work with func-args
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
Pull runtime verifier updates from Steven Rostedt:
- Improve lockdep annotation for reactors
Reactors are not supposed to take locks, so they fit the LD_WAIT_FREE
lockdep class. However, reactors may run in preemptible context and
if the preempting code (tick or simply the scheduler) takes locks, we
see a lockdep splat. Switch to LD_WAIT_SPIN to avoid the splat while
documenting that reactors should still not take locks.
- Add KUnit tests for reactors
Add KUnit test cases to validate registration and reaction, this
requires exporting some symbols and other minor fixes.
- Use the right rv_this variable in the wakeup monitor
LTL monitors were updated to use the generic rv_this static variable
but this change wasn't propagated to the newly introduced wakeup
monitor. This broke reactors for the monitor: update the it.
* tag 'trace-rv-v7.4' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace:
rv/rtapp/wakeup: Use "rv_this" the rv_monitor variable
rv: Add KUnit tests for reactor registration and dispatch
rv: Export rv_register_reactor() and rv_unregister_reactor()
rv: Propagate rv_register_reactor() error from reactor init
rv: Use LD_WAIT_SPIN as the reactor lockdep wait type
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux
Pull hardening updates from Kees Cook:
- Provide kmalloc_obj*-style API for devm_kmalloc()
- Improve seq_buf API for saner C String handling with several bug
fixes along the way (Kees Cook, Bill Wendling)
- Overflow helper readability improvements (Bill Wendling)
- Unexport native_write_cr4() (Dennis Chen)
- um: fix CONFIG_GCOV for built-in code (Johannes Berg)
- Improve stackinit test coverage (Justin Stitt)
- KMSAN: fix memset() when using fortify-source, again (Arnd Bergmann)
- lkdtm: use kmalloc() instead of __get_free_page (Mahad Ibrahim)
- signalfd: don't dequeue the forced fatal signals (Oleg Nesterov)
- Fix randstruct to recognize new container_of() behavior
- coccinelle: kmalloc_obj: Convert multi-byte integral allocations too
- Fix a couple remaining allocation return type mismatches
* tag 'hardening-v7.4-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/kees/linux: (29 commits)
overflow: Match arithmetic in both parts of __builtin_choose_expr
lkdtm: Make UNSET_SMEP built-in-only and unexport native_write_cr4()
extcon: realtek: Remove const from type_c_cfg allocation type
coresight: cti: Add const to con_groups allocation type
docs: core-api: Document the seq_buf API
nvdimm: ndtest: Return the string length from flags_show()
powerpc/papr_scm: Return the string length from the sysfs show functions
seq_buf: Add seq_buf_init_append()
bpf: Remove dead newline stripping from format_disasm_line()
seq_buf: Add seq_buf_terminate()
seq_buf: Add seq_buf_strlen()
seq_buf: Clear what a writer did not claim when a seq_buf overflows
seq_buf: Copy what fits when seq_buf_puts() and seq_buf_putmem() overflow
scsi: lpfc: Print rx monitor records straight into the seq_buf
seq_buf: Do not pop from an overflowed seq_buf
seq_buf: Do not print an empty line from an overflowed seq_buf_do_printk()
.get_maintainer.ignore: Add Kees Cook
devres: Provide kmalloc_obj*-style API for devm_kmalloc
mm: Move default_gfp() into gfp_types.h
kunit/stackinit: Clean up XFAIL under instrumentation
...
|
|
copy_user_syms() and bpf_uprobe_multi_link_attach() check whole user
arrays with access_ok() up front, then read them one element at a time
with __get_user(), with other work in between: allocations and
strncpy_from_user() in copy_user_syms(), validation in the uprobe_multi
loop. Each of those accesses enables and disables user access on its
own, so the up-front check saves nothing. On x86, __get_user() is even
more expensive than get_user(): it does an lfence, where get_user()
checks the address with a couple of plain instructions.
Use get_user(). The access_ok() checks stay, because they also make a
bad array fail with -EFAULT before anything else is looked up or
allocated. Update the comment that said they were there for
__get_user().
This is part of getting rid of __get_user() and __put_user() entirely.
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
|
|
print_rec() leaves 'modname' and 'offset' uninitialized and
kallsyms_lookup() only sets them on success. When the lookup fails like
when a module is being loaded or unloaded, 'modname' holds a garbage
value, passes the 'if (modname)' check, and is dereferenced by
seq_printf().
Oops: general protection fault
RIP: 0010:string+0x48/0xe0
Call Trace:
vsnprintf+0x1f5/0x650
seq_printf+0x8e/0xb0
print_rec+0x6c/0xc0
t_show+0x4c/0x440
seq_read_iter+0x119/0x480
This can be triggered by reading available_filter_functions while a
module is being loaded or unloaded.
Initialize both variables so a failed lookup skips the module name.
Cc: stable@vger.kernel.org
Fixes: b39181f7c690 ("ftrace: Add FTRACE_MCOUNT_MAX_OFFSET to avoid adding weak function")
Link: https://patch.msgid.link/20261016230403.4916-1-aarnahmd@amazon.com
Tested-by: Gavin Inglis <giinglis@amazon.com>
Signed-off-by: Aaron Ahmed <aarnahmd@amazon.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
When ftrace_call_adjust() returns 0 for a patch site,
ftrace_process_locs() does not skip the site. The NULL test runs before
the adjustment since commit 6eeca746fa5f ("ftrace: Test mcount_loc
addr before calling ftrace_call_addr()"), so the 0 is stored as the
record's ip and ftrace_init_nop() later hits the BUG_ON() in arm64's
patch_map(). Test the adjusted address too and count the site as
skipped.
Link: https://patch.msgid.link/20260924-b4-arm64-callops-kcfi-v2-2-587865b6d991@linux.dev
Reviewed-by: Ben Cressey <ben@cressey.dev>
Reviewed-by: Florent Revest (Anthropic) <florent.revest@linux.dev>
Assisted-by: LLM
Signed-off-by: Jose Fernandez (Anthropic) <jose.fernandez@linux.dev>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
When every mcount_loc entry of a module is skipped,
ftrace_process_locs() leaves the module's first page group linked
with no records. ftrace_release_mod() matches a module's groups by
records[0].ip, which is 0 here, so the group is never freed. While
it stays linked, ftrace_free_mem() reads pg->records[pg->index - 1]
with pg->index == 0 on every later module load, as lookup_rec() did
before commit ee92fa443358f ("ftrace: Fix invalid address access in
lookup_rec() when index is 0").
Unlink and free the new page groups when none of them got a record.
Link: https://patch.msgid.link/20260924-b4-arm64-callops-kcfi-v2-1-587865b6d991@linux.dev
Reported-by: sashiko-bot@kernel.org
Closes: https://lore.kernel.org/all/20260904013350.5141B1F000E9@smtp.kernel.org/
Assisted-by: LLM
Signed-off-by: Jose Fernandez (Anthropic) <jose.fernandez@linux.dev>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
For a perf event on a kprobe or uprobe event which was made in tracefs,
bpf_get_kprobe_info() and bpf_get_uprobe_info() look the probe up by
name in the list of dynamic events, and hand a pointer to its symbol or
file name back to the caller. None of the callers holds event_mutex.
The perf event keeps its trace event alive, nothing else. Other dynamic
events are removed from the list and freed during the walk. And a probe
which shares its event with other probes can be deleted while the event
is in use, so even the probe which is found can be freed, together with
the string, before the caller has measured and copied it. The result is
a use-after-free in the walk, in strlen() and in the copy to user space,
or a NULL pointer dereference.
The readers are BPF_OBJ_GET_INFO_BY_FD and fdinfo of a perf link, for
whoever holds a descriptor of the link, and BPF_TASK_FD_QUERY. The other
side is anybody who may write to dynamic_events, kprobe_events or
uprobe_events.
Take event_mutex in bpf_get_perf_event_info() and give the caller a copy
of the name, which it has to free.
Fixes: 41bdc4b40ed6 ("bpf: introduce bpf subcommand BPF_TASK_FD_QUERY")
Signed-off-by: Josef Bacik <josef@toxicpanda.com>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261017163745.3341787-10-alexei.starovoitov@gmail.com
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace
Pull tracing fixes from Steven Rostedt:
- Fix NULL dereference when copying histogram field variable
If event_hist_trigger_parse() creates an action that uses one of
pseudo fields for its keys, the hist_data field element will have a
NULL for its field member. When looking to match that key, the lookup
uses key_field->field->name which will dereference a NULL pointer.
Instead use the helper function that handles such cases:
expr_field_str().
- Take the trace_types_lock when dropping the buffer_meta reference
The tracing_buffer_meta_open() error return drops the trace_array
reference with __trace_array_put() instead of trace_array_put() where
the former doesn't take the lock and the latter does.
- Fix NULL pointer dereference when reading per_cpu/cpuN/buffer_meta
These files are created for every possible CPU. But it only contains
data for CPUs that have come online. If one is read for a CPU that
hasn't come online yet, it triggers a NULL pointer dereference.
Have reading of CPUs that have not come on line yet return -ENODEV.
- Fix an out-of-bounds shift in the log2 histogram modifier
Histograms can take a log2 modifier to have the buckets defined by
log2 increments. If a shift is greater than ULONG_MAX or above 2^63,
the key falls into a lower order bucket.
Use order_base_2() instead of "ilog2(n - 1) + 1" to have all
overflowed keys go into the top bucket.
- Fix the persistent ring buffer rb_meta_subbuf_idx() accounting
The algorithm for finding the index of a sub-buffer in a persistent
ring buffer buffer did not take into account the meta data at the
beginning of the buffer causing the wrong index to be returned.
As rb_subbufs_from_meta() does the proper accounting to find the
first sub-buffer, use that function instead of repeating the logic
(incorrectly) and make the calculations based off of that.
- Fix the symbol name checks for the remote buffer code
The remote ring buffer has a build time check to make sure its not
using any code that will not be in the hypervisor. For various
architectures, there has been false positive reports from this
checker.
Fix them.
- Fix false warning in ring_buffer_map_get_reader()
There is a WARN_ON() after rb_get_reader_page() if no reader is
returned. But there are legitimate cases where this can happen.
Remove the WARN_ON() and also a check that covers one of the
legitimate cases that tries to exit out early. There's no need for
the check if it just simply allows the return of NULL from
rb_get_reader_page().
- Fix the ignored return value from resize_buffer_duplicate_size()
Some clean up of the code in tracing_snapshot_write() removed
checking the return value of resize_buffer_duplicate_size() and if
that failed it would not be detected.
- Fix the histogram timestamps for buffered events
The histogram timestamps are taken from the events that are recorded.
It currently reads ring_buffer_event_time_stamp() but if the event is
buffered it does not read it properly. Use tracing_event_time_stamp()
which handles both buffered and normal events.
- Do not initialize tracefs if tracing is disabled at boot
If tracing fails to allocate or some other error is detected,
tracing_disabled is set. Do not initialize tracefs in this case as it
may expect data to be initialized but is not.
- Free eventfs inode after an RCU grace period
Currently eventfs_inodes are freed after a SRCU grace period as the
iterations in the code use SRCU. There are locations that the inode
may still be accessed from the VFS that uses RCU which does not have
the same synchronization as SRCU. Add an RCU wait before free eventfs
inodes.
- Do not fail persistent and remote buffers under memory pressure
As persistent and remote buffers have their own pre-allocated memory,
there's no reason to check how much memory is available before
created them. By checking the memory pressure, these buffers may fail
to be created even though their memory is already present.
- Account for subbuffer size when checking how much memory is available
Before allocating a ring buffer, the size is checked against the
current memory pressure of the system. But it assumes every
sub-buffer is of page size. This is no longer the case. Use the
actual sub-buffer size when calculating how much memory will be used.
- Reject non-stack fields for synthetic stacktrace fields
Stack traces are special fields and the checks currently do not do
enough to make sure that the fields being processed as stack traces
are indeed stack traces. Update the logic to properly detect what is
or is not a stack trace.
- Fix a crash in print_graph_function_event()
When a tracer does not define static or default flags the
tr->current_trace_flags is set to NULL. If the ring buffer is not
cleared when switching tracers, the previous tracers events could be
read. The function_graph entries will dereference the
tr->current_trace_flags and cause a crash.
Set the tr->current_trace_flags to a empty set of flags to prevent it
from being NULL.
- Separate tracing filter predicate count from allocation
The amount allocated for the predicate program is the number of
predicates defined plus two for adding "true" and "false" at the end.
The nr_preds is incremented by two to accommodate the booleans but is
then used as the limit check for number of predicates. This is
incorrect as it is now two higher than what is expected.
Use a separate variable for the size of the predicate array.
- Fix the filter operator stack check
The operator stack increments the pointer and then writes the content
onto the stack. The check for overflow compared the current pointer
to see if it was greater than the stack size before doing so. This is
an off by 2 error as the write will be at the size - 1 and the
compare is done before the increment. Adjust the check accordingly.
* tag 'trace-v7.3-rc7' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace:
tracing: Check operator stack capacity before pushing
tracing: Keep the predicate limit separate from program capacity
tracing: Fix crash in print_graph_function_event()
tracing/histograms: Reject non-stacktrace fields for synthetic stacktrace fields
ring-buffer: Use subbuf_order in available memory estimation
ring-buffer: Skip si_mem_available() for persistent and remote buffers
eventfs: Free the eventfs_inode after an rcu grace period too
tracing: Do not initialize tracefs work if tracing is disabled
tracing: Fix hist trigger timestamps for buffered events
tracing: Fix ignored return value of resize_buffer_duplicate_size()
ring-buffer: Fix false warning in ring_buffer_map_get_reader()
tracing: Fix simple_ring_buffer undef check on sparc64
tracing: Fix simple_ring_buffer undef check on parisc
tracing: Fix empty symbol names in simple_ring_buffer undef check
ring-buffer: Fix the sub-buffer array base in rb_meta_subbuf_idx()
tracing: Fix the shift out of bounds in the log2 histogram modifier
ring-buffer: Do not show buffer_meta for CPUs without a buffer
tracing: Take trace_types_lock when dropping the buffer_meta reference
tracing: Fix NULL dereference when copying keys for a field variable
|
|
irq_work_is_busy() cannot see the per-CPU send_signal_work while
it is being filled: the check only matches after irq_work_queue()
has claimed the work. A concurrent caller therefore passes it, both
callers race for the same irq_work, and the loser's signal is
silently lost along with its task reference while the queued work
runs with a mix of both callers' fields.
kprobe, tracepoint and perf_event programs exclude each other on
the same CPU through bpf_prog_active, so for two callers to meet,
one of the programs has to be a raw tracepoint or an fentry one.
And since commit 87c544108b61 ("bpf: Send signals asynchronously
if !preemptible") this path runs with IRQs enabled too, so a hard
IRQ can interrupt the fill as well, not only an NMI. Found by code
inspection and reproduced with a perf_event NMI program racing an
fentry one.
Claim the work through work->task itself: cmpxchg() it from NULL
to the target task before touching the other fields, and set it
back to NULL once the callback has consumed them. A context
finding the work claimed returns the documented -EBUSY.
Fixes: 1bc7896e9ef4 ("bpf: Fix deadlock with rq_lock in bpf_send_signal()")
Signed-off-by: Yuan Chen <chenyuan@kylinos.cn>
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261009022427.968304-1-chenyuan@kylinos.cn
|
|
The function profiler (trace_stat/function<cpu>) reports Hit, Time,
Avg and s^2 per function. Add a "max" field recording the maximum
single-call duration, which is valuable for real-time tuning and for
investigating occasional latency spikes.
Like Time/Avg, it honors the graph-time and sleep-time options, so it
reflects entry-to-return wall-clock latency.
Link: https://patch.msgid.link/20260924030108.2673654-1-yun.zhou@windriver.com
Signed-off-by: Yun Zhou <yun.zhou@windriver.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
A CPU coming online for the first time gets its per CPU buffer allocated
by trace_rb_cpu_prepare(), which then adds it to buffer->cpumask. If that
happens while ring_buffer_subbuf_order_set() is running, the CPU shows up
after the allocation loop and the install loop walks it with an empty
new_pages list, corrupting the buffer.
Take cpus_read_lock() as ring_buffer_resize() does.
Cc: stable@vger.kernel.org
Fixes: f9b94daa542a ("ring-buffer: Set new size of the ring buffer sub page")
Link: https://patch.msgid.link/20260916192628.274136-1-devnexen@gmail.com
Signed-off-by: David Carlier <devnexen@gmail.com>
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
for_each_buffer_cpu() already iterates over buffer->cpumask. Remove the
unnecessary cpumask_test_cpu() found in ring_buffer_subbuf_order_set().
Link: https://patch.msgid.link/20260916152813.105082-3-vdonnefort@google.com
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
ring_buffer_subbuf_order_set() checks for resize_disabled twice under
the same buffer->mutex hold. Moreover, this check duplicates the logic
in ring_buffer_resize(). Create a common helper rb_resize_disabled() to
factor out this code.
Link: https://patch.msgid.link/20260916152813.105082-2-vdonnefort@google.com
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
The current funcgraph-retval implementation suffers from two accuracy
issues:
1. Void-returning functions still print a return value, creating
misleading noise in the trace output.
2. For functions returning narrower types (e.g., char, short), the
displayed value can be incorrect because high bits of the register
may contain undefined data.
This patch addresses both problems by leveraging BTF to obtain the exact
return type of each traced kernel function. The key changes are:
1. Void function filtering: Functions with void return type no longer
display any return value in the trace output, eliminating unnecessary
clutter.
2. Type-aware value formatting: The return value is now properly truncated
to match the actual width of the return type before being displayed.
Additionally, the value is formatted according to its type for better
human readability.
Here is an output comparison:
Before:
# perf ftrace -G vfs_read --graph-opts retval
...
1) | touch_atime() {
1) | atime_needs_update() {
1) 0.069 us | make_vfsuid(); /* ret=0x0 */
1) 0.067 us | make_vfsgid(); /* ret=0x0 */
1) | current_time() {
1) 0.197 us | ktime_get_coarse_real_ts64_mg(); /* ret=0x187f886aec3ed6f5 */
1) 0.352 us | } /* current_time ret=0x69380753 */
1) 0.792 us | } /* atime_needs_update ret=0x0 */
1) 0.937 us | } /* touch_atime ret=0x0 */
After:
# perf ftrace -G vfs_read --graph-opts retval
...
2) | touch_atime() {
2) | atime_needs_update() {
2) 0.070 us | make_vfsuid(); /* ret=0x0 */
2) 0.070 us | make_vfsgid(); /* ret=0x0 */
2) | current_time() {
2) 0.162 us | ktime_get_coarse_real_ts64_mg();
2) 0.312 us | } /* current_time ret=0x69380649(trunc) */
2) 0.753 us | } /* atime_needs_update ret=false */
2) 0.899 us | } /* touch_atime */
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
Link: https://patch.msgid.link/20260916133218.1282590-4-dolinux.peng@gmail.com
Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
The trace_btf.c file provides BTF helper functions used by the ftrace
subsystem. Make the compilation of trace_btf.c solely depend on
CONFIG_DEBUG_INFO_BTF, allowing features like funcgraph-retval to also
utilize these helpers.
Additionally, the redundant dependency on CONFIG_PROBE_EVENTS_BTF_ARGS
is removed, as CONFIG_DEBUG_INFO_BTF already depends on
CONFIG_BPF_SYSCALL.
Cc: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Xiaoqin Zhang <zhangxiaoqin@xiaomi.com>
Link: https://patch.msgid.link/20260916133218.1282590-3-dolinux.peng@gmail.com
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
The function graph trace formatter can be called from ftrace_dump_one()
with interrupts disabled. When ftrace_dump_on_oops is enabled, the trace
panic notifier can reach ftrace_dump() from the panic path, including the
nmi_panic() path used by the hard lockup detector.
btf_find_func_proto() calls bpf_find_btf_id(), which may initialize vmlinux
BTF under a mutex. Skip the BTF lookup when running in an atomic context
or with interrupts disabled, and fall back to the existing return value
formatting.
Link: https://patch.msgid.link/20260916133218.1282590-2-dolinux.peng@gmail.com
Fixes: ebeed8d4a555 ("tracing/probes: Move finding func-proto API and getting func-param API to trace_btf")
Signed-off-by: pengdonglin <pengdonglin@xiaomi.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
The codebase scatters hardcoded minimum page limits across multiple
places. Introduce RING_BUFFER_MIN_NR_PAGES and rb_calculate_nr_pages()
to formalize the requirement of two pages in a ring buffer.
Link: https://patch.msgid.link/20260911193937.602202-4-vdonnefort@google.com
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
When both the func_stack_trace option is enabled first, the func-args
option can not be enabled. Same if the func-args option is enabled first,
the func_stack_trace option can not be enabled.
Allow the two to work together. The stack trace is already slow and
testing the current tracer flags should be in the noise of overhead.
Link: https://patch.msgid.link/20260825191036.11b19036@gandalf.local.home
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
predicate_parse() uses the first operator stack entry before processing
any opening parentheses. A push then increments top before writing.
The existing check still permits that write when top already points to
the last allocated entry, which can access beyond the operator stack.
Reject the push when top reaches the last allocated entry.
Cc: stable@vger.kernel.org
Fixes: 39f7c41c908b ("tracing/filters: Enable filtering a cpumask field by another cpumask")
Link: https://patch.msgid.link/atELO9L0I9kg5CpC@Taeyangs-MacBook-Pro.local
Signed-off-by: Taeyang Lee <0wn@theori.io>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
predicate_parse() adds two to nr_preds to reserve the terminal TRUE
and FALSE entries. It also uses that increased value as the limit for
parsed predicates. If the initial count is smaller than the number
actually parsed, the program can fill the allocation and subsequent
accesses can go past its end.
Keep nr_preds as the predicate limit and use a separate value for the
allocation size. This leaves space for both terminal entries and a
zeroed entry when parsing stops at the limit.
Cc: stable@vger.kernel.org
Fixes: 39f7c41c908b ("tracing/filters: Enable filtering a cpumask field by another cpumask")
Link: https://patch.msgid.link/atELCd1XjKVhrkFL@Taeyangs-MacBook-Pro.local
Signed-off-by: Taeyang Lee <0wn@theori.io>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
tracing_set_tracer() assigns the flags of the newly selected tracer
with:
tr->current_trace_flags = t->flags ? : t->tracer->flags;
If the new tracer defines neither static flags nor default flags, like
hwlat, irqsoff or wakeup, add_tracer() has no per instance flags copy
for it, and both operands are NULL, so tr->current_trace_flags ends up
being NULL as well.
The ring buffer is not cleared when switching tracers: it is reset
inside tracer_init(), after current_trace_flags was already assigned.
If the buffer still holds function_graph entries of the previous
tracer, printing one of them dereferences the NULL pointer. This was
hit by cat trace during an ftrace stress test:
BUG: kernel NULL pointer dereference, address: 0000000000000000
#PF: supervisor read access in kernel mode
RIP: 0010:print_graph_function_event+0xe/0x20
Call Trace:
s_show+0x4d/0x110
seq_read_iter+0x2d4/0x490
seq_read+0x130/0x170
vfs_read+0xb5/0x370
ksys_read+0x6d/0xe0
do_syscall_64+0xbc/0x450
The snapshot buffer is not reset at all, so this can be reproduced
without a race:
echo function_graph > /sys/kernel/tracing/current_tracer
echo 1 > /sys/kernel/tracing/snapshot
echo hwlat > /sys/kernel/tracing/current_tracer
cat /sys/kernel/tracing/snapshot
Rather than teaching every user of tr->current_trace_flags about this
corner case, make sure it is never NULL: when the newly selected
tracer has no flags at all, tracing_set_tracer() falls back to a
static empty no_tracer_flags. run_tracer_selftest() can hit the same
case, so the same fallback is added there too. The existing users
already handle opts == NULL, so these tracers keep behaving as before.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20261015130658.221198-1-hao.ge@linux.dev
Fixes: e29aa918a928 ("tracing: Have function graph tracer define options per instance")
Suggested-by: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Hao Ge <hao.ge@linux.dev>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
In check_synth_field(), when a synthetic event field is a stacktrace
(field->is_stack) and the histogram field is not a "long[" stacktrace
array, validation falls through to the generic size and signedness
comparison.
Because parse_synth_field() sets a stacktrace field's type to
"__data_loc unsigned long[]", its size is sizeof(u64) and
synth_field_signed() marks it as signed. Consequently, any 8-byte
signed scalar histogram field passes the fallback check, and its scalar
value is subsequently dereferenced as a kernel stacktrace pointer in
get_field_size() and trace_stack().
Furthermore, checking strstr(hist_field->type, "long[") is insufficient
because tracepoint fields defined with __bitmask() or __rel_bitmask()
also have type "__data_loc unsigned long[]" or "__rel_loc unsigned
long[]", whilst create_hist_field() treats them as 4-byte scalar values
without HIST_FIELD_FL_STACKTRACE.
Propagate HIST_FIELD_FL_STACKTRACE to variable references and field
variables (whilst excluding variable aliases from per-element
stacktrace capture), and reject any mismatch between
HIST_FIELD_FL_STACKTRACE and field->is_stack in check_synth_field().
Cc: stable@vger.kernel.org
Fixes: 00cf3d672a9d ("tracing: Allow synthetic events to pass around stacktraces")
Link: https://patch.msgid.link/20261013112449.1006088-1-lee@kernel.org
Signed-off-by: Lee Jones <lee@kernel.org>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
To avoid failing in the middle of an allocation of a ring-buffer,
__rb_allocate_pages() looks at the available memory estimation with
si_mem_available(). Use the subbuffer order with this estimation to fail
early if necessary for ring-buffers with an order > 0.
Link: https://patch.msgid.link/20261013102047.1488347-3-vdonnefort@google.com
Fixes: f9b94daa542a ("ring-buffer: Set new size of the ring buffer sub page")
Reviewed-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
traceprobe_expand_meta_args expands probe arguments based on BTF info.
Return an error if we have arguments that need expanding but BTF is not
supported.
The params variable is now unused, it can be removed.
Link: https://lore.kernel.org/all/20261014083838.31020-1-martin@kaiser.cx/
Fixes: d157d7694460 ("tracing/probes: Support BTF field access from $retval")
Signed-off-by: Martin Kaiser <martin@kaiser.cx>
Reviewed-by: Steven Rostedt <rostedt@goodmis.org>
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
Persistent and remote buffer memory is pre-allocated by the time
__rb_allocate_pages() is called. Skip si_mem_available() to avoid
wrongly failing the allocation under memory pressure.
For the same reason, unset user_thread.
Link: https://patch.msgid.link/20261013102047.1488347-2-vdonnefort@google.com
Fixes: be68d63a139b ("ring-buffer: Add ring_buffer_alloc_range()")
Fixes: 2e67fabd8b77 ("ring-buffer: Introduce ring-buffer remotes")
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
If trace buffer allocation fails during early boot (e.g. in
early_trace_init() -> allocate_trace_buffers()), tracing_disabled is
left set to 1 and the global trace array events list remains
uninitialized.
Later during boot, tracer_init_tracefs() (invoked via fs_initcall)
queues tracerfs_init_work asynchronously without checking if tracing was
disabled. When tracer_init_tracefs_work_func() runs, it calls
event_trace_init() and init_tracer_tracefs(&global_trace, NULL)
unconditionally. This leads to __find_event_file() dereferencing an
uninitialized tr->events list, causing an early boot NULL pointer
dereference Oops:
BUG: Kernel NULL pointer dereference on read at 0x00000010
Faulting instruction address: 0xc000000000485be0
Oops: Kernel access of bad area, sig: 7 [#1]
...
NIP [c000000000485be0] __find_event_file+0x70/0x3c0
LR [c000000000448874] init_tracer_tracefs+0x274/0xc80
Call Trace:
init_tracer_tracefs+0x274/0xc80
tracer_init_tracefs_work_func+0x50/0x320
process_one_work+0x1e8/0x5c0
worker_thread+0x1dc/0x3d0
kthread+0x194/0x1b0
start_kernel_thread+0x14/0x18
Check tracing_disabled at the beginning of tracer_init_tracefs_work_func()
and return immediately if tracing is disabled or failed to initialize,
preventing the crash.
Cc: stable@vger.kernel.org
Fixes: 6621a7004684 ("tracing: make tracer_init_tracefs initcall asynchronous")
Link: https://patch.msgid.link/20261009163557.88467-2-amachhiw@linux.ibm.com
Reported-by: Michal Suchánek <msuchanek@suse.de>
Closes: https://lore.kernel.org/all/arYskzbiaNzBR9MD@kunlun.suse.cz/
Signed-off-by: Amit Machhiwal <amachhiw@linux.ibm.com>
Tested-by: Michal Suchánek <msuchanek@suse.de>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Syzkaller triggered a WARN_ON_ONCE(!nest) warning:
WARNING: kernel/trace/ring_buffer.c:821 at ring_buffer_event_time_stamp
Call trace:
ring_buffer_event_time_stamp
hist_field_timestamp
hist_fn_call
event_hist_trigger
event_triggers_call
__event_trigger_test_discard
trace_event_buffer_commit
do_trace_event_raw_event_sched_switch
trace_event_raw_event_sched_switch
The warning is probabilistically reproducible and is related to syzkaller
executing the following commands:
echo 'prev_pid == 999999' > events/sched/sched_switch/filter
echo 'hist:keys=common_timestamp' > events/sched/sched_switch/trigger
The specific triggering process is as follows:
CPU 0 CPU 1
(context switching) (echo 'hist:keys=common_timestamp' > \
events/sched/sched_switch/trigger)
do_trace_event_raw_event_##call
trace_event_buffer_lock_reserve
if (!tr->no_filter_buffering_ref &&
trace_file->flags & EVENT_FILE_FL_FILTERED)
return entry; // return directly
event_hist_trigger_parse
hist_register_trigger
tracing_set_filter_buffering(file->tr, true);
tr->no_filter_buffering_ref++;
__trace_buffer_lock_reserve
rb_start_commit
local_inc(&cpu_buffer->committing); // skipped and not executed
trace_event_buffer_commit
__event_trigger_test_discard
event_triggers_call
event_hist_trigger
hist_fn_call
hist_field_timestamp
ring_buffer_event_time_stamp
nest = local_read(&cpu_buffer->committing);
WARN_ON_ONCE(!nest) // trigger warning
The root cause is that when an event file has a filter attached and the
hist trigger has not yet been attached, trace_event_buffer_lock_reserve()
writes the event into the per-CPU trace_buffered_event temp buffer instead
of reserving it in the ring buffer, so cpu_buffer->committing is not
incremented. When the event is later processed at commit time by a hist
trigger, hist_field_timestamp() -> ring_buffer_event_time_stamp() sees a
zero committing count, triggering WARN_ON_ONCE(!nest).
tracing_event_time_stamp() was added by commit d8279bfc5e959 ("tracing:
Add tracing_event_time_stamp() API") for exactly this case: if the event
is the per-CPU trace_buffered_event, it returns the current ring buffer
timestamp. But it never gained a caller. Use it in hist_field_timestamp(),
so that a hist trigger that races onto a buffered event records the
current time, which is only marginally later than when the event was
recorded, instead of warning and recording a bogus timestamp.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260929104015.1689921-1-wutengda@huaweicloud.com
Fixes: b94bc80df648 ("tracing: Use a no_filter_buffering_ref to stop using the filter buffer")
Signed-off-by: Tengda Wu <wutengda@huaweicloud.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
When tr->allocated_snapshot is true, resize_buffer_duplicate_size() is
called to update the snapshot buffer size. However, its return value was
not checked. In case of a failed resize this could lead next snapshot
being incomplete.
Add resize_buffer_duplicate_size() result check and return error if
resize fails.
Compile tested only.
Found by Linux Verification Center (linuxtesting.org) with SVACE.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260928152634.4354-1-d.kandybka@gmail.com
Fixes: 180e4e390978 ("tracing: Add snapshot refcount")
Signed-off-by: Dmitry Kandybka <d.kandybka@gmail.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
The mmap reader can warn when it catches up with a writer that is still
committing events. rb_get_reader_page() returns NULL in this case, but
ring_buffer_map_get_reader() treats that result as an error.
The initial rb_per_cpu_empty() check filters out an empty buffer, but it
does not guarantee that the next reader page has committed data. Writers
do not take reader_lock and can advance commit_page before publishing the
committed length on the new page. rb_get_reader_page() can swap to that
page, catch up with the writer and return NULL.
Handle NULL through the existing no-data exit without warning. Remove the
caller's reader_page == commit_page check as well, since the reader-page
helper already handles that case. The existing metadata update and return
path remain in place.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260924084946.20104-1-krystianmkaniewski@gmail.com
Fixes: 117c39200d9d ("ring-buffer: Introducing ring-buffer mapping functions")
Assisted-by: Gemini:gemini-3.8-flash syzbot
Reported-by: syzbot+de3d7f9bcc9212f3fae1@syzkaller.appspotmail.com
Closes: https://syzkaller.appspot.com/bug?extid=de3d7f9bcc9212f3fae1
Link: https://syzkaller.appspot.com/ai_job?id=488adc18-a10e-42d2-a205-25327c38837f
Signed-off-by: Krystian Kaniewski <krystianmkaniewski@gmail.com>
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
simple_ring_buffer relies on headers that end up including
<asm/thread_info.h>. On sparc64, this emits the undefined symbol
current_thread_info_reg. Allow this symbol.
Link: https://patch.msgid.link/20260923074213.3077182-4-vdonnefort@google.com
Fixes: 1211907ac0b5 ("tracing: Generate undef symbols allowlist for simple_ring_buffer")
Reported-by: Guenter Roeck <linux@roeck-us.net>
Closes: https://lore.kernel.org/all/5574c9e6-9ed2-4ee6-a2b2-fb6ee81e83a1@roeck-us.net/
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Tested-by: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
simple_ring_buffer uses local_set() and cmpxchg() on unsigned long. Add
them to undefsyms_base.c to extend the allowlist for the parisc
architecture.
Link: https://patch.msgid.link/20260923074213.3077182-3-vdonnefort@google.com
Fixes: 1211907ac0b5 ("tracing: Generate undef symbols allowlist for simple_ring_buffer")
Reported-by: Guenter Roeck <linux@roeck-us.net>
Closes: https://lore.kernel.org/all/5574c9e6-9ed2-4ee6-a2b2-fb6ee81e83a1@roeck-us.net/
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Tested-by: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Sparc64 emits empty symmbol names. Skips them when establishing the
allowlist and when extracing undefined symbols.
Link: https://patch.msgid.link/20260923074213.3077182-2-vdonnefort@google.com
Fixes: 1211907ac0b5 ("tracing: Generate undef symbols allowlist for simple_ring_buffer")
Reported-by: Guenter Roeck <linux@roeck-us.net>
Closes: https://lore.kernel.org/all/5574c9e6-9ed2-4ee6-a2b2-fb6ee81e83a1@roeck-us.net/
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Tested-by: Guenter Roeck <linux@roeck-us.net>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
On a persistent ring buffer, rb_meta_subbuf_idx() can return an index one
sub-buffer too high. rb_setup_ids_meta_page() then fills subbuf_ids[]
shifted by one and leaves subbuf_ids[0] NULL, which __rb_map_vma()
dereferences when userspace maps trace_pipe_raw. rb_update_meta_reader()
writes two such indices into meta->buffers[], which rb_cpu_meta_valid()
would reject on the next boot, discarding the previous boot's trace.
A CPU's sub-buffer array starts after its meta header and the nr_subbufs
integers that follow, aligned up to a sub-buffer. rb_meta_subbuf_idx()
inverts that calculation but skips only the integers, not the header, so
its base is one sub-buffer low whenever the omitted header size crosses
the alignment boundary.
Invert the calculation with rb_subbufs_from_meta() rather than repeating
it. rb_range_buffer() maps an index back to an address through the same
helper, so the two directions can no longer disagree.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260918071507.1019251-1-donggeunyoo.kernel@gmail.com
Fixes: b14d032973d4 ("ring-buffer: Add ring_buffer_meta data")
Signed-off-by: Donggeun Yoo <donggeunyoo.kernel@gmail.com>
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
The .log2 key modifier files a value under its base-2 order:
# echo 'hist:keys=bytes_req.log2' > events/kmem/kmalloc/trigger
hist_field_log2() computes that as ilog2(roundup_pow_of_two(val)). The
out-of-line form of roundup_pow_of_two() is
return 1UL << fls_long(n - 1);
which shifts by 64 on a 64-bit kernel both when n is 0, where n - 1
wraps to ULONG_MAX, and when n is above 2^63. Both are ordinary keys,
because the field fetch hands every value over as a u64: a syscall
argument of 0 reaches the first, and one above 2^63 reaches the second.
Keying sys_enter_lseek on offset.log2 and passing 0, 1, 2, 3, 1000,
2^63+1 and U64_MAX gives
{ offset: ~ 2^1 } hitcount: 1
{ offset: ~ 2^10 } hitcount: 1
{ offset: ~ 2^2 } hitcount: 1
{ offset: ~ 2^0 } hitcount: 4
Keys above 2^63 land in the lowest bucket. With CONFIG_UBSAN_SHIFT=y
the shift is also reported:
UBSAN: shift-out-of-bounds in include/linux/log2.h:57:13
shift exponent 64 is too large for 64-bit type 'long unsigned int'
__ubsan_handle_shift_out_of_bounds.cold+0xdd/0x1cb
hist_fn_call.cold+0x8b/0xd9
event_hist_trigger+0x1cc/0x790
ftrace_syscall_enter+0x197/0x360
do_syscall_64+0x402/0x4b0
Use order_base_2(), which is ilog2(n - 1) + 1 for n > 1 and 0 below
that. It returns what the old expression returned on every input the
old one was defined for. Where it was not, 0 stays in bucket 0, which
is where x86_64 had been putting it, and the keys above 2^63 move into
bucket 64.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260916002735.808520-1-donggeunyoo.kernel@gmail.com
Fixes: 4b94f5b7b4a5 ("tracing: Add hist trigger 'log2' modifier")
Signed-off-by: Donggeun Yoo <donggeunyoo.kernel@gmail.com>
Assisted-by: Claude:claude-fable-5
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
The per_cpu/cpuN/buffer_meta files are created for every possible CPU,
but buffer->buffers[cpu] is only allocated once that CPU comes online.
Reading the file of a CPU that never came online dereferences a NULL
cpu_buffer in rbm_start().
Fail the open with -ENODEV when the CPU has no buffer, as the other
per CPU ring buffer interfaces do.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260914133506.4865-3-devnexen@gmail.com
Fixes: 950032ffcee7 ("ring-buffer: Add output of ring buffer meta page")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260913134906.6E8A01F000FF@smtp.kernel.org/
Signed-off-by: David Carlier <devnexen@gmail.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
tracing_buffer_meta_open() drops the trace_array reference with
__trace_array_put() when ring_buffer_meta_seq_init() fails, without
holding trace_types_lock. Use trace_array_put() like every other open
error path.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260914133506.4865-2-devnexen@gmail.com
Fixes: 950032ffcee7 ("ring-buffer: Add output of ring buffer meta page")
Signed-off-by: David Carlier <devnexen@gmail.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
In event_hist_trigger_parse() where it needs to create actions like
"onmatch", it calls:
event_hist_trigger_parse() {
create_actions() {
action_create() {
trace_action_create() {
trace_action_create_field_var() {
create_field_var_hist()
Where create_field_var_hist() does a loop on the hist_data
representing the keys. The issue is, if the keys uses one of the
pseudo field types (like common_cpu), the hist_data field element
will have NULL for its field member causing a NULL pointer
dereference when accessing the key_field->field->name.
# echo 'hist:keys=common_cpu:ts0=common_timestamp.usecs' > \
events/sched/sched_waking/trigger
# echo 'my_synth u64 lat; int prio' > synthetic_events
# echo 'hist:keys=common_cpu:wakeup_lat=common_timestamp.usecs-$ts0:\
onmatch(sched.sched_waking).my_synth($wakeup_lat,prio)' > \
events/sched/sched_switch/trigger
KASAN: null-ptr-deref in range [0x0000000000000010-0x0000000000000017]
RIP: 0010:create_field_var_hist+0x771/0x1380
Call Trace:
trace_action_create_field_var+0x360/0x600
trace_action_create+0x393/0xe80
event_hist_trigger_parse+0x3e47/0x69e0
trigger_process_regex+0x1a6/0x250
event_trigger_write+0xce/0x160
Instead of accessing it directly, use the proper handler
expr_field_str() to get the name.
The three previous patches are prerequisites: without them a .buckets
key and a common_stacktrace key each render into a command that cannot
be parsed back.
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20260914053443.981201-5-donggeunyoo.kernel@gmail.com
Fixes: 02205a6752f2 ("tracing: Add support for 'field variables'")
Signed-off-by: Donggeun Yoo <donggeunyoo.kernel@gmail.com>
Assisted-by: Claude:claude-fable-5
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
kernel-doc
btf_find_struct_member() returns the containing structure or union type
(which can be an inner anonymous struct/union rather than the outer
passed type) via its last parameter. However, the parameter was named
@member_type, which easily confuses readers and static analyzers into
thinking it returns the member field's own datatype.
Rename @member_type to @container_type in the prototype, definition,
and callers to make the relationship explicit. Also convert the comment
above btf_find_struct_member() to standard kernel-doc format, documenting
all parameters and return values.
Link: https://lore.kernel.org/all/179030954365.77418.17193777972248047975.stgit@devnote2/
Assisted-by: LLM
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
check_dup_ids() rejects equal BTF ids, but different ids can resolve to
the same address. Aliases:
ffffffff8161b6b0 t __do_sys_munlockall
ffffffff8161b6b0 T __ia32_sys_munlockall
ffffffff8161b6b0 T __x64_sys_munlockall
or several BTF_KIND_FUNC with the same name that vmlinux BTF has when
CONFIG_DEBUG_INFO_BTF_INLINE is set.
Each id gets its own trampoline and register_fentry_multi() adds them to
data->reg with the same ip. update_ftrace_direct_add() checks that new
entries are not in direct_functions, but doesn't compare them with each
other, so the attach succeeds. At detach update_ftrace_direct_del() sees
that 'direct' of one of the entries doesn't match and returns -EINVAL:
WARNING: kernel/bpf/trampoline.c:1845 at bpf_trampoline_multi_detach+0x24c/0x280
Call Trace:
bpf_link_free+0x51/0x110
bpf_link_release+0x18/0x20
__fput+0x103/0x2d0
__x64_sys_close+0x3d/0x80
The images are freed anyway, while ftrace still calls them:
Oops: int3: 0000 [#1] SMP NOPTI
RIP: 0010:0xffffffffc042b201
Call Trace:
? __audit_syscall_exit+0x9/0x100
? do_syscall_64+0x5ab/0x5e0
entry_SYSCALL_64_after_hwframe+0x77/0x7f
Kernel panic - not syncing: Fatal exception in interrupt
Return -EINVAL from register_fentry_multi() when ip is already in
data->reg. When such ids are attached via different links the second
attach already fails with -EINVAL in update_ftrace_direct_add().
Move the declaration of ftrace_lookup_ip() to include/linux/ftrace.h.
The lookup adds 60 msec to the attach of 64480 functions in
tracing_multi_bench_attach: 1.34 -> 1.40 sec.
Fixes: aef4dfa790b2 ("bpf: Add bpf_trampoline_multi_attach/detach functions")a
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Leon Hwang <leon.hwang@linux.dev>
Acked-by: Jiri Olsa <jolsa@kernel.org>
Link: https://lore.kernel.org/bpf/20261011164836.2233874-2-alexei.starovoitov@gmail.com
Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
|
|
Seven callers call seq_buf_str() only to NUL-terminate the buffer,
discarding the returned pointer. Two of them need a comment to say so.
Add seq_buf_terminate(), wrapping the __seq_buf_terminate() helper that
seq_buf_str() and seq_buf_strlen() already use, and convert those
callers. It returns void: returning the offset would just be
seq_buf_strlen() under another name. A zero-sized seq_buf is left
untouched, as in the other accessors.
Add tests for the three cases: room for the NUL after the data, an
overflowed buffer where it lands in the last byte, and a zero-sized
buffer that must not be written to.
Build tested ARCH=x86_64 defconfig with GCC 16.2.0, plus
CONFIG_HIST_TRIGGERS=y and CONFIG_BPF_SYSCALL=y to reach the converted
call sites in kernel/trace/trace_events_hist.c and
kernel/bpf/diagnostics.c. Tests run 24/24 passing on ARCH=um.
Assisted-by: LLM
Acked-by: Alexei Starovoitov <ast@kernel.org>
Link: https://patch.msgid.link/20261005155708.1471260-7-kees@kernel.org
Signed-off-by: Kees Cook <kees@kernel.org>
|
|
In preparation for converting the kmalloc family of allocators to the
type-aware kmalloc_obj family, we need to make sure that the returned
type from the allocation matches the type of the variable being
assigned. (The kmalloc family returns "void *", which can be implicitly
cast to any pointer type.)
The assigned type is "const char **", but the converted allocation type
would be "char **", which is the same type without the const qualifier.
As there is no general way to safely add const qualifiers, take the size
from the assignment target instead. No change in allocation size
results.
Build tested ARCH=x86_64 allmodconfig with GCC 16.2.0:
kernel/trace/trace_probe.o
Link: https://lore.kernel.org/all/20260917211022.i.677-kees@kernel.org/
Assisted-by: LLM coccinelle
Signed-off-by: Kees Cook <kees+treewide@kernel.org>
Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
|
|
Cross-merge BPF and other fixes after downstream PR.
Conflicts:
arch/arm64/net/bpf_jit_comp.c
arch/x86/net/bpf_jit_comp.c
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
|
|
Commit d9e4c61a12dd ("rv: Use generic rv_this for the rv_monitor
variable in LTL") renamed the rv_monitor variable to rv_this. But
rtapp/wakeup monitor was late to the party and missed that.
So for this monitor, rv_this is an empty struct. The reactor fails to
work, because rv_react() checks monitor->react and sees NULL.
Correct to "rv_this" and make the reactor work again.
Signed-off-by: Nam Cao <namcao@linutronix.de>
Fixes: d9e4c61a12dd ("rv: Use generic rv_this for the rv_monitor variable in LTL")
Link: https://lore.kernel.org/r/20261001044218.3508990-1-namcao@linutronix.de
Signed-off-by: Gabriele Monaco <gmonaco@redhat.com>
|
|
Add KUnit tests covering the reactor register/unregister lifecycle
(including duplicate and name-length rejection) and rv_react() dispatch
(a no-op without a callback, exactly one invocation with one). The
mdelay() callback keeps the CPU busy so a timer interrupt lands inside
rv_react()'s lockdep context, exercising the LD_WAIT_SPIN wait type
from the previous patch; a spurious lockdep splat there would show up
in the test output.
The dispatch tests rely on reacting_on being enabled, since rv_react()
returns early when it is off.
Reviewed-by: Gabriele Monaco <gmonaco@redhat.com>
Signed-off-by: Wen Yang <wen.yang@linux.dev>
Link: https://lore.kernel.org/r/7d213b110815da45595ad8cd7bcce5641fac3404.1789412799.git.wen.yang@linux.dev
Signed-off-by: Gabriele Monaco <gmonaco@redhat.com>
|
|
rv_react() is exported to modules, but the reactor registration helpers
are not. Export them with EXPORT_SYMBOL_GPL() so reactor modules and
the tristate KUnit test module can register and unregister reactors
without hitting undefined symbol errors at link time(modpost).
Commit 3d3800b4f7f4 ("rv: Remove reactor's reference counter") noted
that module-based reactors, if supported, should be pinned with
try_module_get()/module_put() to prevent unloading under an active
monitor. This patch does not add that support. The only in-tree module
consumer, the tristate KUnit reactor test module, registers and
unregisters its reactor within a controlled test lifecycle.
The actual pinning is left to a follow-up series that adds support for
loadable-module reactors.
Reviewed-by: Gabriele Monaco <gmonaco@redhat.com>
Signed-off-by: Wen Yang <wen.yang@linux.dev>
Link: https://lore.kernel.org/r/94b7ec120a92b9368541ac050601e66be8fe653a.1789412799.git.wen.yang@linux.dev
Signed-off-by: Gabriele Monaco <gmonaco@redhat.com>
|
|
Both register_react_printk() and register_react_panic() ignore the
return value of rv_register_reactor() and always return 0. If the
registration fails (e.g. a duplicate reactor name), the init functions
silently report success even though the reactor was not registered.
Propagate the error from rv_register_reactor() so a failed registration
is reported instead of being silently ignored.
Reviewed-by: Gabriele Monaco <gmonaco@redhat.com>
Reviewed-by: Nam Cao <namcao@linutronix.de>
Signed-off-by: Wen Yang <wen.yang@linux.dev>
Link: https://lore.kernel.org/r/1684c66520cee66ba86224f2b9e473c7dd2416ee.1789412799.git.wen.yang@linux.dev
Signed-off-by: Gabriele Monaco <gmonaco@redhat.com>
|
