summaryrefslogtreecommitdiff
path: root/kernel/trace/trace_events.c
AgeCommit message (Collapse)Author
2026-09-11tracing: Restore :mod: trailer after parsing in ftrace_set_clr_event()Thomas Weißschuh
While ftrace_set_clr_event() modifies its input buffer during parsing, before returning to the caller the buffer is supposed to be restored to its original state. This works correctly for the colon between the subsystem and event but not the colon at the beginning of :mod:. Restore the colon, so the :mod: trailer is not stripped after ftrace_set_clr_event(). Cc: stable@vger.kernel.org Fixes: 4c86bc531e60 ("tracing: Add :mod: command to enabled module events") Link: https://patch.msgid.link/20260908-tracing-cli-event-filter-v2-1-05396a3fb663@linutronix.de Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-09-01tracing: Have show_event_filters/triggers files take trace array refSteven Rostedt
The newly added files show_event_filters and show_event_triggers that show all filters or triggers that are set within the trace array do not take a reference for the trace array it is showing. Without taking a reference, the trace_array may be freed via "rmdir" while a task is reading one of theses files. Those files iterate all the events within an instance (trace_array) and nothing prevents that instance from being freed while its data is being read. This causes a use-after-free crash. Have the open of both those files take the trace_array reference via the trace_array_get() that prevents the trace_array from being freed while the files are opened. Cc: stable@vger.kernel.org Link: https://patch.msgid.link/20260828094153.17b95037@gandalf.local.home Fixes: 729757b96a662 ("tracing: Add show_event_filters to expose active event filters") Fixes: 6a80838814eea ("tracing: Add show_event_triggers to expose active event triggers") Reported-by: Farhad Alemi <farhad.alemi@berkeley.edu> Closes: https://lore.kernel.org/all/CA+0ovCjerKZJLwXScM9bF2ga2rLi4_XOpUfK41NDbENpeu98jA@mail.gmail.com/ Reviewed-by: Aaron Tomlin <atomlin@atomlin.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-08-22tracing: Fix crash passing ERR_PTR to kthread_stop()Hui Su
event_test_stuff() calls kthread_run() and unconditionally passes the returned task_struct pointer to kthread_stop(). kthread_run() returns an error pointer such as ERR_PTR(-ENOMEM) when kthread creation fails, for example under memory pressure during the boot-time event self-test. kthread_stop() then dereferences the invalid pointer, crashing the kernel. Check the result of kthread_run() before passing it to kthread_stop(). Use WARN_ON() so that a failure to create the self-test thread does not go unnoticed, matching the ring-buffer self-test fix in commit 91542863abad ("ring-buffer: Fix crash passing ERR_PTR to kthread_stop()"). Cc: stable@vger.kernel.org Fixes: e6187007d6c3 ("tracing/events: add startup tests for events") Link: https://patch.msgid.link/20260817120642.668375-3-sh_def@163.com Signed-off-by: Hui Su <sh_def@163.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-08-19Merge tag 'tracefs-v7.3' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace Pull tracefs updates from Steven Rostedt: - Define event fields before directory creation Move the event_define_fields() call in event_create_dir() before the eventfs directory creation. Previously, a failure after directory creation wouldn't clean up eventfs_inode because the error path didn't call eventfs_remove_dir(). This eliminates the need to clean up the eventfs directories if event_define_fields() fails. - Add warning for out of bounds pos in __eventfs_iterate() Sashiko complains about the ctx->pos causing issues if it is less than 2 or greater than MAX_INT in __eventfs_iterate(). The thing is, the logic prevents that from happening. But to make Sashiko happy, add a WARN_ON() and exit safely if the function ever does get input that is out of the range the function expects. * tag 'tracefs-v7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace: eventfs: Add warning for out of bounds pos in __eventfs_iterate() eventfs: Define event fields before directory creation
2026-08-19Merge tag 'trace-v7.3' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace Pull tracing updates from Steven Rostedt: - Expose btf_ids to trace events In order to allow BPF programs to attach to system call trace events (which are actually pseudo trace events built on top of raw_syscall events), expose the BTF ID of the events. This will allow BPF programs better precision in attaching to events. - Use "u64" to assign to hist_field->type Instead of using kstrdup("u64", GFP_KERNEL) to assign the hist_field->type, just point it to "u64" instead. The hist_field->type is freed via kfree_const(). - Replace kmalloc()/strcpy() with kstrdup() for trace_printk Instead of having two calls to copy the module format string, just use kstrdup(). - Use __free() in trace event histograms and triggres where possible - Use seq_buf in trace event code instead of strcat() Instead of calculating the size of the buffer to use and filling it with strcat(), use the seq_buf infrastructure that takes care of making sure not to overflow the string size. - Reject invalid preemptirq_delay_test CPU affinity The preempt_delay_test module can take an invalid CPU affinity mask and create confusing output. Simply have the module reject invalid affinity masks. - Prevent division by zero in ftrace_ops sample module code If the ftrace_ops sample module code receives the module parameter nr_function_calls set to zero, it can cause a division by zero error. - Warn when an event dereferences a parameter in TP_printk() On boot up and module load, the trace event TP_printk() is scanned for possible bugs. As the TP_printk() code is executed when the user reads the "trace" file and processes the data written when the trace_event executed, the data it reads can be literally days old. The scan currently checks for dereferencing printk formats like "%pI6". But it does not check if the parameters themselves have a dereference like: TP_printk("offset %08x: value %08x", (u32)(__entry->addr - __entry->edma->membase), __entry->value) __entry represents the pointer to the event on the ring buffer. The __entry->edma->membase is dereferencing a pointer on the ring buffer to find membase, but the __entry->edma may no longer be a valid pointer. Warn on this case too. - Replace some strcpy() with strscpy() - Clean up mmiotrace events to use assign_type() macro The assign_type() macro makes sure the event type is indeed the type that is being parsed. The mmiotrace trace was written before that macro was created so it just simply typecasted the pointer. Replace the typecasting with the macro. - Have the ENUM processing to numbers only process what is added The code that converts ENUMs to their numbers in the trace events scanned all events to do the processing. This was true when a module was loaded too. That is, instead of processing just the events for the module, it processed *all* events. Even the builtin ones that were processed at boot up. Add a check for the event->module matching mod if it is a module before processing it. * tag 'trace-v7.3' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace: (21 commits) tracing: Have trace_event_update_all() only handle module that is loading tracing: Cleanup event_enable_trigger_parse() by using __free() tracing: Report every TP_printk double dereference tracing/mmiotrace: Use trace_assign_type() in mmio_print_mark() tracing: Make per-template BTF id lists file-local tracing: Use seq_buf for string concatenation tracing: Use strscpy() instead of strcpy() in trace_sched_switch tracing: Warn when an event dereferences a pointer in TP_printk() samples/ftrace: Prevent division by zero when nr_function_calls is zero tracing: Reject invalid preemptirq_delay_test CPU affinity fgraph: Use trace_seq_putc() in print_graph_return() tracing/user_events: Replace a seq_printf() call by seq_puts() in user_seq_show() tracing/user_events: Use seq_putc() in two functions tracing: Bound histogram expression strings with seq_buf tracing: Return ERR_PTR() from expr_str() tracing: Use __free() for expr_str() buffer kernel/trace/trace_printk: Use kstrdup() instead of kmalloc() and strcpy() tracing: Point constant hist field type to string literal selftests/bpf: Add test for tracepoint btf_ids tracefs file tracing: Expose tracepoint BTF ids via tracefs ...
2026-08-14tracing: Have trace_event_update_all() only handle module that is loadingSteven Rostedt
The function trace_event_update_all() does a scan of events looking to replace enums with their values in the strings that get exported to the event format files. It's run at boot up on all events and again when a module loads. The issue is that when a module loads, it still runs on *all* events. There's no reason to process every event when a module loads as the previous events have already been processed. Only execute on the events that are loaded with the module. Link: https://patch.msgid.link/20260813204226.29563591@gandalf.local.home Fixes: 3673b8e4ce723 ("tracing: Allow for modules to convert their enums to values") Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-08-13tracing: Fix race between update_event_fields and, event_define_fieldsMichael Wu
The following sequence may leads race between event_define_fields() and update_event_fields(): CPU0 (loads module A) CPU1 (loads module B) =============================== =============================== load_module(A) load_module(B) notifier_call_chain notifier_call_chain trace_module_notify trace_module_notify mutex_lock(&event_mutex) trace_event_update_all() trace_module_add_events(A) down_write(&trace_event_sem) __register_event(call_A) __add_event_to_tracers(call_A) event_define_fields(call_A) for each f: list_for_each_entry(field, list_add(&f->link, &class->fields, link) &class->fields) field = class->fields->next; Where access to the class->fields is not protected by the event_mutex in trace_event_update_all(). This produces the following panic: Unable to handle kernel access ... at virtual address 0000000000000018 pc : update_event_fields+0xf8/0x368 Call trace: update_event_fields+0xf8/0x368 trace_event_update_all+0x7c/0x2b4 trace_module_notify+0x4c/0x1dc notifier_call_chain+0x84/0x168 blocking_notifier_call_chain_robust+0x64/0xd4 load_module+0x10c8/0x123c __arm64_sys_finit_module+0x230/0x31c Fix by taking event_mutex in trace_event_update_all() before trace_event_sem. Cc: stable@vger.kernel.org Fixes: b3bc8547d3be ("tracing: Have TRACE_DEFINE_ENUM affect trace event types as well") Link: https://patch.msgid.link/2e5730d2-c631-da41-3a3a-ae35bb4895f3@allwinnertech.com Signed-off-by: Michael Wu <michael@allwinnertech.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-08-13tracing: Fix NULL pointer dereference in module event cache removalHui Su
A module-only event filter such as ":mod:foo" is cached with a NULL event_mod->match when foo has not been loaded. If a later write tries to remove a specific match from the same module, remove_cache_mod() passes the NULL cached match to strcmp(), causing a NULL pointer dereference. The issue can be reproduced from userspace: echo ':mod:trace_events_kunit_missing' > /sys/kernel/tracing/set_event echo '!foo_bar:mod:trace_events_kunit_missing' >> /sys/kernel/tracing/set_event The second write must be a concatenation (">>") to not include O_TRUNC as that would cause ftrace_clear_events() to clear the cached modules lines. The crash was reproduced on x86_64 QEMU while KUnit workers contended on the event tracing path: BUG: kernel NULL pointer dereference, address: 0000000000000000 #PF: supervisor read access in kernel mode RIP: 0010:strcmp+0x10/0x30 Call Trace: __ftrace_set_clr_event_nolock+0x373/0x4a0 ftrace_set_clr_event+0xf0/0x180 ftrace_event_write+0xdf/0x110 vfs_write+0xf6/0x440 ksys_write+0x68/0xe0 do_syscall_64+0xf9/0x540 entry_SYSCALL_64_after_hwframe+0x77/0x7f Check event_mod->match before comparing it, consistent with the existing NULL checks for the cached system and event fields. The mismatched removal continues to return -EINVAL; a broad cached module filter is removed with "!:mod:<module>". Cc: stable@vger.kernel.org Link: https://patch.msgid.link/20260811173902.1927376-2-sh_def@163.com Fixes: b355247df104 ("tracing: Cache \":mod:\" events for modules not loaded yet") Reported-by: syzbot+4d3143c8e28f6266c636@syzkaller.appspotmail.com Closes: https://lore.kernel.org/lkml/6a7a6b7f.9c11d2ce.289b96.00f8.GAE@google.com/ Signed-off-by: Hui Su <sh_def@163.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-08-11tracing: Report every TP_printk double dereferenceDavid Carlier
WARN_ONCE() splats once per call site, so only the first offending event registered is ever reported. The tree currently has six: ice_{rx,tx}_dim_template, two hfi1 txq events, mtu3_ep and edma_log_io. Whichever registers first hides the rest, and each has to be found again on the next boot. Add a pr_warn() next to the WARN_ONCE() so every offender is listed, the same way test_event_printk() already pairs WARN_ON_ONCE() with pr_warn() for unsafe %p* dereferences. The WARN_ONCE() stays so the condition still fails tests and panics under panic_on_warn. Link: https://patch.msgid.link/20260806215256.1680267-1-devnexen@gmail.com Suggested-by: Steven Rostedt <rostedt@goodmis.org> Signed-off-by: David Carlier <devnexen@gmail.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-08-08eventfs: Define event fields before directory creationAnubhav Shelat
Move the event_define_fields() call in event_create_dir() before the eventfs directory creation. Previously, a failure after directory creation wouldn't clean up eventfs_inode because the error path didn't call eventfs_remove_dir(). This eliminates the need to clean up the eventfs directories if event_define_fields() fails. Link: https://patch.msgid.link/20260715135231.338535-3-ashelat@redhat.com Signed-off-by: Anubhav Shelat <ashelat@redhat.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-07-29tracing: Check return value of __register_event() in trace_module_add_events()Masami Hiramatsu (Google)
trace_module_add_events() ignores the return value of __register_event() and unconditionally calls __add_event_to_tracers() for each event. If __register_event() fails (for example, if event_init() fails), the trace_event_call is not added to ftrace_events list, but __add_event_to_tracers() still creates a trace_event_file pointing to it. If module loading subsequently fails and module memory is freed, tracing state retains a stale trace_event_call pointer in trace_event_file, leading to a use-after-free when tracefs or tracing subsystem operations are later executed. Fix this by checking the return value of __register_event() and only calling __add_event_to_tracers() if event registration succeeded. Fixes: ae63b31e4d0e ("tracing: Separate out trace events from global variables") Cc: stable@vger.kernel.org Link: https://patch.msgid.link/178528487878.124250.14170824576025743236.stgit@devnote2 Assisted-by: Antigravity:gemini-3.5-flash Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-07-28tracing: Use seq_buf for string concatenationWoradorn Laodhanadhaworn
In preparation for removing the strlcat API[1], replace the string concatenation logic with a struct seq_buf, which tracks the current position and the remaining space internally. Use seq_buf_str() to NUL-terminate before passing to early_enable_events(). Link: https://github.com/KSPP/linux/issues/370 [1] Link: https://patch.msgid.link/20260713045249.69942-1-woradorn.laon@gmail.com Signed-off-by: Woradorn Laodhanadhaworn <woradorn.laon@gmail.com> [ Moved placement of #include <linux/seq_buf.h> ] Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-07-28tracing: Warn when an event dereferences a pointer in TP_printk()Steven Rostedt
Currently on boot up and when modules are loaded, the trace event infrastructure will examine the TP_printk's of every event looking to see if it dereferences pointers on the ring buffer via printk formats like "%pB" and such. What it doesn't do is check if the arguments themselves do a dereference from a pointer. This was brought with a fix[1] to the fsl_edma event that had in the arguments of the TP_printk(): "__entry->edma->membase" The __entry->edma is a pointer saved in the ring buffer. The dereference from TP_printk() happens when the user reads the "trace" file which can be seconds, minutes, hours, days, weeks, or even months later! There is no guarantee that the __entry->edma pointer will still be pointing to what it was when it was recorded, and could crash the kernel when a user reads the event. Add logic to the test_event_printk() that also checks for this case and warn if the event dereferences a pointer from the ring buffer. [1] https://lore.kernel.org/all/20260630200022.1826420-1-martin@kaiser.cx/ Link: https://patch.msgid.link/20260630184836.74d477b6@gandalf.local.home Signed-off-by: Steven Rostedt <rostedt@goodmis.org> Reviewed-by: Martin Kaiser <martin@kaiser.cx> Reviewed-by: Vinod Koul <vkoul@kernel.org> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-07-28tracing: Expose tracepoint BTF ids via tracefsMykyta Yatsenko
Add events/<sys>/<event>/btf_ids, a per-template file that exposes the BTF ids resolve_btfids fills in for each tracepoint: btf_obj_id BTF object owning the ids below raw_btf_id FUNC_PROTO of __bpf_trace_<call> (named args), consumed by raw_tp / tp_btf BPF programs tp_btf_id trace_event_raw_<call> ring-buffer record, consumed by classic BPF_PROG_TYPE_TRACEPOINT programs DECLARE_EVENT_CLASS now emits a 2-entry BTF_ID_LIST (FUNC __bpf_trace_* and STRUCT trace_event_raw_*) and stores the pointer in trace_event_class. Per-syscall events under syscalls/ share the handcrafted classes event_class_syscall_{enter,exit} instead of going through DECLARE_EVENT_CLASS. Wire those classes to the BTF id lists generated for sys_enter / sys_exit so all ~700 per-syscall events expose the shared dispatcher prototype and record. The per-syscall events do not own their own tracepoint (they share sys_enter/sys_exit), so raw_btf_id is reported as 0 on those events; the meaningful raw_btf_id is exposed on raw_syscalls/sys_{enter,exit}/btf_ids where raw_tp / tp_btf programs can actually attach. Link: https://patch.msgid.link/20260518-generic_tracepoint-v2-2-b755a5cf67bb@meta.com Signed-off-by: Mykyta Yatsenko <yatsenko@meta.com> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-07-24tracing: Fix union collision of module and refcnt for dynamic eventsMasami Hiramatsu (Google)
In 'struct trace_event_call', the 'module' pointer and the 'refcnt' atomic variable share the same memory space in a union. For dynamic events, the union member is 'refcnt', which acts as an active reference counter. When a dynamic event (such as kprobe, uprobe, fprobe, eprobe, or wprobe) has a non-zero reference count (e.g. due to active event triggers or perf attachments), its 'call->module' evaluates to a small non-zero integer instead of NULL. When filtering or setting events for a specific module (e.g., writing ':mod:<module>' to 'set_event'), the code in '__ftrace_set_clr_event_nolock()' and 'update_event_fields()' reads 'call->module' directly without checking whether the event is dynamic. This causes the kernel to treat the small integer (refcnt) as a 'struct module' pointer, leading to a NULL/invalid pointer dereference (Oops) when dereferencing the module name. Fix this by ensuring that the 'TRACE_EVENT_FL_DYNAMIC' flag is checked before treating 'call->module' as a valid pointer in these code paths. Cc: stable@vger.kernel.org Link: https://patch.msgid.link/178425670947.84440.11344393611899824907.stgit@devnote2 Fixes: 4c86bc531e60 ("tracing: Add :mod: command to enabled module events") Assisted-by: Antigravity:gemini-3.5-flash Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org> Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
2026-04-17Merge tag 'trace-v7.1' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace Pull tracing updates from Steven Rostedt: - Fix printf format warning for bprintf sunrpc uses a trace_printk() that triggers a printf warning during the compile. Move the __printf() attribute around for when debugging is not enabled the warning will go away - Remove redundant check for EVENT_FILE_FL_FREED in event_filter_write() The FREED flag is checked in the call to event_file_file() and then checked again right afterward, which is unneeded - Clean up event_file_file() and event_file_data() helpers These helper functions played a different role in the past, but now with eventfs, the READ_ONCE() isn't needed. Simplify the code a bit and also add a warning to event_file_data() if the file or its data is not present - Remove updating file->private_data in tracing open All access to the file private data is handled by the helper functions, which do not use file->private_data. Stop updating it on open - Show ENUM names in function arguments via BTF in function tracing When showing the function arguments when func-args option is set for function tracing, if one of the arguments is found to be an enum, show the name of the enum instead of its number - Add new trace_call__##name() API for tracepoints Tracepoints are enabled via static_branch() blocks, where when not enabled, there's only a nop that is in the code where the execution will just skip over it. When tracing is enabled, the nop is converted to a direct jump to the tracepoint code. Sometimes more calculations are required to be performed to update the parameters of the tracepoint. In this case, trace_##name##_enabled() is called which is a static_branch() that gets enabled only when the tracepoint is enabled. This allows the extra calculations to also be skipped by the nop: if (trace_foo_enabled()) { x = bar(); trace_foo(x); } Where the x=bar() is only performed when foo is enabled. The problem with this approach is that there's now two static_branch() calls. One for checking if the tracepoint is enabled, and then again to know if the tracepoint should be called. The second one is redundant Introduce trace_call__foo() that will call the foo() tracepoint directly without doing a static_branch(): if (trace_foo_enabled()) { x = bar(); trace_call__foo(); } - Update various locations to use the new trace_call__##name() API - Move snapshot code out of trace.c Cleaning up trace.c to not be a "dump all", move the snapshot code out of it and into a new trace_snapshot.c file - Clean up some "%*.s" to "%*s" - Allow boot kernel command line options to be called multiple times Have options like: ftrace_filter=foo ftrace_filter=bar ftrace_filter=zoo Equal to: ftrace_filter=foo,bar,zoo - Fix ipi_raise event CPU field to be a CPU field The ipi_raise target_cpus field is defined as a __bitmask(). There is now a __cpumask() field definition. Update the field to use that - Have hist_field_name() use a snprintf() and not a series of strcat() It's safer to use snprintf() that a series of strcat() - Fix tracepoint regfunc balancing A tracepoint can define a "reg" and "unreg" function that gets called before the tracepoint is enabled, and after it is disabled respectively. But on error, after the "reg" func is called and the tracepoint is not enabled, the "unreg" function is not called to tear down what the "reg" function performed - Fix output that shows what histograms are enabled Event variables are displayed incorrectly in the histogram output Instead of "sched.sched_wakeup.$var", it is showing "$sched.sched_wakeup.var" where the '$' is in the incorrect location - Some other simple cleanups * tag 'trace-v7.1' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace: (24 commits) selftests/ftrace: Add test case for fully-qualified variable references tracing: Fix fully-qualified variable reference printing in histograms tracepoint: balance regfunc() on func_add() failure in tracepoint_add_func() tracing: Rebuild full_name on each hist_field_name() call tracing: Report ipi_raise target CPUs as cpumask tracing: Remove duplicate latency_fsnotify() stub tracing: Preserve repeated trace_trigger boot parameters tracing: Append repeated boot-time tracing parameters tracing: Remove spurious default precision from show_event_trigger/filter formats cpufreq: Use trace_call__##name() at guarded tracepoint call sites tracing: Remove tracing_alloc_snapshot() when snapshot isn't defined tracing: Move snapshot code out of trace.c and into trace_snapshot.c mm: damon: Use trace_call__##name() at guarded tracepoint call sites btrfs: Use trace_call__##name() at guarded tracepoint call sites spi: Use trace_call__##name() at guarded tracepoint call sites i2c: Use trace_call__##name() at guarded tracepoint call sites kernel: Use trace_call__##name() at guarded tracepoint call sites tracepoint: Add trace_call__##name() API tracing: trace_mmap.h: fix a kernel-doc warning tracing: Pretty-print enum parameters in function arguments ...
2026-04-02tracing: Make the backup instance non-reusableMasami Hiramatsu (Google)
Since there is no reason to reuse the backup instance, make it readonly (but erasable). Note that only backup instances are readonly, because other trace instances will be empty unless it is writable. Only backup instances have copy entries from the original. With this change, most of the trace control files are removed from the backup instance, including eventfs enable/filter etc. # find /sys/kernel/tracing/instances/backup/events/ | wc -l 4093 # find /sys/kernel/tracing/instances/boot_map/events/ | wc -l 9573 Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Link: https://patch.msgid.link/177502546939.1311542.1826814401724828930.stgit@mhiramat.tok.corp.google.com Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2026-03-31tracing: Preserve repeated trace_trigger boot parametersWesley Atwell
trace_trigger= tokenizes bootup_trigger_buf in place and stores pointers into that buffer for later trigger registration. Repeated trace_trigger= parameters overwrite the buffer contents from earlier calls, leaving only the last set of parsed event and trigger strings. Keep each new trace_trigger= string at the end of bootup_trigger_buf and parse only the appended range. That preserves the earlier event and trigger strings while still letting repeated parameters queue additional boot-time triggers. This also lets Bootconfig array values work naturally when they expand to repeated trace_trigger= entries. Before this change, only the last trace_trigger= instance survived boot. Link: https://patch.msgid.link/20260330181103.1851230-2-atwellwea@gmail.com Signed-off-by: Wesley Atwell <atwellwea@gmail.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2026-03-28tracing: Remove spurious default precision from show_event_trigger/filter ↵David Laight
formats Change 2d8b7f9bf8e6e ("tracing: Have show_event_trigger/filter format a bit more in columns") added space padding to align the output. However it used ("%*.s", len, "") which requests the default precision. It doesn't matter here whether the userspace default (0) or kernel default (no precision) is used, but the format should be "%*s". Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Rasmus Villemoes <linux@rasmusvillemoes.dk> Cc: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Cc: Sergey Senozhatsky <senozhatsky@chromium.org> Cc: Andrew Morton <akpm@linux-foundation.org> Link: https://patch.msgid.link/20260326201824.3919-1-david.laight.linux@gmail.com Signed-off-by: David Laight <david.laight.linux@gmail.com> Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org> Reviewed-by: Petr Mladek <pmladek@suse.com> Reviewed-by: Aaron Tomlin <atomlin@atomlin.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2026-03-23tracing: Clean up access to trace_event_file from a file pointerPetr Pavlu
The tracing code provides two functions event_file_file() and event_file_data() to obtain a trace_event_file pointer from a file struct. The primary method to use is event_file_file(), as it checks for the EVENT_FILE_FL_FREED flag to determine whether the event is being removed. The second function event_file_data() is an optimization for retrieving the same data when the event_mutex is still held. In the past, when removing an event directory in remove_event_file_dir(), the code set i_private to NULL for all event files and readers were expected to check for this state to recognize that the event is being removed. In the case of event_id_read(), the value was read using event_file_data() without acquiring the event_mutex. This required event_file_data() to use READ_ONCE() when retrieving the i_private data. With the introduction of eventfs, i_private is assigned when an eventfs inode is allocated and remains set throughout its lifetime. Remove the now unnecessary READ_ONCE() access to i_private in both event_file_file() and event_file_data(). Inline the access to i_private in remove_event_file_dir(), which allows event_file_data() to handle i_private solely as a trace_event_file pointer. Add a check in event_file_data() to ensure that the event_mutex is held and that file->flags doesn't have the EVENT_FILE_FL_FREED flag set. Finally, move event_file_data() immediately after event_file_code() since the latter provides a comment explaining how both functions should be used together. Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Tom Zanussi <zanussi@kernel.org> Link: https://patch.msgid.link/20260219162737.314231-5-petr.pavlu@suse.com Signed-off-by: Petr Pavlu <petr.pavlu@suse.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2026-03-23tracing: Remove unnecessary check for EVENT_FILE_FL_FREEDPetr Pavlu
The event_filter_write() function calls event_file_file() to retrieve a trace_event_file associated with a given file struct. If a non-NULL pointer is returned, the function then checks whether the trace_event_file instance has the EVENT_FILE_FL_FREED flag set. This check is redundant because event_file_file() already performs this validation and returns NULL if the flag is set. The err value is also already initialized to -ENODEV. Remove the unnecessary check for EVENT_FILE_FL_FREED in event_filter_write(). Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Tom Zanussi <zanussi@kernel.org> Link: https://patch.msgid.link/20260219162737.314231-4-petr.pavlu@suse.com Signed-off-by: Petr Pavlu <petr.pavlu@suse.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2026-03-06tracing: Fix enabling multiple events on the kernel command line and bootconfigAndrei-Alexandru Tachici
Multiple events can be enabled on the kernel command line via a comma separator. But if the are specified one at a time, then only the last event is enabled. This is because the event names are saved in a temporary buffer, and each call by the init cmdline code will reset that buffer. This also affects names in the boot config file, as it may call the callback multiple times with an example of: kernel.trace_event = ":mod:rproc_qcom_common", ":mod:qrtr", ":mod:qcom_aoss" Change the cmdline callback function to append a comma and the next value if the temporary buffer already has content. Cc: stable@vger.kernel.org Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Link: https://patch.msgid.link/20260302-trace-events-allow-multiple-modules-v1-1-ce4436e37fb8@oss.qualcomm.com Signed-off-by: Andrei-Alexandru Tachici <andrei-alexandru.tachici@oss.qualcomm.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2026-03-03tracing: Disable preemption in the tracepoint callbacks handling filtered pidsMasami Hiramatsu (Google)
Filtering PIDs for events triggered the following during selftests: [37] event tracing - restricts events based on pid notrace filtering [ 155.874095] [ 155.874869] ============================= [ 155.876037] WARNING: suspicious RCU usage [ 155.877287] 7.0.0-rc1-00004-g8cd473a19bc7 #7 Not tainted [ 155.879263] ----------------------------- [ 155.882839] kernel/trace/trace_events.c:1057 suspicious rcu_dereference_check() usage! [ 155.889281] [ 155.889281] other info that might help us debug this: [ 155.889281] [ 155.894519] [ 155.894519] rcu_scheduler_active = 2, debug_locks = 1 [ 155.898068] no locks held by ftracetest/4364. [ 155.900524] [ 155.900524] stack backtrace: [ 155.902645] CPU: 1 UID: 0 PID: 4364 Comm: ftracetest Not tainted 7.0.0-rc1-00004-g8cd473a19bc7 #7 PREEMPT(lazy) [ 155.902648] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.17.0-debian-1.17.0-1 04/01/2014 [ 155.902651] Call Trace: [ 155.902655] <TASK> [ 155.902659] dump_stack_lvl+0x67/0x90 [ 155.902665] lockdep_rcu_suspicious+0x154/0x1a0 [ 155.902672] event_filter_pid_sched_process_fork+0x9a/0xd0 [ 155.902678] kernel_clone+0x367/0x3a0 [ 155.902689] __x64_sys_clone+0x116/0x140 [ 155.902696] do_syscall_64+0x158/0x460 [ 155.902700] ? entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 155.902702] ? trace_irq_disable+0x1d/0xc0 [ 155.902709] entry_SYSCALL_64_after_hwframe+0x77/0x7f [ 155.902711] RIP: 0033:0x4697c3 [ 155.902716] Code: 1f 84 00 00 00 00 00 64 48 8b 04 25 10 00 00 00 45 31 c0 31 d2 31 f6 bf 11 00 20 01 4c 8d 90 d0 02 00 00 b8 38 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 35 89 c2 85 c0 75 2c 64 48 8b 04 25 10 00 00 [ 155.902718] RSP: 002b:00007ffc41150428 EFLAGS: 00000246 ORIG_RAX: 0000000000000038 [ 155.902721] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00000000004697c3 [ 155.902722] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000001200011 [ 155.902724] RBP: 0000000000000000 R08: 0000000000000000 R09: 000000003fccf990 [ 155.902725] R10: 000000003fccd690 R11: 0000000000000246 R12: 0000000000000001 [ 155.902726] R13: 000000003fce8103 R14: 0000000000000001 R15: 0000000000000000 [ 155.902733] </TASK> [ 155.902747] The tracepoint callbacks recently were changed to allow preemption. The event PID filtering callbacks that were attached to the fork and exit tracepoints expected preemption disabled in order to access the RCU protected PID lists. Add a guard(preempt)() to protect the references to the PID list. Cc: stable@vger.kernel.org Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Link: https://patch.msgid.link/20260303215738.6ab275af@fedora Fixes: a46023d5616e ("tracing: Guard __DECLARE_TRACE() use of __DO_TRACE_CALL() with SRCU-fast") Link: https://patch.msgid.link/20260303131706.96057f61a48a34c43ce1e396@kernel.org Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2026-03-03tracing: Fix syscall events activation by ensuring refcount hits zeroHuiwen He
When multiple syscall events are specified in the kernel command line (e.g., trace_event=syscalls:sys_enter_openat,syscalls:sys_enter_close), they are often not captured after boot, even though they appear enabled in the tracing/set_event file. The issue stems from how syscall events are initialized. Syscall tracepoints require the global reference count (sys_tracepoint_refcount) to transition from 0 to 1 to trigger the registration of the syscall work (TIF_SYSCALL_TRACEPOINT) for tasks, including the init process (pid 1). The current implementation of early_enable_events() with disable_first=true used an interleaved sequence of "Disable A -> Enable A -> Disable B -> Enable B". If multiple syscalls are enabled, the refcount never drops to zero, preventing the 0->1 transition that triggers actual registration. Fix this by splitting early_enable_events() into two distinct phases: 1. Disable all events specified in the buffer. 2. Enable all events specified in the buffer. This ensures the refcount hits zero before re-enabling, allowing syscall events to be properly activated during early boot. The code is also refactored to use a helper function to avoid logic duplication between the disable and enable phases. Cc: stable@vger.kernel.org Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Link: https://patch.msgid.link/20260224023544.1250787-1-hehuiwen@kylinos.cn Fixes: ce1039bd3a89 ("tracing: Fix enabling of syscall events on the command line") Signed-off-by: Huiwen He <hehuiwen@kylinos.cn> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2026-02-21Convert 'alloc_obj' family to use the new default GFP_KERNEL argumentLinus Torvalds
This was done entirely with mindless brute force, using git grep -l '\<k[vmz]*alloc_objs*(.*, GFP_KERNEL)' | xargs sed -i 's/\(alloc_objs*(.*\), GFP_KERNEL)/\1)/' to convert the new alloc_obj() users that had a simple GFP_KERNEL argument to just drop that argument. Note that due to the extreme simplicity of the scripting, any slightly more complex cases spread over multiple lines would not be triggered: they definitely exist, but this covers the vast bulk of the cases, and the resulting diff is also then easier to check automatically. For the same reason the 'flex' versions will be done as a separate conversion. Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2026-02-21treewide: Replace kmalloc with kmalloc_obj for non-scalar typesKees Cook
This is the result of running the Coccinelle script from scripts/coccinelle/api/kmalloc_objs.cocci. The script is designed to avoid scalar types (which need careful case-by-case checking), and instead replace kmalloc-family calls that allocate struct or union object instances: Single allocations: kmalloc(sizeof(TYPE), ...) are replaced with: kmalloc_obj(TYPE, ...) Array allocations: kmalloc_array(COUNT, sizeof(TYPE), ...) are replaced with: kmalloc_objs(TYPE, COUNT, ...) Flex array allocations: kmalloc(struct_size(PTR, FAM, COUNT), ...) are replaced with: kmalloc_flex(*PTR, FAM, COUNT, ...) (where TYPE may also be *VAR) The resulting allocations no longer return "void *", instead returning "TYPE *". Signed-off-by: Kees Cook <kees@kernel.org>
2026-02-19tracing: Wake up poll waiters for hist files when removing an eventPetr Pavlu
The event_hist_poll() function attempts to verify whether an event file is being removed, but this check may not occur or could be unnecessarily delayed. This happens because hist_poll_wakeup() is currently invoked only from event_hist_trigger() when a hist command is triggered. If the event file is being removed, no associated hist command will be triggered and a waiter will be woken up only after an unrelated hist command is triggered. Fix the issue by adding a call to hist_poll_wakeup() in remove_event_file_dir() after setting the EVENT_FILE_FL_FREED flag. This ensures that a task polling on a hist file is woken up and receives EPOLLERR. Cc: stable@vger.kernel.org Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Tom Zanussi <zanussi@kernel.org> Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org> Link: https://patch.msgid.link/20260219162737.314231-3-petr.pavlu@suse.com Fixes: 1bd13edbbed6 ("tracing/hist: Add poll(POLLIN) support on hist file") Signed-off-by: Petr Pavlu <petr.pavlu@suse.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2026-02-08tracing: Make tracing_disabled global for tracing systemSteven Rostedt
The tracing_disabled variable is set to one on boot up to prevent some parts of tracing to access the tracing infrastructure before it is set up. It also can be set after boot if an anomaly is discovered. It is currently a static variable in trace.c and can be accessed via a function call trace_is_disabled(). There's really no reason to use a function call as the tracing subsystem should be able to access it directly. By making the variable accessed directly, code can be moved out of trace.c without adding overhead of a function call to see if tracing is disabled or not. Make tracing_disabled global and remove the tracing_is_disabled() helper function. Also add some "unlikely()"s around tracing_disabled where it's checked in hot paths. Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Andrew Morton <akpm@linux-foundation.org> Link: https://patch.msgid.link/20260208032449.483690153@kernel.org Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2026-01-30tracing: Add kerneldoc to trace_event_buffer_reserve()Steven Rostedt
Add a appropriate kerneldoc to trace_event_buffer_reserve() to make it easier to understand how that function is used. Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Link: https://patch.msgid.link/20260130103745.1126e4af@gandalf.local.home Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2026-01-28tracing: Remove duplicate ENABLE_EVENT_STR and DISABLE_EVENT_STR macrosSteven Rostedt
The macros ENABLE_EVENT_STR and DISABLE_EVENT_STR were added to trace.h so that more than one file can have access to them, but was never removed from their original location. Remove the duplicates. Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Tom Zanussi <zanussi@kernel.org> Link: https://patch.msgid.link/20260126130037.4ba201f9@gandalf.local.home Fixes: d0bad49bb0a09 ("tracing: Add enable_hist/disable_hist triggers") Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2026-01-26tracing: Have show_event_trigger/filter format a bit more in columnsSteven Rostedt
By doing: # trace-cmd sqlhist -e -n futex_wait select TIMESTAMP_DELTA_USECS as lat from sys_enter_futex as start join sys_exit_futex as end on start.common_pid = end.common_pid and # trace-cmd start -e futex_wait -f 'lat > 100' -e page_pool_state_release -f 'pfn == 1' The output of the show_event_trigger and show_event_filter files are well aligned because of the inconsistent 'tab' spacing: ~# cat /sys/kernel/tracing/show_event_triggers syscalls:sys_exit_futex hist:keys=common_pid:vals=hitcount:__lat_12046_2=common_timestamp.usecs-$__arg_12046_1:sort=hitcount:size=2048:clock=global:onmatch(syscalls.sys_enter_futex).trace(futex_wait,$__lat_12046_2) [active] syscalls:sys_enter_futex hist:keys=common_pid:vals=hitcount:__arg_12046_1=common_timestamp.usecs:sort=hitcount:size=2048:clock=global [active] ~# cat /sys/kernel/tracing/show_event_filters synthetic:futex_wait (lat > 100) page_pool:page_pool_state_release (pfn == 1) This makes it not so easy to read. Instead, force the spacing to be at least 32 bytes from the beginning (one space if the system:event is longer than 30 bytes): ~# cat /sys/kernel/tracing/show_event_triggers syscalls:sys_exit_futex hist:keys=common_pid:vals=hitcount:__lat_8125_2=common_timestamp.usecs-$__arg_8125_1:sort=hitcount:size=2048:clock=global:onmatch(syscalls.sys_enter_futex).trace(futex_wait,$__lat_8125_2) [active] syscalls:sys_enter_futex hist:keys=common_pid:vals=hitcount:__arg_8125_1=common_timestamp.usecs:sort=hitcount:size=2048:clock=global [active] ~# cat /sys/kernel/tracing/show_event_filters synthetic:futex_wait (lat > 100) page_pool:page_pool_state_release (pfn == 1) Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Link: https://patch.msgid.link/20260112153408.18373e73@gandalf.local.home Reviewed-by: Aaron Tomlin <atomlin@atomlin.com> Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2026-01-26tracing: Add show_event_triggers to expose active event triggersAaron Tomlin
To audit active event triggers, userspace currently must traverse the events/ directory and read each individual trigger file. This is cumbersome for system-wide auditing or debugging. Introduce "show_event_triggers" at the trace root directory. This file displays all events that currently have one or more triggers applied, alongside the trigger configuration, in a consolidated system:event [tab] trigger format. The implementation leverages the existing trace_event_file iterators and uses the trigger's own print() operation to ensure output consistency with the per-event trigger files. Link: https://patch.msgid.link/20260105142939.2655342-3-atomlin@atomlin.com Signed-off-by: Aaron Tomlin <atomlin@atomlin.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2026-01-26tracing: Add show_event_filters to expose active event filtersAaron Tomlin
Currently, to audit active Ftrace event filters, userspace must recursively traverse the events/ directory and read each individual filter file. This is inefficient for monitoring tools and debugging. Introduce "show_event_filters" at the trace root directory. This file displays all events that currently have a filter applied, alongside the actual filter string, in a consolidated system:event [tab] filter format. The implementation reuses the existing trace_event_file iterators to ensure atomic traversal of the event list and utilises guard(rcu)() for automatic, scope-based protection when accessing volatile filter strings. Link: https://patch.msgid.link/20260105142939.2655342-2-atomlin@atomlin.com Signed-off-by: Aaron Tomlin <atomlin@atomlin.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2026-01-07tracing: Drop unneeded assignment to soft_modeJulia Lawall
soft_mode is not read in the enable case, so drop the assignment. Drop also the comment text that refers to the assignment and realign the comment. Cc: "Paul E . McKenney" <paulmck@kernel.org> Cc: Gabriele Paoloni <gpaoloni@redhat.com> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Link: https://patch.msgid.link/20251226110531.4129794-1-Julia.Lawall@inria.fr Signed-off-by: Julia Lawall <Julia.Lawall@inria.fr> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2025-12-17tracing: Do not register unsupported perf eventsSteven Rostedt
Synthetic events currently do not have a function to register perf events. This leads to calling the tracepoint register functions with a NULL function pointer which triggers: ------------[ cut here ]------------ WARNING: kernel/tracepoint.c:175 at tracepoint_add_func+0x357/0x370, CPU#2: perf/2272 Modules linked in: kvm_intel kvm irqbypass CPU: 2 UID: 0 PID: 2272 Comm: perf Not tainted 6.18.0-ftest-11964-ge022764176fc-dirty #323 PREEMPTLAZY Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.17.0-debian-1.17.0-1 04/01/2014 RIP: 0010:tracepoint_add_func+0x357/0x370 Code: 28 9c e8 4c 0b f5 ff eb 0f 4c 89 f7 48 c7 c6 80 4d 28 9c e8 ab 89 f4 ff 31 c0 5b 41 5c 41 5d 41 5e 41 5f 5d c3 cc cc cc cc cc <0f> 0b 49 c7 c6 ea ff ff ff e9 ee fe ff ff 0f 0b e9 f9 fe ff ff 0f RSP: 0018:ffffabc0c44d3c40 EFLAGS: 00010246 RAX: 0000000000000001 RBX: ffff9380aa9e4060 RCX: 0000000000000000 RDX: 000000000000000a RSI: ffffffff9e1d4a98 RDI: ffff937fcf5fd6c8 RBP: 0000000000000001 R08: 0000000000000007 R09: ffff937fcf5fc780 R10: 0000000000000003 R11: ffffffff9c193910 R12: 000000000000000a R13: ffffffff9e1e5888 R14: 0000000000000000 R15: ffffabc0c44d3c78 FS: 00007f6202f5f340(0000) GS:ffff93819f00f000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 000055d3162281a8 CR3: 0000000106a56003 CR4: 0000000000172ef0 Call Trace: <TASK> tracepoint_probe_register+0x5d/0x90 synth_event_reg+0x3c/0x60 perf_trace_event_init+0x204/0x340 perf_trace_init+0x85/0xd0 perf_tp_event_init+0x2e/0x50 perf_try_init_event+0x6f/0x230 ? perf_event_alloc+0x4bb/0xdc0 perf_event_alloc+0x65a/0xdc0 __se_sys_perf_event_open+0x290/0x9f0 do_syscall_64+0x93/0x7b0 ? entry_SYSCALL_64_after_hwframe+0x76/0x7e ? trace_hardirqs_off+0x53/0xc0 entry_SYSCALL_64_after_hwframe+0x76/0x7e Instead, have the code return -ENODEV, which doesn't warn and has perf error out with: # perf record -e synthetic:futex_wait Error: The sys_perf_event_open() syscall returned with 19 (No such device) for event (synthetic:futex_wait). "dmesg | grep -i perf" may provide additional information. Ideally perf should support synthetic events, but for now just fix the warning. The support can come later. Cc: stable@vger.kernel.org Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Arnaldo Carvalho de Melo <acme@kernel.org> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Namhyung Kim <namhyung@kernel.org> Link: https://patch.msgid.link/20251216182440.147e4453@gandalf.local.home Fixes: 4b147936fa509 ("tracing: Add support for 'synthetic' events") Reported-by: Ian Rogers <irogers@google.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2025-12-05tracing: Fix multiple typos in trace_events.cMaurice Hieronymus
Fix multiple typos in comments: "appened" -> "appended" "paranthesis" -> "parenthesis" "parethesis" -> "parenthesis" "wont" -> "won't" Link: https://patch.msgid.link/20251121221835.28032-8-mhi@mailbox.org Signed-off-by: Maurice Hieronymus <mhi@mailbox.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2025-11-04tracing: Allow tracer to add more than 32 optionsMasami Hiramatsu (Google)
Since enum trace_iterator_flags is 32bit, the max number of the option flags is limited to 32 and it is fully used now. To add a new option, we need to expand it. So replace the TRACE_ITER_##flag with TRACE_ITER(flag) macro which is 64bit bitmask. Link: https://lore.kernel.org/all/176187877103.994619.166076000668757232.stgit@devnote2/ Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>
2025-10-03tracing: Fix lock imbalance in s_start() memory allocation failure pathSasha Levin
When s_start() fails to allocate memory for set_event_iter, it returns NULL before acquiring event_mutex. However, the corresponding s_stop() function always tries to unlock the mutex, causing a lock imbalance warning: WARNING: bad unlock balance detected! 6.17.0-rc7-00175-g2b2e0c04f78c #7 Not tainted ------------------------------------- syz.0.85611/376514 is trying to release lock (event_mutex) at: [<ffffffff8dafc7a4>] traverse.part.0.constprop.0+0x2c4/0x650 fs/seq_file.c:131 but there are no more locks to release! The issue was introduced by commit b355247df104 ("tracing: Cache ':mod:' events for modules not loaded yet") which added the kzalloc() allocation before the mutex lock, creating a path where s_start() could return without locking the mutex while s_stop() would still try to unlock it. Fix this by unconditionally acquiring the mutex immediately after allocation, regardless of whether the allocation succeeded. Cc: stable@vger.kernel.org Link: https://lore.kernel.org/20250929113238.3722055-1-sashal@kernel.org Fixes: b355247df104 ("tracing: Cache ":mod:" events for modules not loaded yet") Signed-off-by: Sasha Levin <sashal@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2025-08-01Merge tag 'trace-v6.17' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace Pull tracing updates from Steven Rostedt: - Deprecate auto-mounting tracefs to /sys/kernel/debug/tracing When tracefs was first introduced back in 2014, the directory /sys/kernel/tracing was added and is the designated location to mount tracefs. To keep backward compatibility, tracefs was auto-mounted in /sys/kernel/debug/tracing as well. All distros now mount tracefs on /sys/kernel/tracing. Having it seen in two different locations has lead to various issues and inconsistencies. The VFS folks have to also maintain debugfs_create_automount() for this single user. It's been over 10 years. Tooling and scripts should start replacing the debugfs location with the tracefs one. The reason tracefs was created in the first place was to allow access to the tracing facilities without the need to configure debugfs into the kernel. Using tracefs should now be more robust. A new config is created: CONFIG_TRACEFS_AUTOMOUNT_DEPRECATED which is default y, so that the kernel is still built with the automount. This config allows those that want to remove the automount from debugfs to do so. When tracefs is accessed from /sys/kernel/debug/tracing, the following printk is triggerd: pr_warn("NOTICE: Automounting of tracing to debugfs is deprecated and will be removed in 2030\n"); This gives users another 5 years to fix their scripts. - Use queue_rcu_work() instead of call_rcu() for freeing event filters The number of filters to be free can be many depending on the number of events within an event system. Freeing them from softirq context can potentially cause undesired latency. Use the RCU workqueue to free them instead. - Remove pointless memory barriers in latency code Memory barriers were added to some of the latency code a long time ago with the idea of "making them visible", but that's not what memory barriers are for. They are to synchronize access between different variables. There was no synchronization here making them pointless. - Remove "__attribute__()" from the type field of event format When LLVM is used to compile the kernel with CONFIG_DEBUG_INFO_BTF=y and PAHOLE_HAS_BTF_TAG=y, some of the format fields get expanded with the following: field:const char * filename; offset:24; size:8; signed:0; Turns into: field:const char __attribute__((btf_type_tag("user"))) * filename; offset:24; size:8; signed:0; This confuses parsers. Add code to strip these tags from the strings. - Add eprobe config option CONFIG_EPROBE_EVENTS Eprobes were added back in 5.15 but were only enabled when another probe was enabled (kprobe, fprobe, uprobe, etc). The eprobes had no config option of their own. Add one as they should be a separate entity. It's default y to keep with the old kernels but still has dependencies on TRACING and HAVE_REGS_AND_STACK_ACCESS_API. - Add eprobe documentation When eprobes were added back in 5.15 no documentation was added to describe them. This needs to be rectified. - Replace open coded cpumask_next_wrap() in move_to_next_cpu() - Have preemptirq_delay_run() use off-stack CPU mask - Remove obsolete comment about pelt_cfs event DECLARE_TRACE() appends "_tp" to trace events now, but the comment above pelt_cfs still mentioned appending it manually. - Remove EVENT_FILE_FL_SOFT_MODE flag The SOFT_MODE flag was required when the soft enabling and disabling of trace events was first introduced. But there was a bug with this approach as it only worked for a single instance. When multiple users required soft disabling and disabling the code was changed to have a ref count. The SOFT_MODE flag is now set iff the ref count is non zero. This is redundant and just reading the ref count is good enough. - Fix typo in comment * tag 'trace-v6.17' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace: Documentation: tracing: Add documentation about eprobes tracing: Have eprobes have their own config option tracing: Remove "__attribute__()" from the type field of event format tracing: Deprecate auto-mounting tracefs in debugfs tracing: Fix comment in trace_module_remove_events() tracing: Remove EVENT_FILE_FL_SOFT_MODE flag tracing: Remove pointless memory barriers tracing/sched: Remove obsolete comment on suffixes kernel: trace: preemptirq_delay_test: use offstack cpu mask tracing: Use queue_rcu_work() to free filters tracing: Replace opencoded cpumask_next_wrap() in move_to_next_cpu()
2025-07-29tracing: Remove "__attribute__()" from the type field of event formatMasami Hiramatsu (Google)
With CONFIG_DEBUG_INFO_BTF=y and PAHOLE_HAS_BTF_TAG=y, `__user` is converted to `__attribute__((btf_type_tag("user")))`. In this case, some syscall events have it for __user data, like below; /sys/kernel/tracing # cat events/syscalls/sys_enter_openat/format name: sys_enter_openat ID: 720 format: field:unsigned short common_type; offset:0; size:2; signed:0; field:unsigned char common_flags; offset:2; size:1; signed:0; field:unsigned char common_preempt_count; offset:3; size:1; signed:0; field:int common_pid; offset:4; size:4; signed:1; field:int __syscall_nr; offset:8; size:4; signed:1; field:int dfd; offset:16; size:8; signed:0; field:const char __attribute__((btf_type_tag("user"))) * filename; offset:24; size:8; signed:0; field:int flags; offset:32; size:8; signed:0; field:umode_t mode; offset:40; size:8; signed:0; Then the trace event filter fails to set the string acceptable flag (FILTER_PTR_STRING) to the field and rejects setting string filter; # echo 'filename.ustring ~ "*ftracetest-dir.wbx24v*"' \ >> events/syscalls/sys_enter_openat/filter sh: write error: Invalid argument # cat error_log [ 723.743637] event filter parse error: error: Expecting numeric field Command: filename.ustring ~ "*ftracetest-dir.wbx24v*" Since this __attribute__ makes format parsing complicated and not needed, remove the __attribute__(.*) from the type string. Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Link: https://lore.kernel.org/175376583493.1688759.12333973498014733551.stgit@mhiramat.tok.corp.google.com Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2025-07-22tracing: Fix comment in trace_module_remove_events()Steven Rostedt
Fix typo "allocade" -> "allocated". Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Link: https://lore.kernel.org/20250710095628.42ed6b06@batman.local.home Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2025-07-22tracing: Remove EVENT_FILE_FL_SOFT_MODE flagSteven Rostedt
When soft disabling of trace events was first created, it needed to have a way to know if a file had a user that was using it with soft disabled (for triggers that need to enable or disable events from a context that can not really enable or disable the event, it would set SOFT_DISABLED to state it is disabled). The flag SOFT_MODE was used to denote that an event had a user that would enable or disable it via the SOFT_DISABLED flag. Commit 1cf4c0732db3c ("tracing: Modify soft-mode only if there's no other referrer") fixed a bug where if two users were using the SOFT_DISABLED flag the accounting would get messed up as the SOFT_MODE flag could only handle one user. That commit added the sm_ref counter which kept track of how many users were using the event in "soft mode". This made the SOFT_MODE flag redundant as it should only be set if the sm_ref counter is non zero. Remove the SOFT_MODE flag and just use the sm_ref counter to know the event is in soft mode or not. This makes the code a bit simpler. Link: https://lore.kernel.org/all/20250702111908.03759998@batman.local.home/ Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Gabriele Paoloni <gpaoloni@redhat.com> Link: https://lore.kernel.org/20250702143657.18dd1882@batman.local.home Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2025-07-19tracing: Add down_write(trace_event_sem) when adding trace eventSteven Rostedt
When a module is loaded, it adds trace events defined by the module. It may also need to modify the modules trace printk formats to replace enum names with their values. If two modules are loaded at the same time, the adding of the event to the ftrace_events list can corrupt the walking of the list in the code that is modifying the printk format strings and crash the kernel. The addition of the event should take the trace_event_sem for write while it adds the new event. Also add a lockdep_assert_held() on that semaphore in __trace_add_event_dirs() as it iterates the list. Cc: stable@vger.kernel.org Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org> Link: https://lore.kernel.org/20250718223158.799bfc0c@batman.local.home Reported-by: Fusheng Huang(黄富生) <Fusheng.Huang@luxshare-ict.com> Closes: https://lore.kernel.org/all/20250717105007.46ccd18f@batman.local.home/ Fixes: 110bf2b764eb6 ("tracing: add protection around module events unload") Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2025-05-09tracing: Add a helper function to handle the dereference arg in verifierSteven Rostedt
Add a helper function called handle_dereference_arg() to replace the logic that is identical in two locations of test_event_printk(). Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Link: https://lore.kernel.org/20250507191703.5dd8a61d@gandalf.local.home Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2025-05-08tracing: Just use this_cpu_read() to access ignore_pidSteven Rostedt
The ignore_pid boolean on the per CPU data descriptor is updated at sched_switch when a new task is scheduled in. If the new task is to be ignored, it is set to true, otherwise it is set to false. The current task should always have the correct value as it is updated when the task is scheduled in. Instead of breaking up the read of this value, which requires preemption to be disabled, just use this_cpu_read() which gives a snapshot of the value. Since the value will always be correct for a given task (because it's updated at sched switch) it doesn't need preemption disabled. This will also allow trace events to be called with preemption enabled. Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Andrew Morton <akpm@linux-foundation.org> Link: https://lore.kernel.org/20250505212235.038958766@goodmis.org Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2025-04-02tracing: Verify event formats that have "%*p.."Steven Rostedt
The trace event verifier checks the formats of trace events to make sure that they do not point at memory that is not in the trace event itself or in data that will never be freed. If an event references data that was allocated when the event triggered and that same data is freed before the event is read, then the kernel can crash by reading freed memory. The verifier runs at boot up (or module load) and scans the print formats of the events and checks their arguments to make sure that dereferenced pointers are safe. If the format uses "%*p.." the verifier will ignore it, and that could be dangerous. Cover this case as well. Also add to the sample code a use case of "%*pbl". Link: https://lore.kernel.org/all/bcba4d76-2c3f-4d11-baf0-02905db953dd@oracle.com/ Cc: stable@vger.kernel.org Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Fixes: 5013f454a352c ("tracing: Add check of trace event print fmts for dereferencing pointers") Link: https://lore.kernel.org/20250327195311.2d89ec66@gandalf.local.home Reported-by: Libo Chen <libo.chen@oracle.com> Reviewed-by: Libo Chen <libo.chen@oracle.com> Tested-by: Libo Chen <libo.chen@oracle.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2025-03-31Merge tag 'trace-ringbuffer-v6.15-2' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace Pull ring-buffer updates from Steven Rostedt: - Restructure the persistent memory to have a "scratch" area Instead of hard coding the KASLR offset in the persistent memory by the ring buffer, push that work up to the callers of the persistent memory as they are the ones that need this information. The offsets and such is not important to the ring buffer logic and it should not be part of that. A scratch pad is now created when the caller allocates a ring buffer from persistent memory by stating how much memory it needs to save. - Allow where modules are loaded to be saved in the new scratch pad Save the addresses of modules when they are loaded into the persistent memory scratch pad. - A new module_for_each_mod() helper function was created With the acknowledgement of the module maintainers a new module helper function was created to iterate over all the currently loaded modules. This has a callback to be called for each module. This is needed for when tracing is started in the persistent buffer and the currently loaded modules need to be saved in the scratch area. - Expose the last boot information where the kernel and modules were loaded The last_boot_info file is updated to print out the addresses of where the kernel "_text" location was loaded from a previous boot, as well as where the modules are loaded. If the buffer is recording the current boot, it only prints "# Current" so that it does not expose the KASLR offset of the currently running kernel. - Allow the persistent ring buffer to be released (freed) To have this in production environments, where the kernel command line can not be changed easily, the ring buffer needs to be freed when it is not going to be used. The memory for the buffer will always be allocated at boot up, but if the system isn't going to enable tracing, the memory needs to be freed. Allow it to be freed and added back to the kernel memory pool. - Allow stack traces to print the function names in the persistent buffer Now that the modules are saved in the persistent ring buffer, if the same modules are loaded, the printing of the function names will examine the saved modules. If the module is found in the scratch area and is also loaded, then it will do the offset shift and use kallsyms to display the function name. If the address is not found, it simply displays the address from the previous boot in hex. * tag 'trace-ringbuffer-v6.15-2' of git://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace: tracing: Use _text and the kernel offset in last_boot_info tracing: Show last module text symbols in the stacktrace ring-buffer: Remove the unused variable bmeta tracing: Skip update_last_data() if cleared and remove active check for save_mod() tracing: Initialize scratch_size to zero to prevent UB tracing: Fix a compilation error without CONFIG_MODULES tracing: Freeable reserved ring buffer mm/memblock: Add reserved memory release function tracing: Update modules to persistent instances when loaded tracing: Show module names and addresses of last boot tracing: Have persistent trace instances save module addresses module: Add module_for_each_mod() function tracing: Have persistent trace instances save KASLR offset ring-buffer: Add ring_buffer_meta_scratch() ring-buffer: Add buffer meta data for persistent ring buffer ring-buffer: Use kaslr address instead of text delta ring-buffer: Fix bytes_dropped calculation issue
2025-03-28tracing: Update modules to persistent instances when loadedSteven Rostedt
When a module is loaded and a persistent buffer is actively tracing, add it to the list of modules in the persistent memory. Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Andrew Morton <akpm@linux-foundation.org> Link: https://lore.kernel.org/20250305164609.469844721@goodmis.org Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2025-03-23tracing: fix return value in __ftrace_event_enable_disable for ↵Gabriele Paoloni
TRACE_REG_UNREGISTER When __ftrace_event_enable_disable invokes the class callback to unregister the event, the return value is not reported up to the caller, hence leading to event unregister failures being silently ignored. This patch assigns the ret variable to the invocation of the event unregister callback, so that its return value is stored and reported to the caller, and it raises a warning in case of error. Link: https://lore.kernel.org/20250321170821.101403-1-gpaoloni@redhat.com Signed-off-by: Gabriele Paoloni <gpaoloni@redhat.com> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
2025-02-21tracing: Fix memory leak when reading set_event fileAdrian Huang
kmemleak reports the following memory leak after reading set_event file: # cat /sys/kernel/tracing/set_event # cat /sys/kernel/debug/kmemleak unreferenced object 0xff110001234449e0 (size 16): comm "cat", pid 13645, jiffies 4294981880 hex dump (first 16 bytes): 01 00 00 00 00 00 00 00 a8 71 e7 84 ff ff ff ff .........q...... backtrace (crc c43abbc): __kmalloc_cache_noprof+0x3ca/0x4b0 s_start+0x72/0x2d0 seq_read_iter+0x265/0x1080 seq_read+0x2c9/0x420 vfs_read+0x166/0xc30 ksys_read+0xf4/0x1d0 do_syscall_64+0x79/0x150 entry_SYSCALL_64_after_hwframe+0x76/0x7e The issue can be reproduced regardless of whether set_event is empty or not. Here is an example about the valid content of set_event. # cat /sys/kernel/tracing/set_event sched:sched_process_fork sched:sched_switch sched:sched_wakeup *:*:mod:trace_events_sample The root cause is that s_next() returns NULL when nothing is found. This results in s_stop() attempting to free a NULL pointer because its parameter is NULL. Fix the issue by freeing the memory appropriately when s_next() fails to find anything. Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Link: https://lore.kernel.org/20250220031528.7373-1-ahuang12@lenovo.com Fixes: b355247df104 ("tracing: Cache ":mod:" events for modules not loaded yet") Signed-off-by: Adrian Huang <ahuang12@lenovo.com> Acked-by: Masami Hiramatsu (Google) <mhiramat@kernel.org> Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>