| Age | Commit message (Collapse) | Author |
|---|
|
Each osnoise tracer option (in /sys/kernel/tracing/osnoise) used by RTLA
requires four functions to be defined:
- static osnoise_get_<opt>() - to get the current value of the option
and save it into struct osnoise_context's orig_<opt> field,
- osnoise_set_<opt>() - to set the value of the option requested by the
user after reading and saving the original with osnoise_get_<opt>(),
and save it into <opt> field of struct osnoise_context,
- osnoise_restore_<opt>() - restore the value recorded in orig_<opt>,
- static osnoise_put_<opt>() - restore the value recorded in orig_<opt>
and update <opt> to reflect that.
The logic is duplicated for all the options, except for cpus (which is
the only string option) and period/runtime (which are handled together
and feature extra checks).
Deduplicate by moving the common code in get/set/restore/put into common
helper functions local to osnoise.c. The original set/restore/put
functions are reduced to wrappers generated using a set of X macros:
- OSNOISE_LL_OPTIONS, which invokes the OSNOISE_LL_OPTION macro for all
"long long" options,
- OSNOISE_FLAG_OPTIONS, which invokes the OSNOISE_FLAG_OPTION macro for
all flag options (boolean values in osnoise/options file).
Those are also used to auto-generate the corresponding fields in struct
osnoise_context, plus initialization and restoration code for them.
There are a few minor changes in the refactored code:
- osnoise_set_<opt>() now distinguishes between -2 (option cannot be
set) and -1 (option not present in kernel) for all options.
- OSNOISE_OPTION_INIT_VAL and OSNOISE_TIME_INIT_VAL are removed and
replaced with a plain -1; all unsigned long long fields are changed to
signed long long, consistent with pre-existing "%lld" format string in
osnoise_write_ll_config().
- osnoise_restore_<opt>() now correctly resets the <opt> field (not
orig_<opt>) for all options.
The change is intended to simplify adding and modifying tracer options.
It also makes the code shorter by a bit over 500 lines.
Assisted-by: Claude:claude-opus-4-6
Link: https://lore.kernel.org/r/20260929142944.1841462-1-tglozar@redhat.com
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
|
|
IPIs can now be monitored and accounted by osnoise top. When that is
the case, also record them when saving a trace file.
To match what is being recorded by the tool for its own analysis, event
filters are applied to the events recorded to the trace output.
Signed-off-by: Valentin Schneider <vschneid@redhat.com>
Link: https://lore.kernel.org/r/20260902123942.695822-9-vschneid@redhat.com
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
|
|
Later commits will add IPI tracking to osnoise top. To avoid breaking
existing scripts, this new feature will be gated behind a new --ipi option.
Suggested-by: Tomas Glozar <tglozar@redhat.com>
Signed-off-by: Valentin Schneider <vschneid@redhat.com>
Link: https://lore.kernel.org/r/20260902123942.695822-2-vschneid@redhat.com
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
|
|
In rare situations, sysfs might report a different number of CPUs than
the kernel sees, for example, when /sys is not real sysfs.
In tracefs mode, RTLA will discard any samples that are reported on CPUs
higher than the maximum in sysfs. However, in BPF mode, the incorrectly
sized BPF_MAP_LOOKUP_ELEM syscall on percpu maps will cause a buffer
overflow.
To cover for this corner case, write "all" to osnoise/cpus, and read
back the real kernel-side list of all possible cpus. If it doesn't match
RTLA's nr_cpus (read from sysfs), abort with an error.
Assisted-by: Claude:claude-opus-4-6
Link: https://lore.kernel.org/r/20260814135511.2207190-4-tglozar@redhat.com
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
|
|
The "cpu" field of trace entries processed by rtla in tracefs mode is
used as an index into an array of size nr_cpus. In case the cpu numbers
mismatch because either the kernel or rtla reporting the number
incorrectly, out-of-bounds read/write may occur.
Guard against this by dropping trace entries with cpu >= nr_cpus in
collect_registered_events(). A new counter, "invalid_events", is added
to struct trace_instance, and printed (if non zero) next to the
pre-existing missed events counter.
Link: https://lore.kernel.org/r/20260814135511.2207190-3-tglozar@redhat.com
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
|
|
Add a new option, -A/--aligned, that enables timerlat thread alignment
implemented on the kernel-side in commit 4245bf4dc58f ("tracing/osnoise:
Add option to align tlat threads"). The option takes an argument,
representing alignment between timerlat threads in microseconds.
The feature is modeled after the option of the same name in the
cyclictest tool.
Link: https://lore.kernel.org/r/20260527144928.2944472-1-tglozar@redhat.com
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
|
|
Instead of using getopt_long() directly to parse the command line
arguments given to an RTLA tool, use libsubcmd's parse_options().
Utilizing libsubcmd for parsing command line arguments has several
benefits:
- A help message is automatically generated by libsubcmd from the
specification, removing the need of writing it by hand.
- Options are sorted into groups based on which part of tracing (CPU,
thread, auto-analysis, tuning, histogram) they relate to.
- Common parsing patterns for numerical and boolean values now share
code, with the target variable being stored in the option array.
To avoid duplication of the option parsing logic, RTLA-specific
macros defining struct option values are created:
- RTLA_OPT_* for options common to all tools
- OSNOISE_OPT_* and TIMERLAT_OPT_* for options specific to
osnoise/timerlat tools
- HIST_OPT_* macros for options specific to histogram-based tools.
Individual *_parse_args() functions then construct an array out of
these macros that is then passed to libsubcmd's parse_options().
All code specific to command line options parsing is moved out of the
individual tool files into a new file, cli.c, which also contains the
contents of the rtla.c file. A private header, cli_p.h, is added
alongside the public header cli.h, so that unit tests are able to test
statically declared option callbacks.
Minor changes:
- The return value of tool-level help option changes to 129, as this is
the value set by libsubcmd; this is reflected in affected test cases.
The implementation of help for command-level and tracer-level help
is set to 129 as well for consistency, and the change is reflected in
exit value documentation.
- Related to the above, {rtla,osnoise,timerlat}_usage() are marked
__noreturn and exit() is removed from after they are called for
cleaner code.
- The error messages for invalid argument for options --dma-latency and
-E/--entries were corrected, fixing off-by-one in the limits.
Note that unsetting options (using --no-<opt> syntax) is currently not
implemented for options that use custom callbacks. For --irq and
--thread, it will never be implemented, as they conflict with already
existing --no-irq and --no-thread with a different meaning.
Assisted-by: Composer:composer-1.5
Link: https://lore.kernel.org/r/20260528103254.2990068-5-tglozar@redhat.com
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
|
|
Before, when rtla got a signal, it stopped the main trace but not the
record trace. With "--on-end trace", this can lead to
save_trace_to_file() failing to keep up, especially on a debug kernel.
Plus, it adds post-stoppage noise to the trace file.
Signed-off-by: Crystal Wood <crwood@redhat.com>
Fixes: c73cab9dbed0 ("rtla/timerlat_hist: Stop timerlat tracer on signal")
Fixes: a4dfce7559d7 ("rtla/timerlat_top: Stop timerlat tracer on signal")
Fixes: 3aadb65db5d6 ("rtla/timerlat: Add action on end feature")
Link: https://lore.kernel.org/r/20260512173731.2151841-1-crwood@redhat.com
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
|
|
Several functions duplicate the logic for handling threshold actions.
When a threshold is reached, these functions stop the trace, perform
configured actions, and restart the trace if --on-threshold continue
is set.
Create common_threshold_handler() to centralize this shared logic and
avoid code duplication. The function executes the configured threshold
actions and restarts the necessary trace instances when appropriate.
Also add should_continue_tracing() helper to encapsulate the check
for whether tracing should continue after a threshold event, improving
code readability at call sites.
In timerlat_top_bpf_main_loop(), use common_params directly instead
of casting through timerlat_params when only common fields are needed.
Signed-off-by: Wander Lairson Costa <wander@redhat.com>
Link: https://lore.kernel.org/r/20260309195040.1019085-5-wander@redhat.com
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
|
|
nr_cpus does not change at runtime, so passing it through the macro
argument is unnecessary.
Remove the argument and use the global nr_cpus instead.
Signed-off-by: Costa Shulyupin <costa.shul@redhat.com>
Link: https://lore.kernel.org/r/20260306194953.2511960-5-costa.shul@redhat.com
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
|
|
sysconf(_SC_NPROCESSORS_CONF) (via get_nprocs_conf) reflects
cpu_possible_mask, which is fixed at boot time, so querying it
repeatedly is unnecessary.
Replace multiple calls to sysconf(_SC_NPROCESSORS_CONF) with a single
global nr_cpus variable initialized once at startup.
`#pragma once` in timerlat_u.h is needed for pre-C23 compilers to avoid
redefinition errors.
Signed-off-by: Costa Shulyupin <costa.shul@redhat.com>
Link: https://lore.kernel.org/r/20260306194953.2511960-2-costa.shul@redhat.com
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
|
|
getopt_long() processes short and long options independently.
RTLA, like the majority of applications, uses both short and long
variants for each logical option.
Since the val member of struct option holds the letter of the short
variant, the string of short options can be reconstructed from the
array of long options.
Add getopt_auto() to generate optstring from an array of long options,
eliminating the need to maintain separate short option strings.
Signed-off-by: Costa Shulyupin <costa.shul@redhat.com>
Reviewed-by: Wander Lairson Costa <wander@redhat.com>
Link: https://lore.kernel.org/r/20260108095011.2115719-1-costa.shul@redhat.com
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
|
|
The stop_tracing global variable is accessed from both the signal
handler context and the main program flow without synchronization.
This creates a potential race condition where compiler optimizations
could cache the variable value in registers, preventing the signal
handler's updates from being visible to other parts of the program.
Add the volatile qualifier to stop_tracing in both common.c and
common.h to ensure all accesses to this variable bypass compiler
optimizations and read directly from memory. This guarantees that
when the signal handler sets stop_tracing, the change is immediately
visible to the main program loop, preventing potential hangs or
delayed shutdown when termination signals are received.
Signed-off-by: Wander Lairson Costa <wander@redhat.com>
Link: https://lore.kernel.org/r/20260106133655.249887-16-wander@redhat.com
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
|
|
Each rtla tool duplicates parsing of many common options. This creates
maintenance overhead and risks inconsistencies when updating these
options.
Add common_parse_options() to centralize parsing of options used across
all tools.
Common options to be migrated in future patches.
Changes since v1:
- restore opterr
Signed-off-by: Costa Shulyupin <costa.shul@redhat.com>
Link: https://lore.kernel.org/r/20251209100047.2692515-1-costa.shul@redhat.com
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
|
|
The rtla tools have significant code quadruplication in their usage
functions. Each tool implements its own version of the same help text
formatting and option descriptions, leading to maintenance overhead and
inconsistencies. Documentation/tools/rtla/common_options.rst lists 14
common options.
Add common_usage() infrastructure to consolidate help formatting.
Subsequent patches will extend this to handle other common options.
The refactored output is almost identical to the original, with the
following changes:
- add square brackets to specify optionality: `usage: [rtla] ...`
- remove `-q` from timerlat hist because hist tools don't support it
- minor spacing
Signed-off-by: Costa Shulyupin <costa.shul@redhat.com>
Link: https://lore.kernel.org/r/20251124063204.845425-1-costa.shul@redhat.com
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
|
|
This avoids startup races where one of the instances hit a threshold
before all instances were enabled, and thus tracing stops without
the relevant event. In particular, this is not uncommon with the
tests that set a very tight threshold and then complain if there's
no analysis.
This also ensures that we don't stop tracing during a warmup.
The downside is a small chance of having an event over the threshold
early in the output, without stopping on it, which could cause user
confusion. This should be less likely if the warmup feature is used, but
that doesn't eliminate the race window, just the odds of an unusual spike
right at that moment.
Signed-off-by: Crystal Wood <crwood@redhat.com>
Link: https://lore.kernel.org/r/20251112152529.956778-6-crwood@redhat.com
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
|
|
The rtla tools have many instances of iterating over CPUs while
checking if they are monitored.
Add a for_each_monitored_cpu() helper macro to make the code
more readable and reduce code duplication.
Signed-off-by: Costa Shulyupin <costa.shul@redhat.com>
Reviewed-by: Tomas Glozar <tglozar@redhat.com>
Link: https://lore.kernel.org/r/20251002123553.389467-1-costa.shul@redhat.com
Signed-off-by: Tomas Glozar <tglozar@redhat.com>
|
|
Currently a lot of code is duplicated between the different rtla tools,
making maintenance more difficult, and encouraging divergence such as
features that are only implemented for certain tools even though they
could be more broadly applicable.
Merge the various main() functions into a common run_tool() with an ops
struct for tool-specific details.
Implement enough support for actions on osnoise to not need to keep the
old params->trace_output path.
Cc: John Kacur <jkacur@redhat.com>
Cc: Costa Shulyupin <costa.shul@redhat.com>
Link: https://lore.kernel.org/20250907022325.243930-5-crwood@redhat.com
Reviewed-by: Tomas Glozar <tglozar@redhat.com>
Signed-off-by: Crystal Wood <crwood@redhat.com>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
|
|
Merge the common bits of osnoise_apply_config() and
timerlat_apply_config(). Put the result in a new common.c, and move
enough things to common.h so that common.c does not need to include
osnoise.h.
Cc: John Kacur <jkacur@redhat.com>
Cc: Costa Shulyupin <costa.shul@redhat.com>
Link: https://lore.kernel.org/20250907022325.243930-4-crwood@redhat.com
Reviewed-by: Tomas Glozar <tglozar@redhat.com>
Signed-off-by: Crystal Wood <crwood@redhat.com>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
|
|
The hist members were very similar between timerlat and top, so
just use one common hist struct.
output_divisor, quiet, and pretty printing are pretty generic
concepts that can go in the main struct even if not every
specific tool (currently) uses them.
Cc: John Kacur <jkacur@redhat.com>
Cc: Costa Shulyupin <costa.shul@redhat.com>
Link: https://lore.kernel.org/20250907022325.243930-3-crwood@redhat.com
Reviewed-by: Tomas Glozar <tglozar@redhat.com>
Signed-off-by: Crystal Wood <crwood@redhat.com>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
|
|
timerlat_params and osnoise_params structures contain 15 identical
fields.
Introduce a new header common.h and define a common_params structure to
consolidate shared fields, reduce code duplication, and enhance
maintainability.
Cc: John Kacur <jkacur@redhat.com>
Link: https://lore.kernel.org/20250907022325.243930-2-crwood@redhat.com
Reviewed-by: Tomas Glozar <tglozar@redhat.com>
Signed-off-by: Costa Shulyupin <costa.shul@redhat.com>
Signed-off-by: Crystal Wood <crwood@redhat.com>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>
|
