diff options
Diffstat (limited to 'tools/perf/Documentation')
33 files changed, 1277 insertions, 211 deletions
diff --git a/tools/perf/Documentation/Build.txt b/tools/perf/Documentation/Build.txt index 83dc87c662b6..57b226e7fc2f 100644 --- a/tools/perf/Documentation/Build.txt +++ b/tools/perf/Documentation/Build.txt @@ -99,3 +99,18 @@ configuration paths for cross building: In this case, the variable PKG_CONFIG_SYSROOT_DIR can be used alongside the variable PKG_CONFIG_LIBDIR or PKG_CONFIG_PATH to prepend the sysroot path to the library paths for cross compilation. + +5) Build with Clang +=================== +By default, the makefile uses GCC as compiler. With specifying environment +variables HOSTCC, CC and CXX, it allows to build perf with Clang. + +Using Clang for a native build: + + $ HOSTCC=clang CC=clang CXX=clang++ make -C tools/perf + +Specifying ARCH and CROSS_COMPILE for cross compilation: + + $ HOSTCC=clang CC=clang CXX=clang++ \ + ARCH=arm64 CROSS_COMPILE=aarch64-linux-gnu- \ + make -C tools/perf diff --git a/tools/perf/Documentation/android.txt b/tools/perf/Documentation/android.txt index 24a59998fc91..3f3cc7ac3d13 100644 --- a/tools/perf/Documentation/android.txt +++ b/tools/perf/Documentation/android.txt @@ -1,78 +1,10 @@ How to compile perf for Android -========================================= +=============================== -I. Set the Android NDK environment ------------------------------------------------- +There are two ways to build perf and run it on Android: -(a). Use the Android NDK ------------------------------------------------- -1. You need to download and install the Android Native Development Kit (NDK). -Set the NDK variable to point to the path where you installed the NDK: - export NDK=/path/to/android-ndk +- Method 1: Build perf with static linking. See Build.txt, section + "4) Cross compilation" for how to build a static perf binary. -2. Set cross-compiling environment variables for NDK toolchain and sysroot. -For arm: - export NDK_TOOLCHAIN=${NDK}/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi- - export NDK_SYSROOT=${NDK}/platforms/android-24/arch-arm -For x86: - export NDK_TOOLCHAIN=${NDK}/toolchains/x86-4.9/prebuilt/linux-x86_64/bin/i686-linux-android- - export NDK_SYSROOT=${NDK}/platforms/android-24/arch-x86 - -This method is only tested for Android NDK versions Revision 11b and later. -perf uses some bionic enhancements that are not included in prior NDK versions. -You can use method (b) described below instead. - -(b). Use the Android source tree ------------------------------------------------ -1. Download the master branch of the Android source tree. -Set the environment for the target you want using: - source build/envsetup.sh - lunch - -2. Build your own NDK sysroot to contain latest bionic changes and set the -NDK sysroot environment variable. - cd ${ANDROID_BUILD_TOP}/ndk -For arm: - ./build/tools/build-ndk-sysroot.sh --abi=arm - export NDK_SYSROOT=${ANDROID_BUILD_TOP}/ndk/build/platforms/android-3/arch-arm -For x86: - ./build/tools/build-ndk-sysroot.sh --abi=x86 - export NDK_SYSROOT=${ANDROID_BUILD_TOP}/ndk/build/platforms/android-3/arch-x86 - -3. Set the NDK toolchain environment variable. -For arm: - export NDK_TOOLCHAIN=${ANDROID_TOOLCHAIN}/arm-linux-androideabi- -For x86: - export NDK_TOOLCHAIN=${ANDROID_TOOLCHAIN}/i686-linux-android- - -II. Compile perf for Android ------------------------------------------------- -You need to run make with the NDK toolchain and sysroot defined above: -For arm: - make WERROR=0 ARCH=arm CROSS_COMPILE=${NDK_TOOLCHAIN} EXTRA_CFLAGS="-pie --sysroot=${NDK_SYSROOT}" -For x86: - make WERROR=0 ARCH=x86 CROSS_COMPILE=${NDK_TOOLCHAIN} EXTRA_CFLAGS="-pie --sysroot=${NDK_SYSROOT}" - -III. Install perf ------------------------------------------------ -You need to connect to your Android device/emulator using adb. -Install perf using: - adb push perf /data/perf - -If you also want to use perf-archive you need busybox tools for Android. -For installing perf-archive, you first need to replace #!/bin/bash with #!/system/bin/sh: - sed 's/#!\/bin\/bash/#!\/system\/bin\/sh/g' perf-archive >> /tmp/perf-archive - chmod +x /tmp/perf-archive - adb push /tmp/perf-archive /data/perf-archive - -IV. Environment settings for running perf ------------------------------------------------- -Some perf features need environment variables to run properly. -You need to set these before running perf on the target: - adb shell - # PERF_PAGER=cat - -IV. Run perf ------------------------------------------------- -Run perf on your device/emulator to which you previously connected using adb: - # ./data/perf +- Method 2: Download the Android NDK and use the bundled Clang to + build perf. See Build.txt, section "5) Build with clang" for details. diff --git a/tools/perf/Documentation/callchain-overhead-calculation.txt b/tools/perf/Documentation/callchain-overhead-calculation.txt index 1a757927195e..e0202bf5bd1a 100644 --- a/tools/perf/Documentation/callchain-overhead-calculation.txt +++ b/tools/perf/Documentation/callchain-overhead-calculation.txt @@ -1,7 +1,8 @@ Overhead calculation -------------------- -The overhead can be shown in two columns as 'Children' and 'Self' when -perf collects callchains. The 'self' overhead is simply calculated by +The CPU overhead can be shown in two columns as 'Children' and 'Self' +when perf collects callchains (and corresponding 'Wall' columns for +wall-clock overhead). The 'self' overhead is simply calculated by adding all period values of the entry - usually a function (symbol). This is the value that perf shows traditionally and sum of all the 'self' overhead values should be 100%. diff --git a/tools/perf/Documentation/cpu-and-latency-overheads.txt b/tools/perf/Documentation/cpu-and-latency-overheads.txt new file mode 100644 index 000000000000..3b6d63705465 --- /dev/null +++ b/tools/perf/Documentation/cpu-and-latency-overheads.txt @@ -0,0 +1,85 @@ +CPU and latency overheads +------------------------- +There are two notions of time: wall-clock time and CPU time. +For a single-threaded program, or a program running on a single-core machine, +these notions are the same. However, for a multi-threaded/multi-process program +running on a multi-core machine, these notions are significantly different. +Each second of wall-clock time we have number-of-cores seconds of CPU time. +Perf can measure overhead for both of these times (shown in 'overhead' and +'latency' columns for CPU and wall-clock time correspondingly). + +Optimizing CPU overhead is useful to improve 'throughput', while optimizing +latency overhead is useful to improve 'latency'. It's important to understand +which one is useful in a concrete situation at hand. For example, the former +may be useful to improve max throughput of a CI build server that runs on 100% +CPU utilization, while the latter may be useful to improve user-perceived +latency of a single interactive program build. +These overheads may be significantly different in some cases. For example, +consider a program that executes function 'foo' for 9 seconds with 1 thread, +and then executes function 'bar' for 1 second with 128 threads (consumes +128 seconds of CPU time). The CPU overhead is: 'foo' - 6.6%, 'bar' - 93.4%. +While the latency overhead is: 'foo' - 90%, 'bar' - 10%. If we try to optimize +running time of the program looking at the (wrong in this case) CPU overhead, +we would concentrate on the function 'bar', but it can yield only 10% running +time improvement at best. + +By default, perf shows only CPU overhead. To show latency overhead, use +'perf record --latency' and 'perf report': + +----------------------------------- +Overhead Latency Command + 93.88% 25.79% cc1 + 1.90% 39.87% gzip + 0.99% 10.16% dpkg-deb + 0.57% 1.00% as + 0.40% 0.46% sh +----------------------------------- + +To sort by latency overhead, use 'perf report --latency': + +----------------------------------- +Latency Overhead Command + 39.87% 1.90% gzip + 25.79% 93.88% cc1 + 10.16% 0.99% dpkg-deb + 4.17% 0.29% git + 2.81% 0.11% objtool +----------------------------------- + +To get insight into the difference between the overheads, you may check +parallelization histogram with '--sort=latency,parallelism,comm,symbol --hierarchy' +flags. It shows fraction of (wall-clock) time the workload utilizes different +numbers of cores ('Parallelism' column). For example, in the following case +the workload utilizes only 1 core most of the time, but also has some +highly-parallel phases, which explains significant difference between +CPU and wall-clock overheads: + +----------------------------------- + Latency Overhead Parallelism / Command / Symbol ++ 56.98% 2.29% 1 ++ 16.94% 1.36% 2 ++ 4.00% 20.13% 125 ++ 3.66% 18.25% 124 ++ 3.48% 17.66% 126 ++ 3.26% 0.39% 3 ++ 2.61% 12.93% 123 +----------------------------------- + +By expanding corresponding lines, you may see what commands/functions run +at the given parallelism level: + +----------------------------------- + Latency Overhead Parallelism / Command / Symbol +- 56.98% 2.29% 1 + 32.80% 1.32% gzip + 4.46% 0.18% cc1 + 2.81% 0.11% objtool + 2.43% 0.10% dpkg-source + 2.22% 0.09% ld + 2.10% 0.08% dpkg-genchanges +----------------------------------- + +To see the normal function-level profile for particular parallelism levels +(number of threads actively running on CPUs), you may use '--parallelism' +filter. For example, to see the profile only for low parallelism phases +of a workload use '--latency --parallelism=1-2' flags. diff --git a/tools/perf/Documentation/intel-acr.txt b/tools/perf/Documentation/intel-acr.txt new file mode 100644 index 000000000000..72654fdd9a52 --- /dev/null +++ b/tools/perf/Documentation/intel-acr.txt @@ -0,0 +1,53 @@ +Intel Auto Counter Reload Support +--------------------------------- +Support for Intel Auto Counter Reload in perf tools + +Auto counter reload provides a means for software to specify to hardware +that certain counters, if supported, should be automatically reloaded +upon overflow of chosen counters. By taking a sample only if the rate of +one event exceeds some threshold relative to the rate of another event, +this feature enables software to sample based on the relative rate of +two or more events. To enable this, the user must provide a sample period +term and a bitmask ("acr_mask") for each relevant event specifying the +counters in an event group to reload if the event's specified sample +period is exceeded. + +For example, if the user desires to measure a scenario when IPC > 2, +the event group might look like the one below: + + perf record -e {cpu_atom/instructions,period=200000,acr_mask=0x2/, \ + cpu_atom/cycles,period=100000,acr_mask=0x3/} -- true + +In this case, if the "instructions" counter exceeds the sample period of +200000, the second counter, "cycles", will be reset and a sample will be +taken. If "cycles" is exceeded first, both counters in the group will be +reset. In this way, samples will only be taken for cases where IPC > 2. + +The acr_mask term is a hexadecimal value representing a bitmask of the +events in the group to be reset when the period is exceeded. In the +example above, "instructions" is assigned an acr_mask of 0x2, meaning +only the second event in the group is reloaded and a sample is taken +for the first event. "cycles" is assigned an acr_mask of 0x3, meaning +that both event counters will be reset if the sample period is exceeded +first. + +ratio-to-prev Event Term +------------------------ +To simplify this, an event term "ratio-to-prev" is provided which is used +alongside the sample period term n or the -c/--count option. This would +allow users to specify the desired relative rate between events as a +ratio. Note: Both events compared must belong to the same PMU. + +The command above would then become + + perf record -e {cpu_atom/instructions/, \ + cpu_atom/cycles,period=100000,ratio-to-prev=0.5/} -- true + +ratio-to-prev is the ratio of the event using the term relative +to the previous event in the group, which will always be 1, +for a 1:0.5 or 2:1 ratio. + +To sample for IPC < 2 for example, the events need to be reordered: + + perf record -e {cpu_atom/cycles/, \ + cpu_atom/instructions,period=200000,ratio-to-prev=2.0/} -- true diff --git a/tools/perf/Documentation/intel-hybrid.txt b/tools/perf/Documentation/intel-hybrid.txt index e7a776ad25d7..0379903673a4 100644 --- a/tools/perf/Documentation/intel-hybrid.txt +++ b/tools/perf/Documentation/intel-hybrid.txt @@ -8,15 +8,15 @@ Part of events are available on core cpu, part of events are available on atom cpu and even part of events are available on both. Kernel exports two new cpu pmus via sysfs: -/sys/devices/cpu_core -/sys/devices/cpu_atom +/sys/bus/event_source/devices/cpu_core +/sys/bus/event_source/devices/cpu_atom The 'cpus' files are created under the directories. For example, -cat /sys/devices/cpu_core/cpus +cat /sys/bus/event_source/devices/cpu_core/cpus 0-15 -cat /sys/devices/cpu_atom/cpus +cat /sys/bus/event_source/devices/cpu_atom/cpus 16-23 It indicates cpu0-cpu15 are core cpus and cpu16-cpu23 are atom cpus. @@ -60,8 +60,8 @@ can't carry pmu information. So now this type is extended to be PMU aware type. The PMU type ID is stored at attr.config[63:32]. PMU type ID is retrieved from sysfs. -/sys/devices/cpu_atom/type -/sys/devices/cpu_core/type +/sys/bus/event_source/devices/cpu_atom/type +/sys/bus/event_source/devices/cpu_core/type The new attr.config layout for PERF_TYPE_HARDWARE: diff --git a/tools/perf/Documentation/perf-amd-ibs.txt b/tools/perf/Documentation/perf-amd-ibs.txt index 2fd31d9d7b71..253a7375c88a 100644 --- a/tools/perf/Documentation/perf-amd-ibs.txt +++ b/tools/perf/Documentation/perf-amd-ibs.txt @@ -69,6 +69,14 @@ Per-cpu profile (cpu10), cycles event, sampling period: 100000 # perf record -e ibs_op// -c 100000 -C 10 +Userspace only, per-cpu profile (cpu10), cycles event, sampling period: 100000 + + Zen6 onward (See NOTES): + # perf record -e ibs_op//u -c 100000 -C 10 + + Until Zen5: + # perf record -e ibs_op/swfilt=1/u -c 100000 -C 10 + Per-cpu profile (cpu10), cycles event, sampling freq: 1000 # perf record -e ibs_op// -F 1000 -C 10 @@ -85,6 +93,20 @@ System-wide profile, uOps event, sampling period: 100000, L3MissOnly (Zen4 onwar # perf record -e ibs_op/cnt_ctl=1,l3missonly=1/ -c 100000 -a +System-wide profile, cycles event, sampling period: 100000, LdLat filtering (Zen5 +onward) + + # perf record -e ibs_op/ldlat=128/ -c 100000 -a + + Supported load latency threshold values are 128 to 2048 (both inclusive). + Latency value which is a multiple of 128 incurs a little less profiling + overhead compared to other values. + +System-wide profile, cycles event, sampling period: 100000, streaming store +filter (Zen6 onward) + + # perf record -e ibs_op/strmst=1/ -c 100000 -a + Per process(upstream v6.2 onward), uOps event, sampling period: 100000 # perf record -e ibs_op/cnt_ctl=1/ -c 100000 -p 1234 @@ -141,6 +163,14 @@ System-wide profile, fetch ops event, sampling period: 100000 # perf record -e ibs_fetch// -c 100000 -a +Userspace only, system-wide profile, fetch ops event, sampling period: 100000 + + Zen6 onward (See NOTES): + # perf record -e ibs_fetch//u -c 100000 -a + + Until Zen5: + # perf record -e ibs_fetch/swfilt=1/u -c 100000 -a + System-wide profile, fetch ops event, sampling period: 100000, Random enable # perf record -e ibs_fetch/rand_en=1/ -c 100000 -a @@ -149,6 +179,15 @@ System-wide profile, fetch ops event, sampling period: 100000, Random enable helps in cases like long running loops where PMU is tagging the same instruction over and over because of fixed sample period. +System-wide profile, fetch ops event, sampling period: 10000, fetch latency +filter (Zen6 onward) + + # perf record -e ibs_fetch/fetchlat=128/ -c 10000 -a + + Supported fetch latency threshold values are 128 to 1920 (both inclusive). + Latency value which is a multiple of 128 incurs a little less profiling + overhead compared to other values. + etc. PERF MEM AND PERF C2C @@ -162,26 +201,60 @@ Below is a simple example of the perf mem tool. # perf mem report A normal perf mem report output will provide detailed memory access profile. -However, it can also be aggregated based on output fields. For example: - - # perf mem report -F mem,sample,snoop - Samples: 3M of event 'ibs_op//', Event count (approx.): 23524876 - Memory access Samples Snoop - N/A 1903343 N/A - L1 hit 1056754 N/A - L2 hit 75231 N/A - L3 hit 9496 HitM - L3 hit 2270 N/A - RAM hit 8710 N/A - Remote node, same socket RAM hit 3241 N/A - Remote core, same node Any cache hit 1572 HitM - Remote core, same node Any cache hit 514 N/A - Remote node, same socket Any cache hit 1216 HitM - Remote node, same socket Any cache hit 350 N/A - Uncached hit 18 N/A +New output fields will show related access info together. For example: + + # perf mem report -F overhead,cache,snoop,comm + ... + # Samples: 92K of event 'ibs_op//' + # Total weight : 531104 + # + # ---------- Cache ----------- --- Snoop ---- + # Overhead L1 L2 L1-buf Other HitM Other Command + # ........ ............................ .............. .......... + # + 76.07% 5.8% 35.7% 0.0% 34.6% 23.3% 52.8% cc1 + 5.79% 0.2% 0.0% 0.0% 5.6% 0.1% 5.7% make + 5.78% 0.1% 4.4% 0.0% 1.2% 0.5% 5.3% gcc + 5.33% 0.3% 3.9% 0.0% 1.1% 0.2% 5.2% as + 5.00% 0.1% 3.8% 0.0% 1.0% 0.3% 4.7% sh + 1.56% 0.1% 0.1% 0.0% 1.4% 0.6% 0.9% ld + 0.28% 0.1% 0.0% 0.0% 0.2% 0.1% 0.2% pkg-config + 0.09% 0.0% 0.0% 0.0% 0.1% 0.0% 0.1% git + 0.03% 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% rm + ... + +Also, it can be aggregated based on various memory access info using the +sort keys. For example: + + # perf mem report -s mem,snoop + ... + # Samples: 92K of event 'ibs_op//' + # Total weight : 531104 + # Sort order : mem,snoop + # + # Overhead Samples Memory access Snoop + # ........ ............ ....................................... ............ + # + 47.99% 1509 L2 hit N/A + 25.08% 338 core, same node Any cache hit HitM + 10.24% 54374 N/A N/A + 6.77% 35938 L1 hit N/A + 6.39% 101 core, same node Any cache hit N/A + 3.50% 69 RAM hit N/A + 0.03% 158 LFB/MAB hit N/A + 0.00% 2 Uncached hit N/A Please refer to their man page for more detail. +NOTES +----- +Hardware privilege filtering uses bit 63 to distinguish between kernel +and userspace addresses. Hardware privilege filtering is not supported +on 32-bit systems. Also, the bit 63 convention is not universal and can +fail in specific environments, such as, using 64-bit host IBS to profile +a 32-bit guest, using 64-bit host IBS to profile non-Linux 64-bit guests +that do not adhere to the bit 63 privilege standard etc. + SEE ALSO -------- diff --git a/tools/perf/Documentation/perf-annotate.txt b/tools/perf/Documentation/perf-annotate.txt index 156c5f37b051..a688738809c4 100644 --- a/tools/perf/Documentation/perf-annotate.txt +++ b/tools/perf/Documentation/perf-annotate.txt @@ -110,8 +110,11 @@ include::itrace.txt[] Interleave source code with assembly code. Enabled by default, disable with --no-source. ---symfs=<directory>:: - Look for files with symbols relative to this directory. +--symfs=<directory[,layout]>:: + Look for files with symbols relative to this directory. The optional + layout can be 'hierarchy' (default, matches full path) or 'flat' + (only matches base name). This is useful when debug files are stored + in a flat directory structure. -M:: --disassembler-style=:: Set disassembler style for objdump. @@ -168,6 +171,9 @@ include::itrace.txt[] --skip-empty:: Do not display empty (or dummy) events. +--code-with-type:: + Show data type info in code annotation (for memory instructions only). + SEE ALSO -------- diff --git a/tools/perf/Documentation/perf-arm-spe.txt b/tools/perf/Documentation/perf-arm-spe.txt index 37afade4f1b2..201a82bec0de 100644 --- a/tools/perf/Documentation/perf-arm-spe.txt +++ b/tools/perf/Documentation/perf-arm-spe.txt @@ -141,27 +141,64 @@ Config parameters These are placed between the // in the event and comma separated. For example '-e arm_spe/load_filter=1,min_latency=10/' - branch_filter=1 - collect branches only (PMSFCR.B) - event_filter=<mask> - filter on specific events (PMSEVFR) - see bitfield description below + event_filter=<mask> - logical AND filter on specific events (PMSEVFR) - see bitfield description below + inv_event_filter=<mask> - logical OR to filter out specific events (PMSNEVFR, FEAT_SPEv1p2) - see bitfield description below jitter=1 - use jitter to avoid resonance when sampling (PMSIRR.RND) - load_filter=1 - collect loads only (PMSFCR.LD) min_latency=<n> - collect only samples with this latency or higher* (PMSLATFR) pa_enable=1 - collect physical address (as well as VA) of loads/stores (PMSCR.PA) - requires privilege pct_enable=1 - collect physical timestamp instead of virtual timestamp (PMSCR.PCT) - requires privilege - store_filter=1 - collect stores only (PMSFCR.ST) ts_enable=1 - enable timestamping with value of generic timer (PMSCR.TS) discard=1 - enable SPE PMU events but don't collect sample data - see 'Discard mode' (PMBLIMITR.FM = DISCARD) + inv_data_src_filter=<mask> - mask to filter from 0-63 possible data sources (PMSDSFR, FEAT_SPE_FDS) - See 'Data source filtering' +++*+++ Latency is the total latency from the point at which sampling started on that instruction, rather than only the execution latency. -Only some events can be filtered on; these include: - - bit 1 - instruction retired (i.e. omit speculative instructions) +Only some events can be filtered on using 'event_filter' bits. The overall +filter is the logical AND of these bits, for example if bits 3 and 5 are set +only samples that have both 'L1D cache refill' AND 'TLB walk' are recorded. When +FEAT_SPEv1p2 is implemented 'inv_event_filter' can also be used to exclude +events that have any (OR) of the filter's bits set. For example setting bits 3 +and 5 in 'inv_event_filter' will exclude any events that are either L1D cache +refill OR TLB walk. If the same bit is set in both filters it's UNPREDICTABLE +whether the sample is included or excluded. Filter bits for both event_filter +and inv_event_filter are: + + bit 1 - Instruction retired (i.e. omit speculative instructions) + bit 2 - L1D access (FEAT_SPEv1p4) bit 3 - L1D refill + bit 4 - TLB access (FEAT_SPEv1p4) bit 5 - TLB refill - bit 7 - mispredict - bit 11 - misaligned access + bit 6 - Not taken event (FEAT_SPEv1p2) + bit 7 - Mispredict + bit 8 - Last level cache access (FEAT_SPEv1p4) + bit 9 - Last level cache miss (FEAT_SPEv1p4) + bit 10 - Remote access (FEAT_SPEv1p4) + bit 11 - Misaligned access (FEAT_SPEv1p1) + bit 12-15 - IMPLEMENTATION DEFINED events (when implemented) + bit 17 - Partial or empty SME or SVE predicate (FEAT_SPEv1p1) + bit 18 - Empty SME or SVE predicate (FEAT_SPEv1p1) + bit 19 - L2D access (FEAT_SPEv1p4) + bit 20 - L2D miss (FEAT_SPEv1p4) + bit 21 - Cache data modified (FEAT_SPEv1p4) + bit 22 - Recently fetched (FEAT_SPEv1p4) + bit 23 - Data snooped (FEAT_SPEv1p4) + bit 24 - Streaming SVE mode event (when FEAT_SPE_SME is implemented), or + IMPLEMENTATION DEFINED event 24 (when implemented, only versions + less than FEAT_SPEv1p4) + bit 25 - SMCU or external coprocessor operation event when FEAT_SPE_SME is + implemented, or IMPLEMENTATION DEFINED event 25 (when implemented, + only versions less than FEAT_SPEv1p4) + bit 26-31 - IMPLEMENTATION DEFINED events (only versions less than FEAT_SPEv1p4) + bit 48-63 - IMPLEMENTATION DEFINED events (when implemented) + +For IMPLEMENTATION DEFINED bits, refer to the CPU TRM if these bits are +implemented. + +The driver will reject events if requested filter bits require unimplemented SPE +versions, but will not reject filter bits for unimplemented IMPDEF bits or when +their related feature is not present (e.g. SME). For example, if FEAT_SPEv1p2 is +not implemented, filtering on "Not taken event" (bit 6) will be rejected. So to sample just retired instructions: @@ -171,6 +208,31 @@ or just mispredicted branches: perf record -e arm_spe/event_filter=0x80/ -- ./mybench +When set, the following filters can be used to select samples that match any of +the operation types (OR filtering). If only one is set then only samples of that +type are collected: + + branch_filter=1 - Collect branches (PMSFCR.B) + load_filter=1 - Collect loads (PMSFCR.LD) + store_filter=1 - Collect stores (PMSFCR.ST) + +When extended filtering is supported (FEAT_SPE_EFT), SIMD and float +pointer operations can also be selected: + + simd_filter=1 - Collect SIMD loads, stores and operations (PMSFCR.SIMD) + float_filter=1 - Collect floating point loads, stores and operations (PMSFCR.FP) + +When extended filtering is supported (FEAT_SPE_EFT), operation type filters can +be changed to AND using _mask fields. For example samples could be selected if +they are store AND SIMD by setting 'store_filter=1,simd_filter=1, +store_filter_mask=1,simd_filter_mask=1'. The new masks are as follows: + + branch_filter_mask=1 - Change branch filter behavior from OR to AND (PMSFCR.Bm) + load_filter_mask=1 - Change load filter behavior from OR to AND (PMSFCR.LDm) + store_filter_mask=1 - Change store filter behavior from OR to AND (PMSFCR.STm) + simd_filter_mask=1 - Change SIMD filter behavior from OR to AND (PMSFCR.SIMDm) + float_filter_mask=1 - Change floating point filter behavior from OR to AND (PMSFCR.FPm) + Viewing the data ~~~~~~~~~~~~~~~~~ @@ -191,19 +253,29 @@ groups: 36 branch 0 remote-access 900 memory + 1800 instructions The arm_spe// and dummy:u events are implementation details and are expected to be empty. -To get a full list of unique samples that are not sorted into groups, set the itrace option to -generate 'instruction' samples. The period option is also taken into account, so set it to 1 -instruction unless you want to further downsample the already sampled SPE data: +The instructions group contains the full list of unique samples that are not +sorted into other groups. To generate only this group use --itrace=i1i. - perf report --itrace=i1i +1i (1 instruction interval) signifies no further downsampling. Rather than an +instruction interval, this generates a sample every n SPE samples. For example +to generate the default set of events for every 100 SPE samples: + + perf report --itrace==bxofmtMai100i + +Other period types, for example nanoseconds (ns) are not currently supported. Memory access details are also stored on the samples and this can be viewed with: perf report --mem-mode +The latency value from the SPE sample is stored in the 'weight' field of the +Perf samples and can be displayed in Perf script and report outputs by enabling +its display from the command line. + Common errors ~~~~~~~~~~~~~ @@ -247,6 +319,25 @@ to minimize output. Then run perf stat: perf record -e arm_spe/discard/ -a -N -B --no-bpf-event -o - > /dev/null & perf stat -e SAMPLE_FEED_LD +Data source filtering +~~~~~~~~~~~~~~~~~~~~~ + +When FEAT_SPE_FDS is present, 'inv_data_src_filter' can be used as a mask to +filter on a subset (0 - 63) of possible data source IDs. The full range of data +sources is 0 - 65535 although these are unlikely to be used in practice. Data +sources are IMPDEF so refer to the TRM for the mappings. Each bit N of the +filter maps to data source N. The filter is an OR of all the bits, and the value +provided inv_data_src_filter is inverted before writing to PMSDSFR_EL1 so that +set bits exclude that data source and cleared bits include that data source. +Therefore the default value of 0 is equivalent to no filtering (all data sources +included). + +For example, to include only data sources 0 and 3, clear bits 0 and 3 +(0xFFFFFFFFFFFFFFF6) + +When 'inv_data_src_filter' is set to 0xFFFFFFFFFFFFFFFF, any samples with any +data source set are excluded. + SEE ALSO -------- diff --git a/tools/perf/Documentation/perf-bench.txt b/tools/perf/Documentation/perf-bench.txt index 8331bd28b10e..c5913cf59c98 100644 --- a/tools/perf/Documentation/perf-bench.txt +++ b/tools/perf/Documentation/perf-bench.txt @@ -177,11 +177,21 @@ Suite for evaluating performance of simple memory copy in various ways. Options of *memcpy* ^^^^^^^^^^^^^^^^^^^ --l:: +-s:: --size:: Specify size of memory to copy (default: 1MB). Available units are B, KB, MB, GB and TB (case insensitive). +-p:: +--page:: +Specify page-size for mapping memory buffers (default: 4KB). +Available values are 4KB, 2MB, 1GB (case insensitive). + +-k:: +--chunk:: +Specify the chunk-size for each invocation. (default: 0, or full-extent) +Available units are B, KB, MB, GB and TB (case insensitive). + -f:: --function:: Specify function to copy (default: default). @@ -201,11 +211,21 @@ Suite for evaluating performance of simple memory set in various ways. Options of *memset* ^^^^^^^^^^^^^^^^^^^ --l:: +-s:: --size:: Specify size of memory to set (default: 1MB). Available units are B, KB, MB, GB and TB (case insensitive). +-p:: +--page:: +Specify page-size for mapping memory buffers (default: 4KB). +Available values are 4KB, 2MB, 1GB (case insensitive). + +-k:: +--chunk:: +Specify the chunk-size for each invocation. (default: 0, or full-extent) +Available units are B, KB, MB, GB and TB (case insensitive). + -f:: --function:: Specify function to set (default: default). @@ -220,6 +240,44 @@ Repeat memset invocation this number of times. --cycles:: Use perf's cpu-cycles event instead of gettimeofday syscall. +*mmap*:: +Suite for evaluating memory subsystem performance for mmap()'d memory. + +Options of *mmap* +^^^^^^^^^^^^^^^^^ +-s:: +--size:: +Specify size of memory to set (default: 1MB). +Available units are B, KB, MB, GB and TB (case insensitive). + +-p:: +--page:: +Specify page-size for mapping memory buffers (default: 4KB). +Available values are 4KB, 2MB, 1GB (case insensitive). + +-r:: +--randomize:: +Specify seed to randomize page access offset (default: 0, or not randomized). + +-f:: +--function:: +Specify function to set (default: all). +Available functions are 'demand' and 'populate', with the first +demand faulting pages in the region and the second using an eager +mapping. + +-l:: +--nr_loops:: +Repeat mmap() invocation this number of times. + +-c:: +--cycles:: +Use perf's cpu-cycles event instead of gettimeofday syscall. + +-t:: +--threads=<NUM>:: +Create multiple threads to call mmap/munmap concurrently. + SUITES FOR 'numa' ~~~~~~~~~~~~~~~~~ *mem*:: diff --git a/tools/perf/Documentation/perf-c2c.txt b/tools/perf/Documentation/perf-c2c.txt index 856f0dfb8e5a..8775889bc0a3 100644 --- a/tools/perf/Documentation/perf-c2c.txt +++ b/tools/perf/Documentation/perf-c2c.txt @@ -54,8 +54,15 @@ RECORD OPTIONS -l:: --ldlat:: - Configure mem-loads latency. Supported on Intel and Arm64 processors - only. Ignored on other archs. + Configure mem-loads latency. Supported on Intel, Arm64 and some AMD + processors. Ignored on other archs. + + On supported AMD processors: + - /sys/bus/event_source/devices/ibs_op/caps/ldlat file contains '1'. + - Supported latency values are 128 to 2048 (both inclusive). + - Latency value which is a multiple of 128 incurs a little less profiling + overhead compared to other values. + - Load latency filtering is disabled by default. -k:: --all-kernel:: @@ -136,6 +143,13 @@ REPORT OPTIONS feature, which causes cacheline sharing to behave like the cacheline size is doubled. +-M:: +--disassembler-style=:: + Set disassembler style for objdump. + +--objdump=<path>:: + Path to objdump binary. + C2C RECORD ---------- The perf c2c record command setup options related to HITM cacheline analysis @@ -146,20 +160,43 @@ Following perf record options are configured by default: -W,-d,--phys-data,--sample-cpu -Unless specified otherwise with '-e' option, following events are monitored by -default on Intel: - - cpu/mem-loads,ldlat=30/P - cpu/mem-stores/P - -following on AMD: - - ibs_op// - -and following on PowerPC: - - cpu/mem-loads/ - cpu/mem-stores/ +The following table lists the events monitored on different architectures. +Unless specified otherwise with the -e option, the tool will select the +default events. + + +--------+---------------+-----------------+--------------------------------------------------------------------------------+ + | Arch | Configuration | Options | Events | + +--------+---------------+-----------------+--------------------------------------------------------------------------------+ + | Intel | Default | -e ldlat-loads | cpu/mem-loads,ldlat=30/P | + | | | -e ldlat-stores | cpu/mem-stores/P | + | |---------------+-----------------+--------------------------------------------------------------------------------+ + | | Load only | -e ldlat-loads | cpu/mem-loads,ldlat=30/P | + | |---------------+-----------------+--------------------------------------------------------------------------------+ + | | Store only | -e ldlat-stores | cpu/mem-stores/P | + +--------+---------------+-----------------+--------------------------------------------------------------------------------+ + | Intel | Default | -e ldlat-loads | {cpu/mem-loads-aux/,cpu/mem-loads,ldlat=30/}:P | + | with | | -e ldlat-stores | cpu/mem-stores/P | + | AUX |--------------+------------------+--------------------------------------------------------------------------------+ + | | Load only | -e ldlat-loads | {cpu/mem-loads-aux/,cpu/mem-loads,ldlat=30/}:P | + | |---------------+-----------------+--------------------------------------------------------------------------------+ + | | Store only | -e ldlat-stores | cpu/mem-stores/P | + +--------+---------------+-----------------+--------------------------------------------------------------------------------+ + | AMD | Default | -e mem-ldst | ibs_op// (without latency support) | + | | | | ibs_op/ldlat=30/ (with latency support) | + +--------+---------------+-----------------+--------------------------------------------------------------------------------+ + | PowerPC| Default | -e ldlat-loads | cpu/mem-loads/ | + | | | -e ldlat-stores | cpu/mem-stores/ | + | |---------------+-----------------+--------------------------------------------------------------------------------+ + | | Load only | -e ldlat-loads | cpu/mem-loads/ | + | |---------------+-----------------+--------------------------------------------------------------------------------+ + | | Store only | -e ldlat-stores | cpu/mem-stores/ | + +--------+---------------+-----------------+--------------------------------------------------------------------------------+ + | Arm | Default | -e spe-ldst | arm_spe_0/ts_enable=1,pa_enable=1,load_filter=1,store_filter=1,min_latency=30/ | + | SPE |---------------+-----------------+--------------------------------------------------------------------------------+ + | | Load only | -e spe-load | arm_spe_0/ts_enable=1,pa_enable=1,load_filter=1,min_latency=30/ | + | |---------------+-----------------+--------------------------------------------------------------------------------+ + | | Store only | -e spe-store | arm_spe_0/ts_enable=1,pa_enable=1,store_filter=1/ | + +--------+---------------+-----------------+--------------------------------------------------------------------------------+ User can pass any 'perf record' option behind '--' mark, like (to enable callchains and system wide monitoring): @@ -328,6 +365,77 @@ TUI OUTPUT The TUI output provides interactive interface to navigate through cachelines list and to display offset details. +Pressing the 'TAB' key in the cacheline view switches to the function +view. The function view shows a three-level hierarchy of the symbolized +entries retained in the cacheline view, organized around functions rather +than cachelines. Levels 1 and 2 normally show function names, while level 3 +shows cacheline addresses. Lower levels are indented beneath their parents. +Verbose mode also includes code addresses in function rows, and code addresses +remain available in the per-cacheline detail view ('d'). + +The function view requires `iaddr` in the cacheline coalescing fields. If +`--coalesce` omits it, TAB reports that the view is unavailable rather than +attributing already-coalesced samples to an arbitrary function. + + Level 1: the read-side function, sorted by Cycles % (estimated load + cycles: HITM, peer-snoop and other-load cycles) + Level 2: the functions sampled writing the shared lines read by the + level-1 function, sorted by store count. This can be the same + function when it has both read and write samples + Level 3: the specific cachelines shared by the reader/writer pair + +The Cycles % value is the function's share of event-provided load +latency/weight estimates from cacheline-detail entries retained in the +current view. It can include non-HITM and non-peer loads coalesced into +entries that pass the C2C filter, so it is not a pure contention-cycle +percentage. The share is relative to the functions and entries retained +for the current report and is not comparable across recordings or different +`--coalesce` settings. + +The store count on a level-1 row is the number of sampled stores by writers +shown in the function view into the cachelines that function reads, including +stores from the same function. It decomposes into the level-2 writer rows; +each level-2 count in turn decomposes into that writer's stores on its level-3 +cachelines. A level-3 count is therefore not the cacheline's total store +count. The level-1 value is not the number of stores made by the reader and +is not additive across level-1 rows: two functions reading the same line each +carry the stores into that line. + +Each function aggregates all of its code addresses into a single entry, +and a level-2 writer aggregates all of its shared cachelines, so a +reader/writer pair is a single row with its total shown -- there is no +need to sum a writer's traffic across cachelines by hand. + +In the function view the 'd' key opens the detail view of the selected +level-3 cacheline, 'e'/'+' expands or collapses the current entry, and 'TAB', +'ESC', 'q' or Ctrl-C returns to the cacheline view. + +For example, with the first two read-side functions collapsed and +dequeue_pushable_task expanded to show the functions writing the lines it +reads -- two of which are further expanded to their individual cachelines: + + Shared Data Functions Table (19 entries, sorted on Cycles %) + Cycles Store + % count Function / Contending function / Cacheline + ---------------------------------------------------------------------- + + 35.67% 876 + [k] cpupri_set + + 24.31% 424 + [k] pull_rt_task + - 16.53% 555 - [k] dequeue_pushable_task + 145 - [k] pull_rt_task + 145 0xff2d0082809da080 + 139 - [k] enqueue_pushable_task + 70 0xff2d00a2071f9640 + 69 0xff2d0082809da000 + +Here dequeue_pushable_task pays 16.53% of the estimated read-side load-cycle +cost. Its store count decomposes into its level-2 writers, and each writer's +count decomposes into its level-3 cachelines: pull_rt_task's 145 stores fall +on a single line, while enqueue_pushable_task's 139 stores split across two +lines (70 and 69). A writer can be the same function as the reader when it +has both read and write samples; after cacheline coalescing and +function-level grouping, the view cannot distinguish same-thread accesses +from different threads running the same function. + For details please refer to the help window by pressing '?' key. CREDITS diff --git a/tools/perf/Documentation/perf-check.txt b/tools/perf/Documentation/perf-check.txt index a764a4629220..09e1d35677f5 100644 --- a/tools/perf/Documentation/perf-check.txt +++ b/tools/perf/Documentation/perf-check.txt @@ -50,12 +50,12 @@ feature:: dwarf / HAVE_LIBDW_SUPPORT dwarf_getlocations / HAVE_LIBDW_SUPPORT dwarf-unwind / HAVE_DWARF_UNWIND_SUPPORT - auxtrace / HAVE_AUXTRACE_SUPPORT libbfd / HAVE_LIBBFD_SUPPORT + libbpf-strings / HAVE_LIBBPF_STRINGS_SUPPORT libcapstone / HAVE_LIBCAPSTONE_SUPPORT - libcrypto / HAVE_LIBCRYPTO_SUPPORT libdw-dwarf-unwind / HAVE_LIBDW_SUPPORT libelf / HAVE_LIBELF_SUPPORT + libLLVM / HAVE_LIBLLVM_SUPPORT libnuma / HAVE_LIBNUMA_SUPPORT libopencsd / HAVE_CSTRACE_SUPPORT libperl / HAVE_LIBPERL_SUPPORT diff --git a/tools/perf/Documentation/perf-config.txt b/tools/perf/Documentation/perf-config.txt index 36ebebc875ea..9b223f892829 100644 --- a/tools/perf/Documentation/perf-config.txt +++ b/tools/perf/Documentation/perf-config.txt @@ -210,6 +210,12 @@ core.*:: Sets a timeout (in milliseconds) for parsing /proc/<pid>/maps files. Can be overridden by the --proc-map-timeout option on supported subcommands. The default timeout is 500ms. + addr2line-disable-warn:: + When set to 'true' disable all warnings from 'addr2line' output. + Default setting is 'false' to show these warnings. + addr2line-timeout:: + Sets a timeout (in milliseconds) for parsing 'addr2line' + output. The default timeout is 5s. tui.*, gtk.*:: Subcommands that can be configured here are 'top', 'report' and 'annotate'. @@ -452,6 +458,9 @@ call-graph.*:: kernel space is controlled not by this option but by the kernel config (CONFIG_UNWINDER_*). + The 'defer' mode can be used with 'fp' mode to enable deferred + user callchains (like 'fp,defer'). + call-graph.dump-size:: The size of stack to dump in order to do post-unwinding. Default is 8192 (byte). When using dwarf into record-mode, the default size will be used if omitted. @@ -708,6 +717,10 @@ intel-pt.*:: the maximum is exceeded there will be a "Never-ending loop" error. The default is 100000. + intel-pt.all-switch-events:: + If the user has permission to do so, always record all context + switch events on all CPUs. + auxtrace.*:: auxtrace.dumpdir:: diff --git a/tools/perf/Documentation/perf-data.txt b/tools/perf/Documentation/perf-data.txt index 417bf17e265c..20f178d61ed7 100644 --- a/tools/perf/Documentation/perf-data.txt +++ b/tools/perf/Documentation/perf-data.txt @@ -40,6 +40,34 @@ OPTIONS for 'convert' --force:: Don't complain, do it. +--time:: + Only convert samples within given time window: <start>,<stop>. Times + have the format seconds.nanoseconds. If start is not given (i.e. time + string is ',x.y') then analysis starts at the beginning of the file. If + stop time is not given (i.e. time string is 'x.y,') then analysis goes + to end of file. Multiple ranges can be separated by spaces, which + requires the argument to be quoted e.g. --time "1234.567,1234.789 1235," + + Also support time percent with multiple time ranges. Time string is + 'a%/n,b%/m,...' or 'a%-b%,c%-%d,...'. + + For example: + Select the second 10% time slice: + + perf data convert --to-json out.json --time 10%/2 + + Select from 0% to 10% time slice: + + perf data convert --to-json out.json --time 0%-10% + + Select the first and second 10% time slices: + + perf data convert --to-json out.json --time 10%/1,10%/2 + + Select from 0% to 10% and 30% to 40% slices: + + perf data convert --to-json out.json --time 0%-10%,30%-40% + -v:: --verbose:: Be more verbose (show counter open errors, etc). diff --git a/tools/perf/Documentation/perf-diff.txt b/tools/perf/Documentation/perf-diff.txt index f3067a4af294..8e4a3f093135 100644 --- a/tools/perf/Documentation/perf-diff.txt +++ b/tools/perf/Documentation/perf-diff.txt @@ -81,8 +81,11 @@ OPTIONS --force:: Don't do ownership validation. ---symfs=<directory>:: - Look for files with symbols relative to this directory. +--symfs=<directory[,layout]>:: + Look for files with symbols relative to this directory. The optional + layout can be 'hierarchy' (default, matches full path) or 'flat' + (only matches base name). This is useful when debug files are stored + in a flat directory structure. -b:: --baseline-only:: @@ -285,7 +288,7 @@ If specified the 'Weighted diff' column is displayed with value 'd' computed as: - period being the hist entry period value - - WEIGHT-A/WEIGHT-B being user supplied weights in the the '-c' option + - WEIGHT-A/WEIGHT-B being user supplied weights in the '-c' option behind ':' separator like '-c wdiff:1,2'. - WEIGHT-A being the weight of the data file - WEIGHT-B being the weight of the baseline data file diff --git a/tools/perf/Documentation/perf-ftrace.txt b/tools/perf/Documentation/perf-ftrace.txt index b77f58c4d2fd..3f3808e513fe 100644 --- a/tools/perf/Documentation/perf-ftrace.txt +++ b/tools/perf/Documentation/perf-ftrace.txt @@ -123,6 +123,10 @@ OPTIONS for 'perf ftrace trace' --graph-opts:: List of options allowed to set: + - args - Show function arguments. + - retval - Show function return value. + - retval-hex - Show function return value in hexadecimal format. + - retaddr - Show function return address. - nosleep-time - Measure on-CPU time only for function_graph tracer. - noirqs - Ignore functions that happen inside interrupt. - verbose - Show process names, PIDs, timestamps, etc. @@ -139,6 +143,12 @@ OPTIONS for 'perf ftrace latency' Set the function name to get the histogram. Unlike perf ftrace trace, it only allows single function to calculate the histogram. +-e:: +--events=:: + Set the pair of events to get the histogram. The histogram is calculated + by the time difference between the two events from the same thread. This + requires -b/--use-bpf option. + -b:: --use-bpf:: Use BPF to measure function latency instead of using the ftrace (it diff --git a/tools/perf/Documentation/perf-inject.txt b/tools/perf/Documentation/perf-inject.txt index c972032f4ca0..95dfdf39666e 100644 --- a/tools/perf/Documentation/perf-inject.txt +++ b/tools/perf/Documentation/perf-inject.txt @@ -109,6 +109,11 @@ include::itrace.txt[] should be used, and also --buildid-all and --switch-events may be useful. +--convert-callchain:: + Parse DWARF callchains and convert them to usual callchains. This also + discards stack and register data from the samples. This will lose + inlined callchain entries. + :GMEXAMPLECMD: inject :GMEXAMPLESUBCMD: include::guestmount.txt[] diff --git a/tools/perf/Documentation/perf-kwork.txt b/tools/perf/Documentation/perf-kwork.txt index 21e607669d78..5c33a1fb2ffe 100644 --- a/tools/perf/Documentation/perf-kwork.txt +++ b/tools/perf/Documentation/perf-kwork.txt @@ -169,8 +169,11 @@ OPTIONS for 'perf kwork timehist' --max-stack:: Maximum number of functions to display in backtrace, default 5. ---symfs=<directory>:: - Look for files with symbols relative to this directory. +--symfs=<directory[,layout]>:: + Look for files with symbols relative to this directory. The optional + layout can be 'hierarchy' (default, matches full path) or 'flat' + (only matches base name). This is useful when debug files are stored + in a flat directory structure. --time:: Only analyze samples within given time window: <start>,<stop>. Times diff --git a/tools/perf/Documentation/perf-list.txt b/tools/perf/Documentation/perf-list.txt index d0c65fad419a..a4378a0cd914 100644 --- a/tools/perf/Documentation/perf-list.txt +++ b/tools/perf/Documentation/perf-list.txt @@ -27,7 +27,7 @@ Don't print descriptions. -v:: --long-desc:: -Print longer event descriptions. +Print longer event descriptions and all similar PMUs with alphanumeric suffixes. --debug:: Enable debugging output. @@ -73,6 +73,7 @@ counted. The following modifiers exist: e - group or event are exclusive and do not share the PMU b - use BPF aggregration (see perf stat --bpf-counters) R - retire latency value of the event + X - don't regroup the event to match PMUs The 'p' modifier can be used for specifying how precise the instruction address should be. The 'p' modifier can be specified multiple times: @@ -188,7 +189,7 @@ in the CPU vendor specific documentation. The available PMUs and their raw parameters can be listed with - ls /sys/devices/*/format + ls /sys/bus/event_source/devices/*/format For example the raw event "LSD.UOPS" core pmu event above could be specified as @@ -278,17 +279,33 @@ also be supplied. For example: perf stat -C 0 -e 'hv_gpci/dtbp_ptitc,phys_processor_idx=0x2/' ... -EVENT QUALIFIERS: +EVENT QUALIFIERS +---------------- It is also possible to add extra qualifiers to an event: percore: -Sums up the event counts for all hardware threads in a core, e.g.: + Sums up the event counts for all hardware threads in a core, e.g.: + perf stat -e cpu/event=0,umask=0x3,percore=1/ +cpu: - perf stat -e cpu/event=0,umask=0x3,percore=1/ + Specifies a CPU or a range of CPUs to open the event upon. It may + also reference a PMU to copy the CPU mask from. The value may be + repeated to specify opening the event on multiple CPUs. + Example 1: to open the instructions event on CPUs 0 and 2, the + cycles event on CPUs 1 and 2: + perf stat -e instructions/cpu=0,cpu=2/,cycles/cpu=1-2/ -a sleep 1 + + Example 2: to open the data_read uncore event on CPU 0 and the + data_write uncore event on CPU 1: + perf stat -e data_read/cpu=0/,data_write/cpu=1/ -a sleep 1 + + Example 3: to open the software msr/tsc/ event only on the CPUs + matching those from the cpu_core PMU: + perf stat -e msr/tsc,cpu=cpu_core/ -a sleep 1 EVENT GROUPS ------------ @@ -376,6 +393,8 @@ Support raw format: . '--raw-dump [hw|sw|cache|tracepoint|pmu|event_glob]', shows the raw-dump of a certain kind of events. +include::intel-acr.txt[] + SEE ALSO -------- linkperf:perf-stat[1], linkperf:perf-top[1], diff --git a/tools/perf/Documentation/perf-lock.txt b/tools/perf/Documentation/perf-lock.txt index d3793054f7d3..c17b3e318169 100644 --- a/tools/perf/Documentation/perf-lock.txt +++ b/tools/perf/Documentation/perf-lock.txt @@ -179,8 +179,9 @@ CONTENTION OPTIONS -o:: --lock-owner:: - Show lock contention stat by owners. Implies --threads and - requires --use-bpf. + Show lock contention stat by owners. This option can be combined with -t, + which shows owner's per thread lock stats, or -v, which shows owner's + stacktrace. Requires --use-bpf. -Y:: --type-filter=<value>:: @@ -215,6 +216,21 @@ CONTENTION OPTIONS --cgroup-filter=<value>:: Show lock contention only in the given cgroups (comma separated list). +-J:: +--inject-delay=<time@function>:: + Add delays to the given lock. It's added to the contention-end part so + that the (new) owner of the lock will be delayed. But by slowing down + the owner, the waiters will also be delayed as well. This is working + only with -b/--use-bpf. + + The 'time' is specified in nsec but it can have a unit suffix. Available + units are "ms", "us" and "ns". Currently it accepts up to 10ms of delays + for safety reasons. + + Note that it will busy-wait after it gets the lock. Delaying locks can + have significant consequences including potential kernel crashes. Please + use it at your own risk. + SEE ALSO -------- diff --git a/tools/perf/Documentation/perf-mem.txt b/tools/perf/Documentation/perf-mem.txt index 8a1bd9ff0f86..4d164836d094 100644 --- a/tools/perf/Documentation/perf-mem.txt +++ b/tools/perf/Documentation/perf-mem.txt @@ -28,6 +28,8 @@ and kernel support is required. See linkperf:perf-arm-spe[1] for a setup guide. Due to the statistical nature of SPE sampling, not every memory operation will be sampled. +On AMD this use IBS Op PMU to sample load-store operations. + COMMON OPTIONS -------------- -f:: @@ -67,8 +69,15 @@ RECORD OPTIONS Configure all used events to run in user space. --ldlat <n>:: - Specify desired latency for loads event. Supported on Intel and Arm64 - processors only. Ignored on other archs. + Specify desired latency for loads event. Supported on Intel, Arm64 and + some AMD processors. Ignored on other archs. + + On supported AMD processors: + - /sys/bus/event_source/devices/ibs_op/caps/ldlat file contains '1'. + - Supported latency values are 128 to 2048 (both inclusive). + - Latency value which is a multiple of 128 incurs a little less profiling + overhead compared to other values. + - Load latency filtering is disabled by default. REPORT OPTIONS -------------- @@ -110,6 +119,22 @@ REPORT OPTIONS And the default sort keys are changed to local_weight, mem, sym, dso, symbol_daddr, dso_daddr, snoop, tlb, locked, blocked, local_ins_lat. +-F:: +--fields=:: + Specify output field - multiple keys can be specified in CSV format. + Please see linkperf:perf-report[1] for details. + + In addition to the default fields, 'perf mem report' will provide the + following fields to break down sample periods. + + - op: operation in the sample instruction (load, store, prefetch, ...) + - cache: location in CPU cache (L1, L2, ...) where the sample hit + - mem: location in memory or other places the sample hit + - dtlb: location in Data TLB (L1, L2) where the sample hit + - snoop: snoop result for the sampled data access + + Please take a look at the OUTPUT FIELD SELECTION section for caveats. + -T:: --type-profile:: Show data-type profile result instead of code symbols. This requires @@ -128,6 +153,59 @@ REPORT OPTIONS In addition, for report all perf report options are valid, and for record all perf record options. +OVERHEAD CALCULATION +-------------------- +Unlike linkperf:perf-report[1], which calculates overhead from the actual +sample period, perf-mem overhead is calculated using sample weight. E.g. +there are two samples in perf.data file, both with the same sample period, +but one sample with weight 180 and the other with weight 20: + + $ perf script -F period,data_src,weight,ip,sym + 100000 629080842 |OP LOAD|LVL L3 hit|... 20 7e69b93ca524 strcmp + 100000 1a29081042 |OP LOAD|LVL RAM hit|... 180 ffffffff82429168 memcpy + + $ perf report -F overhead,symbol + 50% [.] strcmp + 50% [k] memcpy + + $ perf mem report -F overhead,symbol + 90% [k] memcpy + 10% [.] strcmp + +OUTPUT FIELD SELECTION +---------------------- +"perf mem report" adds a number of new output fields specific to data source +information in the sample. Some of them have the same name with the existing +sort keys ("mem" and "snoop"). So unlike other fields and sort keys, they'll +behave differently when it's used by -F/--fields or -s/--sort. + +Using those two as output fields will aggregate samples altogether and show +breakdown. + + $ perf mem report -F mem,snoop + ... + # ------ Memory ------- --- Snoop ---- + # RAM Uncach Other HitM Other + # ..................... .............. + # + 3.5% 0.0% 96.5% 25.1% 74.9% + +But using the same name for sort keys will aggregate samples for each type +separately. + + $ perf mem report -s mem,snoop + # Overhead Samples Memory access Snoop + # ........ ............ ....................................... ............ + # + 47.99% 1509 L2 hit N/A + 25.08% 338 core, same node Any cache hit HitM + 10.24% 54374 N/A N/A + 6.77% 35938 L1 hit N/A + 6.39% 101 core, same node Any cache hit N/A + 3.50% 69 RAM hit N/A + 0.03% 158 LFB/MAB hit N/A + 0.00% 2 Uncached hit N/A + SEE ALSO -------- linkperf:perf-record[1], linkperf:perf-report[1], linkperf:perf-arm-spe[1] diff --git a/tools/perf/Documentation/perf-probe.txt b/tools/perf/Documentation/perf-probe.txt index 5c43a6edc0e5..2e5790325430 100644 --- a/tools/perf/Documentation/perf-probe.txt +++ b/tools/perf/Documentation/perf-probe.txt @@ -50,6 +50,12 @@ OPTIONS --source=PATH:: Specify path to kernel source. +--symfs=<directory[,layout]>:: + Look for files with symbols relative to this directory. The optional + layout can be 'hierarchy' (default, matches full path) or 'flat' + (only matches base name). This is useful when debug files are stored + in a flat directory structure. + -v:: --verbose:: Be more verbose (show parsed arguments, etc). diff --git a/tools/perf/Documentation/perf-record.txt b/tools/perf/Documentation/perf-record.txt index 80686d590de2..178f483140ed 100644 --- a/tools/perf/Documentation/perf-record.txt +++ b/tools/perf/Documentation/perf-record.txt @@ -227,6 +227,10 @@ OPTIONS '--filter' exists, the new filter expression will be combined with them by '&&'. +--latency:: + Enable data collection for latency profiling. + Use perf report --latency for latency-centric profile. + -a:: --all-cpus:: System-wide collection from all CPUs (default if no target is specified). @@ -321,6 +325,10 @@ OPTIONS by default. User can change the number by passing it after comma like "--call-graph fp,32". + Also "defer" can be used with "fp" (like "--call-graph fp,defer") to + enable deferred user callchain which will collect user-space callchains + when the thread returns to the user space. + -q:: --quiet:: Don't print any warnings or messages, useful for scripting. @@ -336,7 +344,8 @@ OPTIONS -d:: --data:: - Record the sample virtual addresses. + Record the sample virtual addresses. Implies --sample-mem-info and + --data-mmap. --phys-data:: Record the sample physical addresses. @@ -364,6 +373,11 @@ OPTIONS the sample_type member of the struct perf_event_attr argument to the perf_event_open system call. +--sample-mem-info:: + Record the sample data source information for memory operations. + It requires hardware supports and may work on specific events only. + Please consider using 'perf mem record' instead if you're not sure. + -n:: --no-samples:: Don't sample. @@ -441,7 +455,7 @@ following filters are defined: - no_tx: only when the target is not in a hardware transaction - abort_tx: only when the target is a hardware transaction abort - cond: conditional branches - - call_stack: save call stack + - stack: save call stack - no_flags: don't save branch flags e.g prediction, misprediction etc - no_cycles: don't save branch cycles - hw_index: save branch hardware index @@ -554,7 +568,9 @@ Specify vmlinux path which has debuginfo. Record build-id of all DSOs regardless whether it's actually hit or not. --buildid-mmap:: -Record build ids in mmap2 events, disables build id cache (implies --no-buildid). +Legacy record build-id in map events option which is now the +default. Behaves indentically to --no-buildid. Disable with +--no-buildid-mmap. --aio[=n]:: Use <n> control blocks in asynchronous (Posix AIO) trace writing mode (default: 1, max: 4). @@ -833,10 +849,24 @@ filtered through the mask provided by -C option. only, as of now. So the applications built without the frame pointer might see bogus addresses. + off-cpu profiling consists two types of samples: direct samples, which + share the same behavior as regular samples, and the accumulated + samples, stored in BPF stack trace map, presented after all the regular + samples. + +--off-cpu-thresh:: + Once a task's off-cpu time reaches this threshold (in milliseconds), it + generates a direct off-cpu sample. The default is 500ms. + --setup-filter=<action>:: Prepare BPF filter to be used by regular users. The action should be either "pin" or "unpin". The filter can be used after it's pinned. +--data-mmap:: + Enable recording MMAP events for non-executable mappings. Basically + perf only records executable mappings but data mmaping can be useful + when you analyze data access with sample addresses. So using -d option + would enable this unless you specify --no-data-mmap manually. include::intel-hybrid.txt[] diff --git a/tools/perf/Documentation/perf-report.txt b/tools/perf/Documentation/perf-report.txt index 87f864519406..22f87eaa3279 100644 --- a/tools/perf/Documentation/perf-report.txt +++ b/tools/perf/Documentation/perf-report.txt @@ -44,7 +44,7 @@ OPTIONS --comms=:: Only consider symbols in these comms. CSV that understands file://filename entries. This option will affect the percentage of - the overhead column. See --percentage for more info. + the overhead and latency columns. See --percentage for more info. --pid=:: Only show events for given process ID (comma separated list). @@ -54,12 +54,12 @@ OPTIONS --dsos=:: Only consider symbols in these dsos. CSV that understands file://filename entries. This option will affect the percentage of - the overhead column. See --percentage for more info. + the overhead and latency columns. See --percentage for more info. -S:: --symbols=:: Only consider these symbols. CSV that understands file://filename entries. This option will affect the percentage of - the overhead column. See --percentage for more info. + the overhead and latency columns. See --percentage for more info. --symbol-filter=:: Only show symbols that match (partially) with this filter. @@ -68,17 +68,33 @@ OPTIONS --hide-unresolved:: Only display entries resolved to a symbol. +--parallelism:: + Only consider these parallelism levels. Parallelism level is the number + of threads that actively run on CPUs at the time of sample. The flag + accepts single number, comma-separated list, and ranges (for example: + "1", "7,8", "1,64-128"). This is useful in understanding what a program + is doing during sequential/low-parallelism phases as compared to + high-parallelism phases. This option will affect the percentage of + the overhead and latency columns. See --percentage for more info. + Also see the `CPU and latency overheads' section for more details. + +--latency:: + Show latency-centric profile rather than the default + CPU-consumption-centric profile + (requires perf record --latency flag). + -s:: --sort=:: Sort histogram entries by given key(s) - multiple keys can be specified in CSV format. Following sort keys are available: pid, comm, dso, symbol, parent, cpu, socket, srcline, weight, - local_weight, cgroup_id, addr. + local_weight, cgroup_id, addr, comm_nodigit. Each key has following meaning: - comm: command (name) of the task which can be read via /proc/<pid>/comm - pid: command and tid of the task + - tgid: command and tgid of the task - dso: name of library or module executed at the time of sample - dso_size: size of library or module executed at the time of sample - symbol: name of function executed at the time of sample @@ -87,6 +103,7 @@ OPTIONS entries are displayed as "[other]". - cpu: cpu number the task ran at the time of sample - socket: processor socket number the task ran at the time of sample + - parallelism: number of running threads at the time of sample - srcline: filename and line number executed at the time of sample. The DWARF debugging info must be provided. - srcfile: file name of the source file of the samples. Requires dwarf @@ -97,12 +114,14 @@ OPTIONS - cgroup_id: ID derived from cgroup namespace device and inode numbers. - cgroup: cgroup pathname in the cgroupfs. - transaction: Transaction abort flags. - - overhead: Overhead percentage of sample - - overhead_sys: Overhead percentage of sample running in system mode - - overhead_us: Overhead percentage of sample running in user mode - - overhead_guest_sys: Overhead percentage of sample running in system mode + - overhead: CPU overhead percentage of sample. + - latency: latency (wall-clock) overhead percentage of sample. + See the `CPU and latency overheads' section for more details. + - overhead_sys: CPU overhead percentage of sample running in system mode + - overhead_us: CPU overhead percentage of sample running in user mode + - overhead_guest_sys: CPU overhead percentage of sample running in system mode on guest machine - - overhead_guest_us: Overhead percentage of sample running in user mode on + - overhead_guest_us: CPU overhead percentage of sample running in user mode on guest machine - sample: Number of sample - period: Raw number of event count of sample @@ -117,16 +136,20 @@ OPTIONS - addr: (Full) virtual address of the sampled instruction - retire_lat: On X86, this reports pipeline stall of this instruction compared to the previous instruction in cycles. And currently supported only on X86 - - simd: Flags describing a SIMD operation. "e" for empty Arm SVE predicate. "p" for partial Arm SVE predicate + - simd: Flags describing a SIMD operation. The architecture type can be Arm's + ASE (Advanced SIMD extension), SVE, SME. It provides an extra tag for + predicate: "e" for empty predicate, "p" for partial predicate, "d" for + predicate disabled, and "f" for full predicate. - type: Data type of sample memory access. - typeoff: Offset in the data type of sample memory access. - symoff: Offset in the symbol. - weight1: Average value of event specific weight (1st field of weight_struct). - weight2: Average value of event specific weight (2nd field of weight_struct). - weight3: Average value of event specific weight (3rd field of weight_struct). + - comm_nodigit: same as comm, with numbers replaced by "<N>" - By default, comm, dso and symbol keys are used. - (i.e. --sort comm,dso,symbol) + By default, overhead, comm, dso and symbol keys are used. + (i.e. --sort overhead,comm,dso,symbol). If --branch-stack option is used, following sort keys are also available: @@ -201,9 +224,9 @@ OPTIONS --fields=:: Specify output field - multiple keys can be specified in CSV format. Following fields are available: - overhead, overhead_sys, overhead_us, overhead_children, sample, period, - weight1, weight2, weight3, ins_lat, p_stage_cyc and retire_lat. The - last 3 names are alias for the corresponding weights. When the weight + overhead, latency, overhead_sys, overhead_us, overhead_children, sample, + period, weight1, weight2, weight3, ins_lat, p_stage_cyc and retire_lat. + The last 3 names are alias for the corresponding weights. When the weight fields are used, they will show the average value of the weight. Also it can contain any sort key(s). @@ -289,7 +312,7 @@ OPTIONS Accumulate callchain of children to parent entry so that then can show up in the output. The output will have a new "Children" column and will be sorted on the data. It requires callchains are recorded. - See the `overhead calculation' section for more details. Enabled by + See the `Overhead calculation' section for more details. Enabled by default, disable with --no-children. --max-stack:: @@ -349,8 +372,11 @@ OPTIONS --force:: Don't do ownership validation. ---symfs=<directory>:: - Look for files with symbols relative to this directory. +--symfs=<directory[,layout]>:: + Look for files with symbols relative to this directory. The optional + layout can be 'hierarchy' (default, matches full path) or 'flat' + (only matches base name). This is useful when debug files are stored + in a flat directory structure. -C:: --cpu:: Only report samples for the list of CPUs provided. Multiple CPUs can @@ -442,9 +468,9 @@ OPTIONS --call-graph option for details. --percentage:: - Determine how to display the overhead percentage of filtered entries. - Filters can be applied by --comms, --dsos and/or --symbols options and - Zoom operations on the TUI (thread, dso, etc). + Determine how to display the CPU and latency overhead percentage + of filtered entries. Filters can be applied by --comms, --dsos, --symbols + and/or --parallelism options and Zoom operations on the TUI (thread, dso, etc). "relative" means it's relative to filtered entries only so that the sum of shown entries will be always 100%. "absolute" means it retains @@ -627,6 +653,8 @@ include::itrace.txt[] --skip-empty:: Do not print 0 results in the --stat output. +include::cpu-and-latency-overheads.txt[] + include::callchain-overhead-calculation.txt[] SEE ALSO diff --git a/tools/perf/Documentation/perf-sched.txt b/tools/perf/Documentation/perf-sched.txt index 6dbbddb6464d..4da06215163a 100644 --- a/tools/perf/Documentation/perf-sched.txt +++ b/tools/perf/Documentation/perf-sched.txt @@ -8,7 +8,7 @@ perf-sched - Tool to trace/measure scheduler properties (latencies) SYNOPSIS -------- [verse] -'perf sched' {record|latency|map|replay|script|timehist} +'perf sched' {record|latency|map|replay|script|timehist|stats} DESCRIPTION ----------- @@ -40,6 +40,12 @@ There are several variants of 'perf sched': Tasks with the same command name are merged and the merge count is given within (), However if -p option is used, pid is mentioned. + If -H or --histogram option is passed, a CPU wait latency distribution + histogram is displayed illustrating how long tasks waited for CPU + runtime across latency buckets between snapshots. The --time + option (start,stop) limits analysis to a specific snapshot time interval. + The --hist-mode option (log or linear) configures the latency bucketing scheme. + 'perf sched script' to see a detailed trace of the workload that was recorded (aliased to 'perf script' for now). @@ -80,8 +86,267 @@ There are several variants of 'perf sched': Times are in msec.usec. + 'perf sched stats {record | report | diff} <command>' to capture, report the diff + in schedstat counters and show the difference between perf sched stats report + respectively. schedstat counters which are present in the linux kernel and are + exposed through the file ``/proc/schedstat``. These counters are enabled or disabled + via the sysctl governed by the file ``/proc/sys/kernel/sched_schedstats``. These + counters accounts for many scheduler events such as ``schedule()`` calls, load-balancing + events, ``try_to_wakeup()`` call among others. This is useful in understanding the + scheduler behavior for the workload. + + Note: The tool will not give correct results if there is topological reordering or + online/offline of cpus in between capturing snapshots of `/proc/schedstat`. + + Example usage: + perf sched stats record -- sleep 1 + perf sched stats report + perf sched stats diff + + A detailed description of the schedstats can be found in the Kernel Documentation: + https://www.kernel.org/doc/html/latest/scheduler/sched-stats.html + + The result can be interpreted as follows: + + The `perf sched stats report` starts with description of the columns present in + the report. These column names are given before cpu and domain stats to improve + the readability of the report. + + ---------------------------------------------------------------------------------------------------- + DESC -> Description of the field + COUNT -> Value of the field + PCT_CHANGE -> Percent change with corresponding base value + AVG_JIFFIES -> Avg time in jiffies between two consecutive occurrence of event + ---------------------------------------------------------------------------------------------------- + + Next is the total profiling time in terms of jiffies: + + ---------------------------------------------------------------------------------------------------- + Time elapsed (in jiffies) : 2323 + ---------------------------------------------------------------------------------------------------- + + Next is CPU scheduling statistics. These are simple diffs of /proc/schedstat CPU lines + along with description. The report also prints % relative to base stat. + + In the example below, schedule() left the CPU0 idle 36.58% of the time. 0.45% of total + try_to_wake_up() was to wakeup local CPU. And, the total waittime by tasks on CPU0 is + 48.70% of the total runtime by tasks on the same CPU. + + ---------------------------------------------------------------------------------------------------- + CPU 0 + ---------------------------------------------------------------------------------------------------- + DESC COUNT PCT_CHANGE + ---------------------------------------------------------------------------------------------------- + yld_count : 0 + array_exp : 0 + sched_count : 402267 + sched_goidle : 147161 ( 36.58% ) + ttwu_count : 236309 + ttwu_local : 1062 ( 0.45% ) + rq_cpu_time : 7083791148 + run_delay : 3449973971 ( 48.70% ) + pcount : 255035 + ---------------------------------------------------------------------------------------------------- + + Next is load balancing statistics. For each of the sched domains + (eg: `SMT`, `MC`, `DIE`...), the scheduler computes statistics under + the following three categories: + + 1) Idle Load Balance: Load balancing performed on behalf of a long + idling CPU by some other CPU. + 2) Busy Load Balance: Load balancing performed when the CPU was busy. + 3) New Idle Balance : Load balancing performed when a CPU just became + idle. + + Under each of these three categories, sched stats report provides + different load balancing statistics. Along with direct stats, the + report also contains derived metrics prefixed with *. Example: + + ---------------------------------------------------------------------------------------------------- + CPU 0, DOMAIN SMT CPUS 0,64 + ---------------------------------------------------------------------------------------------------- + DESC COUNT AVG_JIFFIES + ----------------------------------------- <Category busy> ------------------------------------------ + busy_lb_count : 136 $ 17.08 $ + busy_lb_balanced : 131 $ 17.73 $ + busy_lb_failed : 0 $ 0.00 $ + busy_lb_imbalance_load : 58 + busy_lb_imbalance_util : 0 + busy_lb_imbalance_task : 0 + busy_lb_imbalance_misfit : 0 + busy_lb_gained : 7 + busy_lb_hot_gained : 0 + busy_lb_nobusyq : 2 $ 1161.50 $ + busy_lb_nobusyg : 129 $ 18.01 $ + *busy_lb_success_count : 5 + *busy_lb_avg_pulled : 1.40 + ----------------------------------------- <Category idle> ------------------------------------------ + idle_lb_count : 449 $ 5.17 $ + idle_lb_balanced : 382 $ 6.08 $ + idle_lb_failed : 3 $ 774.33 $ + idle_lb_imbalance_load : 0 + idle_lb_imbalance_util : 0 + idle_lb_imbalance_task : 71 + idle_lb_imbalance_misfit : 0 + idle_lb_gained : 67 + idle_lb_hot_gained : 0 + idle_lb_nobusyq : 0 $ 0.00 $ + idle_lb_nobusyg : 382 $ 6.08 $ + *idle_lb_success_count : 64 + *idle_lb_avg_pulled : 1.05 + ---------------------------------------- <Category newidle> ---------------------------------------- + newidle_lb_count : 30471 $ 0.08 $ + newidle_lb_balanced : 28490 $ 0.08 $ + newidle_lb_failed : 633 $ 3.67 $ + newidle_lb_imbalance_load : 0 + newidle_lb_imbalance_util : 0 + newidle_lb_imbalance_task : 2040 + newidle_lb_imbalance_misfit : 0 + newidle_lb_gained : 1348 + newidle_lb_hot_gained : 0 + newidle_lb_nobusyq : 6 $ 387.17 $ + newidle_lb_nobusyg : 26634 $ 0.09 $ + *newidle_lb_success_count : 1348 + *newidle_lb_avg_pulled : 1.00 + ---------------------------------------------------------------------------------------------------- + + Consider following line: + + newidle_lb_balanced : 28490 $ 0.08 $ + + While profiling was active, the load-balancer found 28490 times the load + needs to be balanced on a newly idle CPU 0. Following value encapsulated + inside $ is average jiffies between two events (2323 / 28490 = 0.08). + + Next are active_load_balance() stats. alb did not trigger while the + profiling was active, hence it's all 0s. + + --------------------------------- <Category active_load_balance()> --------------------------------- + alb_count : 0 + alb_failed : 0 + alb_pushed : 0 + ---------------------------------------------------------------------------------------------------- + + Next are sched_balance_exec() and sched_balance_fork() stats. They are + not used but we kept it in RFC just for legacy purpose. Unless opposed, + we plan to remove them in next revision. + + Next are wakeup statistics. For every domain, the report also shows + task-wakeup statistics. Example: + + ------------------------------------------ <Wakeup Info> ------------------------------------------- + ttwu_wake_remote : 1590 + ttwu_move_affine : 84 + ttwu_move_balance : 0 + ---------------------------------------------------------------------------------------------------- + + Same set of stats are reported for each CPU and each domain level. + + How to interpret the diff + ~~~~~~~~~~~~~~~~~~~~~~~~~ + + The `perf sched stats diff` will also start with explaining the columns + present in the diff. Then it will show the diff in time in terms of + jiffies. The order of the values depends on the order of input data + files. It will take `perf.data.old` and `perf.data` respectively as the + defaults for comparison. Example: + + ---------------------------------------------------------------------------------------------------- + Time elapsed (in jiffies) : 2009, 2001 + ---------------------------------------------------------------------------------------------------- + + Below is the sample representing the difference in cpu and domain stats of + two runs. Here third column or the values enclosed in `|...|` shows the + percent change between the two. Second and fourth columns shows the + side-by-side representions of the corresponding fields from `perf sched + stats report`. + + ---------------------------------------------------------------------------------------------------- + CPU <ALL CPUS SUMMARY> + ---------------------------------------------------------------------------------------------------- + DESC COUNT1 COUNT2 PCT_CHANG> + ---------------------------------------------------------------------------------------------------- + yld_count : 0, 0 | 0.00> + array_exp : 0, 0 | 0.00> + sched_count : 528533, 412573 | -21.94> + sched_goidle : 193426, 146082 | -24.48> + ttwu_count : 313134, 385975 | 23.26> + ttwu_local : 1126, 1282 | 13.85> + rq_cpu_time : 8257200244, 8301250047 | 0.53> + run_delay : 4728347053, 3997100703 | -15.47> + pcount : 335031, 266396 | -20.49> + ---------------------------------------------------------------------------------------------------- + + Below is the sample of domain stats diff: + + ---------------------------------------------------------------------------------------------------- + CPU <ALL CPUS SUMMARY>, DOMAIN SMT + ---------------------------------------------------------------------------------------------------- + DESC COUNT1 COUNT2 PCT_CHANG> + ----------------------------------------- <Category busy> ------------------------------------------ + busy_lb_count : 122, 80 | -34.43> + busy_lb_balanced : 115, 76 | -33.91> + busy_lb_failed : 1, 3 | 200.00> + busy_lb_imbalance_load : 35, 49 | 40.00> + busy_lb_imbalance_util : 0, 0 | 0.00> + busy_lb_imbalance_task : 0, 0 | 0.00> + busy_lb_imbalance_misfit : 0, 0 | 0.00> + busy_lb_gained : 7, 2 | -71.43> + busy_lb_hot_gained : 0, 0 | 0.00> + busy_lb_nobusyq : 0, 0 | 0.00> + busy_lb_nobusyg : 115, 76 | -33.91> + *busy_lb_success_count : 6, 1 | -83.33> + *busy_lb_avg_pulled : 1.17, 2.00 | 71.43> + ----------------------------------------- <Category idle> ------------------------------------------ + idle_lb_count : 568, 620 | 9.15> + idle_lb_balanced : 462, 449 | -2.81> + idle_lb_failed : 11, 21 | 90.91> + idle_lb_imbalance_load : 0, 0 | 0.00> + idle_lb_imbalance_util : 0, 0 | 0.00> + idle_lb_imbalance_task : 115, 189 | 64.35> + idle_lb_imbalance_misfit : 0, 0 | 0.00> + idle_lb_gained : 103, 169 | 64.08> + idle_lb_hot_gained : 0, 0 | 0.00> + idle_lb_nobusyq : 0, 0 | 0.00> + idle_lb_nobusyg : 462, 449 | -2.81> + *idle_lb_success_count : 95, 150 | 57.89> + *idle_lb_avg_pulled : 1.08, 1.13 | 3.92> + ---------------------------------------- <Category newidle> ---------------------------------------- + newidle_lb_count : 16961, 3155 | -81.40> + newidle_lb_balanced : 15646, 2556 | -83.66> + newidle_lb_failed : 397, 142 | -64.23> + newidle_lb_imbalance_load : 0, 0 | 0.00> + newidle_lb_imbalance_util : 0, 0 | 0.00> + newidle_lb_imbalance_task : 1376, 655 | -52.40> + newidle_lb_imbalance_misfit : 0, 0 | 0.00> + newidle_lb_gained : 917, 457 | -50.16> + newidle_lb_hot_gained : 0, 0 | 0.00> + newidle_lb_nobusyq : 3, 1 | -66.67> + newidle_lb_nobusyg : 14480, 2103 | -85.48> + *newidle_lb_success_count : 918, 457 | -50.22> + *newidle_lb_avg_pulled : 1.00, 1.00 | 0.11> + --------------------------------- <Category active_load_balance()> --------------------------------- + alb_count : 0, 1 | 0.00> + alb_failed : 0, 0 | 0.00> + alb_pushed : 0, 1 | 0.00> + --------------------------------- <Category sched_balance_exec()> ---------------------------------- + sbe_count : 0, 0 | 0.00> + sbe_balanced : 0, 0 | 0.00> + sbe_pushed : 0, 0 | 0.00> + --------------------------------- <Category sched_balance_fork()> ---------------------------------- + sbf_count : 0, 0 | 0.00> + sbf_balanced : 0, 0 | 0.00> + sbf_pushed : 0, 0 | 0.00> + ------------------------------------------ <Wakeup Info> ------------------------------------------- + ttwu_wake_remote : 2031, 2914 | 43.48> + ttwu_move_affine : 73, 124 | 69.86> + ttwu_move_balance : 0, 0 | 0.00> + ---------------------------------------------------------------------------------------------------- + OPTIONS ------- +Applicable to {record|latency|map|replay|script} + -i:: --input=<file>:: Input file name. (default: perf.data unless stdin is a fifo) @@ -178,8 +443,11 @@ OPTIONS for 'perf sched timehist' Show all scheduling events followed by a summary by thread with min, max, and average run times (in sec) and relative stddev. ---symfs=<directory>:: - Look for files with symbols relative to this directory. +--symfs=<directory[,layout]>:: + Look for files with symbols relative to this directory. The optional + layout can be 'hierarchy' (default, matches full path) or 'flat' + (only matches base name). This is useful when debug files are stored + in a flat directory structure. -V:: --cpu-visual:: diff --git a/tools/perf/Documentation/perf-script.txt b/tools/perf/Documentation/perf-script.txt index b72866ef270b..200ea25891d8 100644 --- a/tools/perf/Documentation/perf-script.txt +++ b/tools/perf/Documentation/perf-script.txt @@ -98,8 +98,10 @@ OPTIONS -g:: --gen-script=:: - Generate perf-script.[ext] starter script for given language, - using current perf.data. + Generate a starter script. If a language is given then the + script is named perf-script.[ext] according to the + language. If a file path is given then python is used for + files ending '.py' and perl used for files ending '.pl'. --dlfilter=<file>:: Filter sample events using the given shared object file. @@ -239,13 +241,22 @@ OPTIONS i.e., -F "" is not allowed. The brstack output includes branch related information with raw addresses using the - /v/v/v/v/cycles syntax in the following order: - FROM: branch source instruction - TO : branch target instruction - M/P/-: M=branch target mispredicted or branch direction was mispredicted, P=target predicted or direction predicted, -=not supported - X/- : X=branch inside a transactional region, -=not in transaction region or not supported - A/- : A=TSX abort entry, -=not aborted region or not supported - cycles + FROM/TO/EVENT/INTX/ABORT/CYCLES/TYPE/SPEC syntax in the following order: + FROM : branch source instruction + TO : branch target instruction + EVENT : M=branch target or direction was mispredicted + P=branch target or direction was predicted + N=branch not-taken + -=no event or not supported + INTX : X=branch inside a transactional region + -=branch not in transaction region or not supported + ABORT : A=TSX abort entry + -=not aborted region or not supported + CYCLES: the number of cycles that have elapsed since the last branch was recorded + TYPE : branch type: COND/UNCOND/IND/CALL/IND_CALL/RET etc. + -=not supported + SPEC : branch speculation info: SPEC_WRONG_PATH/NON_SPEC_CORRECT_PATH/SPEC_CORRECT_PATH + -=not supported The brstacksym is identical to brstack, except that the FROM and TO addresses are printed in a symbolic form if possible. @@ -296,8 +307,11 @@ OPTIONS --kallsyms=<file>:: kallsyms pathname ---symfs=<directory>:: - Look for files with symbols relative to this directory. +--symfs=<directory[,layout]>:: + Look for files with symbols relative to this directory. The optional + layout can be 'hierarchy' (default, matches full path) or 'flat' + (only matches base name). This is useful when debug files are stored + in a flat directory structure. -G:: --hide-call-graph:: @@ -518,6 +532,11 @@ include::itrace.txt[] The known limitations include exception handing such as setjmp/longjmp will have calls/returns not match. +--merge-callchains:: + Enable merging deferred user callchains if available. This is the + default behavior. If you want to see separate CALLCHAIN_DEFERRED + records for some reason, use --no-merge-callchains explicitly. + :GMEXAMPLECMD: script :GMEXAMPLESUBCMD: include::guest-files.txt[] diff --git a/tools/perf/Documentation/perf-stat.txt b/tools/perf/Documentation/perf-stat.txt index 2bc063672486..f334aabdc809 100644 --- a/tools/perf/Documentation/perf-stat.txt +++ b/tools/perf/Documentation/perf-stat.txt @@ -162,6 +162,9 @@ null run - Don't start any counters. This can be useful to measure just elapsed wall-clock time - or to assess the raw overhead of perf stat itself, without running any counters. +--hide-zero-events:: +Do not show events with a zero count. + -v:: --verbose:: be more verbose (show counter open errors, etc) @@ -382,6 +385,11 @@ color the metric's computed value. Don't print output, warnings or messages. This is useful with perf stat record below to only write data to the perf.data file. +--no-affinity:: +Don't change scheduler CPU affinities when iterating over +CPUs. Disables an optimization aimed at minimizing interprocessor +interrupts. + STAT RECORD ----------- Stores stat data into perf data file. @@ -506,6 +514,13 @@ this option is not set. The TPEBS hardware feature starts from Intel Granite Rapids microarchitecture. This option only exists in X86_64 and is meaningful on Intel platforms with TPEBS feature. +--tpebs-mode=[mean|min|max|last]:: +Set how retirement latency events have their sample times +combined. The default "mean" gives the average of retirement +latency. "min" or "max" give the smallest or largest retirment latency +times respectively. "last" uses the last retirment latency sample's +time. + --td-level:: Print the top-down statistics that equal the input level. It allows users to print the interested top-down metrics level instead of the @@ -566,6 +581,10 @@ $ perf config stat.no-csv-summary=true Only enable events on applying cpu with this type for hybrid platform (e.g. core or atom)" +--pmu-filter:: +Only enable events on applying pmu with specified for multiple +pmus with same type (e.g. hisi_sicl2_cpa0 or hisi_sicl0_cpa0) + EXAMPLES -------- @@ -633,18 +652,20 @@ JSON FORMAT With -j, perf stat is able to print out a JSON format output that can be used for parsing. -- timestamp : optional usec time stamp in fractions of second (with -I) +- interval : optional timestamp in fractions of second (with -I) - optional aggregate options: - core : core identifier (with --per-core) - die : die identifier (with --per-die) - socket : socket identifier (with --per-socket) - node : node identifier (with --per-node) - thread : thread identifier (with --per-thread) +- counters : number of aggregated PMU counters - counter-value : counter value - unit : unit of the counter value or empty - event : event name - variance : optional variance if multiple values are collected (with -r) -- runtime : run time of counter +- event-runtime : run time of the event +- pcnt-running : percentage of time the event was running - metric-value : optional metric value - metric-unit : optional unit of metric diff --git a/tools/perf/Documentation/perf-test.txt b/tools/perf/Documentation/perf-test.txt index 32da0d1fa86a..859df74e62ef 100644 --- a/tools/perf/Documentation/perf-test.txt +++ b/tools/perf/Documentation/perf-test.txt @@ -55,17 +55,35 @@ OPTIONS -w:: --workload=:: - Run a built-in workload, to list them use '--list-workloads', current ones include: - noploop, thloop, leafloop, sqrtloop, brstack, datasym and landlock. + Run a built-in workload, to list them use '--list-workloads', current + ones include: noploop, thloop, leafloop, sqrtloop, brstack, datasym, + context_switch_loop, deterministic, named_threads, landlock and + callchain. Used with the shell script regression tests. Some accept an extra parameter: seconds: leafloop, noploop, sqrtloop, thloop - nrloops: brstack + nrloops: brstack, context_switch_loop - The datasym and landlock workloads don't accept any. + 'named_threads' accepts the number of threads and the number of loops to + do in each thread. + + The datasym, landlock, deterministic and callchain workloads don't accept + any. --list-workloads:: List the available workloads to use with -w/--workload. + +--record-ctl=fifo:ctl-fifo[,ack-fifo]:: + This option is used to communicate with a perf record session in order + to control the recording scope to only the workload and avoid recording + setup and teardown code. When specifying this option, the same FIFO path + must be specified in the record session via: + + perf record -D -1 --control=fifo:ctl-fifo[,ack-fifo] ... + + Perf test sends 'enable' and 'disable' commands through ctl-fifo to + control event recording. If 'ack-fifo' is provided, the workload runner + waits for an 'ack' response after each command. diff --git a/tools/perf/Documentation/perf-timechart.txt b/tools/perf/Documentation/perf-timechart.txt index ef0c7565bd5c..bacc5df3c400 100644 --- a/tools/perf/Documentation/perf-timechart.txt +++ b/tools/perf/Documentation/perf-timechart.txt @@ -53,8 +53,11 @@ TIMECHART OPTIONS -f:: --force:: Don't complain, do it. ---symfs=<directory>:: - Look for files with symbols relative to this directory. +--symfs=<directory[,layout]>:: + Look for files with symbols relative to this directory. The optional + layout can be 'hierarchy' (default, matches full path) or 'flat' + (only matches base name). This is useful when debug files are stored + in a flat directory structure. -n:: --proc-num:: Print task info for at least given number of tasks. @@ -94,6 +97,9 @@ RECORD OPTIONS -g:: --callchain:: Do call-graph (stack chain/backtrace) recording +-o:: +--output=:: + Select the output file (default: perf.data) EXAMPLES -------- diff --git a/tools/perf/Documentation/perf-trace.txt b/tools/perf/Documentation/perf-trace.txt index fb3d2af33844..d20b43ea3d37 100644 --- a/tools/perf/Documentation/perf-trace.txt +++ b/tools/perf/Documentation/perf-trace.txt @@ -150,6 +150,11 @@ the thread executes on the designated CPUs. Default is to monitor all CPUs. To be used with -s or -S, to show stats for the errnos experienced by syscalls, using only this option will trigger --summary. +--summary-mode=mode:: + To be used with -s or -S, to select how to show summary. By default it'll + show the syscall summary by thread. Possible values are: thread, total, + cgroup. + --tool_stats:: Show tool stats such as number of times fd->pathname was discovered thru hooking the open syscall return + vfs_getname or via reading /proc/pid/fd, etc. @@ -194,6 +199,9 @@ the thread executes on the designated CPUs. Default is to monitor all CPUs. --show-on-off-events:: Show the --switch-on/off events too. +--show-cpu:: + Show cpu id. + --max-stack:: Set the stack depth limit when parsing the callchain, anything beyond the specified depth will be ignored. Note that at this point @@ -233,20 +241,26 @@ the thread executes on the designated CPUs. Default is to monitor all CPUs. the same beautifiers used in the strace-like enter+exit lines to augment the tracepoint arguments. ---map-dump:: - Dump BPF maps setup by events passed via -e, for instance the augmented_raw_syscalls - living in tools/perf/examples/bpf/augmented_raw_syscalls.c. For now this - dumps just boolean map values and integer keys, in time this will print in hex - by default and use BTF when available, as well as use functions to do pretty - printing using the existing 'perf trace' syscall arg beautifiers to map integer - arguments to strings (pid to comm, syscall id to syscall name, etc). - --force-btf:: Use btf_dump to pretty print syscall argument data, instead of using hand-crafted pretty printers. This option is intended for testing BTF integration in perf trace. btf_dump-based pretty-printing serves as a fallback to hand-crafted pretty printers, as the latter can better pretty-print integer flags and struct pointers. +--bitmask-list:: + Show bitmasks as a human-readable, condensed list (e.g. "0,2-5,7") + instead of the default hexadecimal representation. + +--bpf-summary:: + Collect system call statistics in BPF. This is only for live mode and + works well with -s/--summary option where no argument information is + required. + +--max-summary=N:: + Maximum number of lines in the summary mode. Note that this applies to + each entry (thread or cgroup). + + PAGEFAULTS ---------- diff --git a/tools/perf/Documentation/perf.data-file-format.txt b/tools/perf/Documentation/perf.data-file-format.txt index 010a4edcd384..b90cba9168f8 100644 --- a/tools/perf/Documentation/perf.data-file-format.txt +++ b/tools/perf/Documentation/perf.data-file-format.txt @@ -348,6 +348,16 @@ to special needs. struct perf_bpil, which contains detailed information about a BPF program, including type, id, tag, jited/xlated instructions, etc. +The format of data in HEADER_BPF_PROG_INFO is as follows: + u32 count + + struct perf_bpil { + u32 info_len; /* size of struct bpf_prog_info, when the tool is compiled */ + u32 data_len; /* total bytes allocated for data, round up to 8 bytes */ + u64 arrays; /* which arrays are included in data */ + struct bpf_prog_info info; + u8 data[]; + }[count]; HEADER_BPF_BTF = 26, @@ -370,7 +380,7 @@ struct { u32 mmap_len; }; -Indicates that trace contains records of PERF_RECORD_COMPRESSED type +Indicates that trace contains records of PERF_RECORD_COMPRESSED2 type that have perf_events records in compressed form. HEADER_CPU_PMU_CAPS = 28, @@ -437,7 +447,38 @@ struct { } [nr_pmu]; }; - other bits are reserved and should ignored for now + HEADER_CPU_DOMAIN_INFO = 32, + +List of cpu-domain relation info. The format of the data is as below. + +struct domain_info { + int domain; + char dname[]; + char cpumask[]; + char cpulist[]; +}; + +struct cpu_domain_info { + int cpu; + int nr_domains; + struct domain_info domains[]; +}; + + HEADER_E_MACHINE = 33, + +ELF machine and flags data. e_machine is expanded from 16 to 32 bits +for alignment. Format: + + u32 e_machine; + u32 e_flags; + + HEADER_CLN_SIZE = 34, + +The size of the cacheline in bytes. Format: + + unsigned int cln_size; + + other bits are reserved and should be ignored for now HEADER_FEAT_BITS = 256, Attributes @@ -602,7 +643,14 @@ struct auxtrace_error_event { Describes a header feature. These are records used in pipe-mode that contain information that otherwise would be in perf.data file's header. - PERF_RECORD_COMPRESSED = 81, + PERF_RECORD_COMPRESSED = 81, /* deprecated */ + +The header is followed by compressed data frame that can be decompressed +into array of perf trace records. The size of the entire compressed event +record including the header is limited by the max value of header.size. + +It is deprecated and new files should use PERF_RECORD_COMPRESSED2 to gurantee +8-byte alignment. struct compressed_event { struct perf_event_header header; @@ -618,10 +666,17 @@ This is used, for instance, to 'perf inject' events after init and before regular events, those emitted by the kernel, to support combining guest and host records. + PERF_RECORD_COMPRESSED2 = 83, -The header is followed by compressed data frame that can be decompressed -into array of perf trace records. The size of the entire compressed event -record including the header is limited by the max value of header.size. +8-byte aligned version of `PERF_RECORD_COMPRESSED`. `header.size` indicates the +total record size, including padding for 8-byte alignment, and `data_size` +specifies the actual size of the compressed data. + +struct perf_record_compressed2 { + struct perf_event_header header; + __u64 data_size; + char data[]; +}; Event types diff --git a/tools/perf/Documentation/tips.txt b/tools/perf/Documentation/tips.txt index 67b326ba0040..ebf12a8c5db5 100644 --- a/tools/perf/Documentation/tips.txt +++ b/tools/perf/Documentation/tips.txt @@ -11,7 +11,7 @@ Search options using a keyword: perf report -h <keyword> Use parent filter to see specific call path: perf report -p <regex> List events using substring match: perf list <keyword> To see list of saved events and attributes: perf evlist -v -Use --symfs <dir> if your symbol files are in non-standard locations +Use --symfs <dir>[,layout] if your symbol files are in non-standard locations. To see callchains in a more compact form: perf report -g folded To see call chains by final symbol taking CPU time (bottom up) use perf report -G Show individual samples with: perf script @@ -62,3 +62,7 @@ To show context switches in perf report sample context add --switch-events to pe To show time in nanoseconds in record/report add --ns To compare hot regions in two workloads use perf record -b -o file ... ; perf diff --stream file1 file2 To compare scalability of two workload samples use perf diff -c ratio file1 file2 +For latency profiling, try: perf record/report --latency +For parallelism histogram, try: perf report --hierarchy --sort latency,parallelism,comm,symbol +To analyze particular parallelism levels, try: perf report --latency --parallelism=32-64 +To see how parallelism changes over time, try: perf report -F time,latency,parallelism --time-quantum=1s diff --git a/tools/perf/Documentation/topdown.txt b/tools/perf/Documentation/topdown.txt index 5c17fff694ee..fd4214f9a7b0 100644 --- a/tools/perf/Documentation/topdown.txt +++ b/tools/perf/Documentation/topdown.txt @@ -87,7 +87,7 @@ if (slots_fd < 0) /* Memory mapping the fd permits _rdpmc calls from userspace */ void *slots_p = mmap(0, getpagesize(), PROT_READ, MAP_SHARED, slots_fd, 0); -if (!slot_p) +if (slots_p == MAP_FAILED) .... error ... /* @@ -107,7 +107,7 @@ if (metrics_fd < 0) /* Memory mapping the fd permits _rdpmc calls from userspace */ void *metrics_p = mmap(0, getpagesize(), PROT_READ, MAP_SHARED, metrics_fd, 0); -if (!metrics_p) +if (metrics_p == MAP_FAILED) ... error ... Note: the file descriptors returned by the perf_event_open calls must be memory |