lwn.git/drivers/cpuidle/cpuidle.c, branch docs-6.1

Merge tag 'pm-5.20-rc1-2' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm

2022-08-08T21:29:00+00:00

Pull more power management updates from Rafael Wysocki: "These are ARM cpufreq updates and operating performance points (OPP) updates plus one cpuidle update adding a new trace point. Specifics: - Fix return error code in mtk_cpu_dvfs_info_init (Yang Yingliang). - Minor cleanups and support for new boards for Qcom cpufreq drivers (Bryan O'Donoghue, Konrad Dybcio, Pierre Gondois, and Yicong Yang). - Fix sparse warnings for Tegra cpufreq driver (Viresh Kumar). - Make dev_pm_opp_set_regulators() accept NULL terminated list (Viresh Kumar). - Add dev_pm_opp_set_config() and friends and migrate other users and helpers to using them (Viresh Kumar). - Add support for multiple clocks for a device (Viresh Kumar and Krzysztof Kozlowski). - Configure resources before adding OPP table for Venus (Stanimir Varbanov). - Keep reference count up for opp->np and opp_table->np while they are still in use (Liang He). - Minor OPP cleanups (Viresh Kumar and Yang Li). - Add a trace event for cpuidle to track missed (too deep or too shallow) wakeups (Kajetan Puchalski)" * tag 'pm-5.20-rc1-2' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm: (55 commits) cpuidle: Add cpu_idle_miss trace event venus: pm_helpers: Fix warning in OPP during probe OPP: Don't drop opp->np reference while it is still in use OPP: Don't drop opp_table->np reference while it is still in use cpufreq: tegra194: Staticize struct tegra_cpufreq_soc instances dt-bindings: cpufreq: cpufreq-qcom-hw: Add SM6375 compatible dt-bindings: opp: Add msm8939 to the compatible list dt-bindings: opp: Add missing compat devices dt-bindings: opp: opp-v2-kryo-cpu: Fix example binding checks cpufreq: Change order of online() CB and policy->cpus modification cpufreq: qcom-hw: Remove deprecated irq_set_affinity_hint() call cpufreq: qcom-hw: Disable LMH irq when disabling policy cpufreq: qcom-hw: Reset cancel_throttle when policy is re-enabled cpufreq: qcom-cpufreq-hw: use HZ_PER_KHZ macro in units.h cpufreq: mediatek: fix error return code in mtk_cpu_dvfs_info_init() OPP: Remove dev{m}_pm_opp_of_add_table_noclk() PM / devfreq: tegra30: Register config_clks helper OPP: Allow config_clks helper for single clk case OPP: Provide a simple implementation to configure multiple clocks OPP: Assert clk_count == 1 for single clk helpers ...

cpuidle: Add cpu_idle_miss trace event

2022-08-03T15:50:58+00:00

Add a trace event for cpuidle to track missed (too deep or too shallow) wakeups. After each wakeup, CPUIdle already computes whether the entered state was optimal, above or below the desired one and updates the relevant counters. This patch makes it possible to trace those events in addition to just reading the counters. The patterns of types and percentages of misses across different workloads appear to be very consistent. This makes the trace event very useful for comparing the relative correctness of different CPUIdle governors for different types of workloads, or for finding the optimal governor for a given device. Signed-off-by: Kajetan Puchalski Reviewed-by: Steven Rostedt (Google) Signed-off-by: Rafael J. Wysocki

context_tracking: Take idle eqs entrypoints over RCU

2022-07-05T20:32:16+00:00

The RCU dynticks counter is going to be merged into the context tracking subsystem. Start with moving the idle extended quiescent states entrypoints to context tracking. For now those are dumb redirections to existing RCU calls. [ paulmck: Apply kernel test robot feedback. ] Signed-off-by: Frederic Weisbecker Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Neeraj Upadhyay Cc: Uladzislau Rezki Cc: Joel Fernandes Cc: Boqun Feng Cc: Nicolas Saenz Julienne Cc: Marcelo Tosatti Cc: Xiongfeng Wang Cc: Yu Liao Cc: Phil Auld Cc: Paul Gortmaker Cc: Alex Belits Signed-off-by: Paul E. McKenney Reviewed-by: Nicolas Saenz Julienne Tested-by: Nicolas Saenz Julienne

cpuidle: Select polling interval based on a c-state with a longer target residency

2020-12-01T16:59:18+00:00

It was noted that a few workloads that idle rapidly regressed when commit 36fcb4292473 ("cpuidle: use first valid target residency as poll time") was merged. The workloads in question were heavy communicators that idle rapidly and were impacted by the c-state exit latency as the active CPUs were not polling at the time of wakeup. As they were not particularly realistic workloads, it was not considered to be a major problem. Unfortunately, a bug was reported for a real workload in a production environment that relied on large numbers of threads operating in a worker pool pattern. These threads would idle for periods of time longer than the C1 target residency and so incurred the c-state exit latency penalty. The application is very sensitive to wakeup latency and indirectly relying on behaviour prior to commit on a37b969a61c1 ("cpuidle: poll_state: Add time limit to poll_idle()") to poll for long enough to avoid the exit latency cost. The target residency of C1 is typically very short. On some x86 machines, it can be as low as 2 microseconds. In poll_idle(), the clock is checked every POLL_IDLE_RELAX_COUNT interations of cpu_relax() and even one iteration of that loop can be over 1 microsecond so the polling interval is very close to the granularity of what poll_idle() can detect. Furthermore, a basic ping pong workload like perf bench pipe has a longer round-trip time than the 2 microseconds meaning that the CPU will almost certainly not be polling when the ping-pong completes. This patch selects a polling interval based on an enabled c-state that has an target residency longer than 10usec. If there is no enabled-cstate then polling will be up to a TICK_NSEC/16 similar to what it was up until kernel 4.20. Polling for a full tick is unlikely (rescheduling event) and is much longer than the existing target residencies for a deep c-state. As an example, consider a CPU with the following c-state information from an Intel CPU; residency exit_latency C1 2 2 C1E 20 10 C3 100 33 C6 400 133 The polling interval selected is 20usec. If booted with intel_idle.max_cstate=1 then the polling interval is 250usec as the deeper c-states were not available. On an AMD EPYC machine, the c-state information is more limited and looks like residency exit_latency C1 2 1 C2 800 400 The polling interval selected is 250usec. While C2 was considered, the polling interval was clamped by CPUIDLE_POLL_MAX. Note that it is not expected that polling will be a universal win. As well as potentially trading power for performance, the performance is not guaranteed if the extra polling prevented a turbo state being reached. Making it a tunable was considered but it's driver-specific, may be overridden by a governor and is not a guaranteed polling interval making it difficult to describe without knowledge of the implementation. tbench4 vanilla polling Hmean 1 497.89 ( 0.00%) 543.15 * 9.09%* Hmean 2 975.88 ( 0.00%) 1059.73 * 8.59%* Hmean 4 1953.97 ( 0.00%) 2081.37 * 6.52%* Hmean 8 3645.76 ( 0.00%) 4052.95 * 11.17%* Hmean 16 6882.21 ( 0.00%) 6995.93 * 1.65%* Hmean 32 10752.20 ( 0.00%) 10731.53 * -0.19%* Hmean 64 12875.08 ( 0.00%) 12478.13 * -3.08%* Hmean 128 21500.54 ( 0.00%) 21098.60 * -1.87%* Hmean 256 21253.70 ( 0.00%) 21027.18 * -1.07%* Hmean 320 20813.50 ( 0.00%) 20580.64 * -1.12%* Signed-off-by: Mel Gorman Signed-off-by: Rafael J. Wysocki

Merge back cpuidle material for 5.10.

2020-09-28T14:31:25+00:00

cpuidle: record state entry rejection statistics

2020-09-23T12:10:31+00:00

CPUs may fail to enter the chosen idle state if there was a pending interrupt, causing the cpuidle driver to return an error value. Record that and export it via sysfs along with the other idle state statistics. This could prove useful in understanding behavior of the governor and the system during usecases that involve multiple CPUs. Signed-off-by: Lina Iyer [ rjw: Changelog and documentation edits ] Signed-off-by: Rafael J. Wysocki

cpuidle: Drop misleading comments about RCU usage

2020-09-22T17:32:03+00:00

The commit 1098582a0f6c ("sched,idle,rcu: Push rcu_idle deeper into the idle path"), moved the calls rcu_idle_enter|exit() into the cpuidle core. However, it forgot to remove a couple of comments in enter_s2idle_proper() about why RCU_NONIDLE earlier was needed. So, let's drop them as they have become a bit misleading. Fixes: 1098582a0f6c ("sched,idle,rcu: Push rcu_idle deeper into the idle path") Signed-off-by: Ulf Hansson Signed-off-by: Rafael J. Wysocki

cpuidle: Allow cpuidle drivers to take over RCU-idle

2020-09-16T17:36:26+00:00

Some drivers have to do significant work, some of which relies on RCU still being active. Instead of using RCU_NONIDLE in the drivers and flipping RCU back on, allow drivers to take over RCU-idle duty. Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Ulf Hansson Tested-by: Borislav Petkov Signed-off-by: Rafael J. Wysocki

cpuidle: Make CPUIDLE_FLAG_TLB_FLUSHED generic

2020-08-26T10:41:53+00:00

This allows moving the leave_mm() call into generic code before rcu_idle_enter(). Gets rid of more trace_*_rcuidle() users. Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Steven Rostedt (VMware) Reviewed-by: Thomas Gleixner Acked-by: Rafael J. Wysocki Tested-by: Marco Elver Link: https://lkml.kernel.org/r/20200821085348.369441600@infradead.org

sched,idle,rcu: Push rcu_idle deeper into the idle path

2020-08-26T10:41:53+00:00

Lots of things take locks, due to a wee bug, rcu_lockdep didn't notice that the locking tracepoints were using RCU. Push rcu_idle_{enter,exit}() as deep as possible into the idle paths, this also resolves a lot of _rcuidle()/RCU_NONIDLE() usage. Specifically, sched_clock_idle_wakeup_event() will use ktime which will use seqlocks which will tickle lockdep, and stop_critical_timings() uses lock. Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Steven Rostedt (VMware) Reviewed-by: Thomas Gleixner Acked-by: Rafael J. Wysocki Tested-by: Marco Elver Link: https://lkml.kernel.org/r/20200821085348.310943801@infradead.org