Merge tag 'pm-6.17-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm

Pull power management updates from Rafael Wysocki:
 "As is tradition, cpufreq is the part with the largest number of
  updates that include core fixes and cleanups as well as updates of
  several assorted drivers, but there are also quite a few updates
  related to system sleep, mostly focused on asynchronous suspend and
  resume of devices and on making the integration of system suspend
  and resume with runtime PM easier.

  Runtime PM is also updated to allow some code duplication in drivers
  to be eliminated going forward and to work more consistently overall
  in some cases.

  Apart from that, there are some driver core updates related to PM
  domains that should help to address ordering issues with devm_ cleanup
  routines relying on PM domains, some assorted devfreq updates
  including core fixes and cleanups, tooling updates, and documentation
  and MAINTAINERS updates.

  Specifics:

   - Fix two initialization ordering issues in the cpufreq core and a
     governor initialization error path in it, and clean it up (Lifeng
     Zheng)

   - Add Granite Rapids support in no-HWP mode to the intel_pstate
     cpufreq driver (Li RongQing)

   - Make intel_pstate always use HWP_DESIRED_PERF when operating in the
     passive mode (Rafael Wysocki)

   - Allow building the tegra124 cpufreq driver as a module (Aaron
     Kling)

   - Do minor cleanups for Rust cpufreq and cpumask APIs and fix
     MAINTAINERS entry for cpu.rs (Abhinav Ananthu, Ritvik Gupta, Lukas
     Bulwahn)

   - Clean up assorted cpufreq drivers (Arnd Bergmann, Dan Carpenter,
     Krzysztof Kozlowski, Sven Peter, Svyatoslav Ryhel, Lifeng Zheng)

   - Add the NEED_UPDATE_LIMITS flag to the CPPC cpufreq driver
     (Prashant Malani)

   - Fix minimum performance state label error in the amd-pstate driver
     documentation (Shouye Liu)

   - Add the CPUFREQ_GOV_STRICT_TARGET flag to the userspace cpufreq
     governor and explain HW coordination influence on it in the
     documentation (Shashank Balaji)

   - Fix opencoded for_each_cpu() in idle_state_valid() in the DT
     cpuidle driver (Yury Norov)

   - Remove info about non-existing QoS interfaces from the PM QoS
     documentation (Ulf Hansson)

   - Use c_* types via kernel prelude in Rust for OPP (Abhinav Ananthu)

   - Add HiSilicon uncore frequency scaling driver to devfreq (Jie Zhan)

   - Allow devfreq drivers to add custom sysfs ABIs (Jie Zhan)

   - Simplify the sun8i-a33-mbus devfreq driver by using more devm
     functions (Uwe Kleine-König)

   - Fix an index typo in trans_stat() in devfreq (Chanwoo Choi)

   - Check devfreq governor before using governor->name (Lifeng Zheng)

   - Remove a redundant devfreq_get_freq_range() call from
     devfreq_add_device() (Lifeng Zheng)

   - Limit max_freq with scaling_min_freq in devfreq (Lifeng Zheng)

   - Replace sscanf() with kstrtoul() in set_freq_store() (Lifeng Zheng)

   - Extend the asynchronous suspend and resume of devices to handle
     suppliers like parents and consumers like children (Rafael Wysocki)

   - Make pm_runtime_force_resume() work for drivers that set the
     DPM_FLAG_SMART_SUSPEND flag and allow PCI drivers and drivers that
     collaborate with the general ACPI PM domain to set it (Rafael
     Wysocki)

   - Add kernel parameter to disable asynchronous suspend/resume of
     devices (Tudor Ambarus)

   - Drop redundant might_sleep() calls from some functions in the
     device suspend/resume core code (Zhongqiu Han)

   - Fix the handling of monitors connected right before waking up the
     system from sleep (tuhaowen)

   - Clean up MAINTAINERS entries for suspend and hibernation (Rafael
     Wysocki)

   - Fix error code path in the KEXEC_JUMP flow and drop a redundant
     pm_restore_gfp_mask() call from it (Rafael Wysocki)

   - Rearrange suspend/resume error handling in the core device suspend
     and resume code (Rafael Wysocki)

   - Fix up white space that does not follow coding style in the
     hibernation core code (Darshan Rathod)

   - Document return values of suspend-related API functions in the
     runtime PM framework (Sakari Ailus)

   - Mark last busy stamp in multiple autosuspend-related functions in
     the runtime PM framework and update its documentation (Sakari
     Ailus)

   - Take active children into account in pm_runtime_get_if_in_use() for
     consistency (Rafael Wysocki)

   - Fix NULL pointer dereference in get_pd_power_uw() in the dtpm_cpu
     power capping driver (Sivan Zohar-Kotzer)

   - Add support for the Bartlett Lake platform to the Intel RAPL power
     capping driver (Qiao Wei)

   - Add PL4 support for Panther Lake to the intel_rapl_msr power
     capping driver (Zhang Rui)

   - Update contact information in the PM ABI docs and maintainer
     information in the power domains DT binding (Rafael Wysocki)

   - Update PM header inclusions to follow the IWYU (Include What You
     Use) principle (Andy Shevchenko)

   - Add flags to specify power on attach/detach for PM domains, make
     the driver core detach PM domains in device_unbind_cleanup(), and
     drop the dev_pm_domain_detach() call from the platform bus type
     (Claudiu Beznea)

   - Improve Python binding's Makefile for cpupower (John B. Wyatt IV)

   - Fix printing of CORE, CPU fields in cpupower-monitor (Gautham
     Shenoy)"

* tag 'pm-6.17-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm: (75 commits)
  cpufreq: CPPC: Mark driver with NEED_UPDATE_LIMITS flag
  PM: docs: Use my kernel.org address in ABI docs and DT bindings
  PM: hibernate: Fix up white space that does not follow coding style
  PM: sleep: Rearrange suspend/resume error handling in the core
  Documentation: amd-pstate:fix minimum performance state label error
  PM: runtime: Take active children into account in pm_runtime_get_if_in_use()
  kexec_core: Drop redundant pm_restore_gfp_mask() call
  kexec_core: Fix error code path in the KEXEC_JUMP flow
  PM: sleep: Clean up MAINTAINERS entries for suspend and hibernation
  drivers: cpufreq: add Tegra114 support
  rust: cpumask: Replace `MaybeUninit` and `mem::zeroed` with `Opaque` APIs
  cpufreq: Exit governor when failed to start old governor
  cpufreq: Move the check of cpufreq_driver->get into cpufreq_verify_current_freq()
  cpufreq: Init policy->rwsem before it may be possibly used
  cpufreq: Initialize cpufreq-based frequency-invariance later
  cpufreq: Remove duplicate check in __cpufreq_offline()
  cpufreq: Contain scaling_cur_freq.attr in cpufreq_attrs
  cpufreq: intel_pstate: Add Granite Rapids support in no-HWP mode
  cpufreq: intel_pstate: Always use HWP_DESIRED_PERF in passive mode
  PM / devfreq: Add HiSilicon uncore frequency scaling driver
  ...

1 files changed, 106 insertions, 48 deletions

diff --git a/drivers/base/power/runtime.c b/drivers/base/power/runtime.c
index c55a7c70bc1a..f835b85bd4d9 100644
--- a/drivers/base/power/runtime.c
+++ b/drivers/base/power/runtime.c
@@ -19,10 +19,24 @@
 
 typedef int (*pm_callback_t)(struct device *);
 
+static inline pm_callback_t get_callback_ptr(const void *start, size_t offset)
+{
+	return *(pm_callback_t *)(start + offset);
+}
+
+static pm_callback_t __rpm_get_driver_callback(struct device *dev,
+					       size_t cb_offset)
+{
+	if (dev->driver && dev->driver->pm)
+		return get_callback_ptr(dev->driver->pm, cb_offset);
+
+	return NULL;
+}
+
 static pm_callback_t __rpm_get_callback(struct device *dev, size_t cb_offset)
 {
-	pm_callback_t cb;
 	const struct dev_pm_ops *ops;
+	pm_callback_t cb = NULL;
 
 	if (dev->pm_domain)
 		ops = &dev->pm_domain->ops;
@@ -36,12 +50,10 @@ static pm_callback_t __rpm_get_callback(struct device *dev, size_t cb_offset)
 		ops = NULL;
 
 	if (ops)
-		cb = *(pm_callback_t *)((void *)ops + cb_offset);
-	else
-		cb = NULL;
+		cb = get_callback_ptr(ops, cb_offset);
 
-	if (!cb && dev->driver && dev->driver->pm)
-		cb = *(pm_callback_t *)((void *)dev->driver->pm + cb_offset);
+	if (!cb)
+		cb = __rpm_get_driver_callback(dev, cb_offset);
 
 	return cb;
 }
@@ -1191,10 +1203,12 @@ EXPORT_SYMBOL_GPL(__pm_runtime_resume);
  *
  * Return -EINVAL if runtime PM is disabled for @dev.
  *
- * Otherwise, if the runtime PM status of @dev is %RPM_ACTIVE and either
- * @ign_usage_count is %true or the runtime PM usage counter of @dev is not
- * zero, increment the usage counter of @dev and return 1. Otherwise, return 0
- * without changing the usage counter.
+ * Otherwise, if its runtime PM status is %RPM_ACTIVE and (1) @ign_usage_count
+ * is set, or (2) @dev is not ignoring children and its active child count is
+ * nonero, or (3) the runtime PM usage counter of @dev is not zero, increment
+ * the usage counter of @dev and return 1.
+ *
+ * Otherwise, return 0 without changing the usage counter.
  *
  * If @ign_usage_count is %true, this function can be used to prevent suspending
  * the device when its runtime PM status is %RPM_ACTIVE.
@@ -1216,7 +1230,8 @@ static int pm_runtime_get_conditional(struct device *dev, bool ign_usage_count)
 		retval = -EINVAL;
 	} else if (dev->power.runtime_status != RPM_ACTIVE) {
 		retval = 0;
-	} else if (ign_usage_count) {
+	} else if (ign_usage_count || (!dev->power.ignore_children &&
+		   atomic_read(&dev->power.child_count) > 0)) {
 		retval = 1;
 		atomic_inc(&dev->power.usage_count);
 	} else {
@@ -1249,10 +1264,16 @@ EXPORT_SYMBOL_GPL(pm_runtime_get_if_active);
  * @dev: Target device.
  *
  * Increment the runtime PM usage counter of @dev if its runtime PM status is
- * %RPM_ACTIVE and its runtime PM usage counter is greater than 0, in which case
- * it returns 1. If the device is in a different state or its usage_count is 0,
- * 0 is returned. -EINVAL is returned if runtime PM is disabled for the device,
- * in which case also the usage_count will remain unmodified.
+ * %RPM_ACTIVE and its runtime PM usage counter is greater than 0 or it is not
+ * ignoring children and its active child count is nonzero.  1 is returned in
+ * this case.
+ *
+ * If @dev is in a different state or it is not in use (that is, its usage
+ * counter is 0, or it is ignoring children, or its active child count is 0),
+ * 0 is returned.
+ *
+ * -EINVAL is returned if runtime PM is disabled for the device, in which case
+ * also the usage counter of @dev is not updated.
  */
 int pm_runtime_get_if_in_use(struct device *dev)
 {
@@ -1827,7 +1848,7 @@ void pm_runtime_init(struct device *dev)
 	dev->power.request_pending = false;
 	dev->power.request = RPM_REQ_NONE;
 	dev->power.deferred_resume = false;
-	dev->power.needs_force_resume = 0;
+	dev->power.needs_force_resume = false;
 	INIT_WORK(&dev->power.work, pm_runtime_work);
 
 	dev->power.timer_expires = 0;
@@ -1854,6 +1875,11 @@ void pm_runtime_reinit(struct device *dev)
 				pm_runtime_put(dev->parent);
 		}
 	}
+	/*
+	 * Clear power.needs_force_resume in case it has been set by
+	 * pm_runtime_force_suspend() invoked from a driver remove callback.
+	 */
+	dev->power.needs_force_resume = false;
 }
 
 /**
@@ -1941,13 +1967,23 @@ void pm_runtime_drop_link(struct device_link *link)
 	pm_request_idle(link->supplier);
 }
 
-bool pm_runtime_need_not_resume(struct device *dev)
+static pm_callback_t get_callback(struct device *dev, size_t cb_offset)
 {
-	return atomic_read(&dev->power.usage_count) <= 1 &&
-		(atomic_read(&dev->power.child_count) == 0 ||
-		 dev->power.ignore_children);
+	/*
+	 * Setting power.strict_midlayer means that the middle layer
+	 * code does not want its runtime PM callbacks to be invoked via
+	 * pm_runtime_force_suspend() and pm_runtime_force_resume(), so
+	 * return a direct pointer to the driver callback in that case.
+	 */
+	if (dev_pm_strict_midlayer_is_set(dev))
+		return __rpm_get_driver_callback(dev, cb_offset);
+
+	return __rpm_get_callback(dev, cb_offset);
 }
 
+#define GET_CALLBACK(dev, callback) \
+		get_callback(dev, offsetof(struct dev_pm_ops, callback))
+
 /**
  * pm_runtime_force_suspend - Force a device into suspend state if needed.
  * @dev: Device to suspend.
@@ -1964,10 +2000,6 @@ bool pm_runtime_need_not_resume(struct device *dev)
  * sure the device is put into low power state and it should only be used during
  * system-wide PM transitions to sleep states.  It assumes that the analogous
  * pm_runtime_force_resume() will be used to resume the device.
- *
- * Do not use with DPM_FLAG_SMART_SUSPEND as this can lead to an inconsistent
- * state where this function has called the ->runtime_suspend callback but the
- * PM core marks the driver as runtime active.
  */
 int pm_runtime_force_suspend(struct device *dev)
 {
@@ -1975,10 +2007,10 @@ int pm_runtime_force_suspend(struct device *dev)
 	int ret;
 
 	pm_runtime_disable(dev);
-	if (pm_runtime_status_suspended(dev))
+	if (pm_runtime_status_suspended(dev) || dev->power.needs_force_resume)
 		return 0;
 
-	callback = RPM_GET_CALLBACK(dev, runtime_suspend);
+	callback = GET_CALLBACK(dev, runtime_suspend);
 
 	dev_pm_enable_wake_irq_check(dev, true);
 	ret = callback ? callback(dev) : 0;
@@ -1990,15 +2022,16 @@ int pm_runtime_force_suspend(struct device *dev)
 	/*
 	 * If the device can stay in suspend after the system-wide transition
 	 * to the working state that will follow, drop the children counter of
-	 * its parent, but set its status to RPM_SUSPENDED anyway in case this
-	 * function will be called again for it in the meantime.
+	 * its parent and the usage counters of its suppliers.  Otherwise, set
+	 * power.needs_force_resume to let pm_runtime_force_resume() know that
+	 * the device needs to be taken care of and to prevent this function
+	 * from handling the device again in case the device is passed to it
+	 * once more subsequently.
 	 */
-	if (pm_runtime_need_not_resume(dev)) {
+	if (pm_runtime_need_not_resume(dev))
 		pm_runtime_set_suspended(dev);
-	} else {
-		__update_runtime_status(dev, RPM_SUSPENDED);
-		dev->power.needs_force_resume = 1;
-	}
+	else
+		dev->power.needs_force_resume = true;
 
 	return 0;
 
@@ -2009,33 +2042,37 @@ err:
 }
 EXPORT_SYMBOL_GPL(pm_runtime_force_suspend);
 
+#ifdef CONFIG_PM_SLEEP
+
 /**
  * pm_runtime_force_resume - Force a device into resume state if needed.
  * @dev: Device to resume.
  *
- * Prior invoking this function we expect the user to have brought the device
- * into low power state by a call to pm_runtime_force_suspend(). Here we reverse
- * those actions and bring the device into full power, if it is expected to be
- * used on system resume.  In the other case, we defer the resume to be managed
- * via runtime PM.
+ * This function expects that either pm_runtime_force_suspend() has put the
+ * device into a low-power state prior to calling it, or the device had been
+ * runtime-suspended before the preceding system-wide suspend transition and it
+ * was left in suspend during that transition.
  *
- * Typically this function may be invoked from a system resume callback.
+ * The actions carried out by pm_runtime_force_suspend(), or by a runtime
+ * suspend in general, are reversed and the device is brought back into full
+ * power if it is expected to be used on system resume, which is the case when
+ * its needs_force_resume flag is set or when its smart_suspend flag is set and
+ * its runtime PM status is "active".
+ *
+ * In other cases, the resume is deferred to be managed via runtime PM.
+ *
+ * Typically, this function may be invoked from a system resume callback.
  */
 int pm_runtime_force_resume(struct device *dev)
 {
 	int (*callback)(struct device *);
 	int ret = 0;
 
-	if (!dev->power.needs_force_resume)
+	if (!dev->power.needs_force_resume && (!dev_pm_smart_suspend(dev) ||
+	    pm_runtime_status_suspended(dev)))
 		goto out;
 
-	/*
-	 * The value of the parent's children counter is correct already, so
-	 * just update the status of the device.
-	 */
-	__update_runtime_status(dev, RPM_ACTIVE);
-
-	callback = RPM_GET_CALLBACK(dev, runtime_resume);
+	callback = GET_CALLBACK(dev, runtime_resume);
 
 	dev_pm_disable_wake_irq_check(dev, false);
 	ret = callback ? callback(dev) : 0;
@@ -2046,9 +2083,30 @@ int pm_runtime_force_resume(struct device *dev)
 	}
 
 	pm_runtime_mark_last_busy(dev);
+
 out:
-	dev->power.needs_force_resume = 0;
+	/*
+	 * The smart_suspend flag can be cleared here because it is not going
+	 * to be necessary until the next system-wide suspend transition that
+	 * will update it again.
+	 */
+	dev->power.smart_suspend = false;
+	/*
+	 * Also clear needs_force_resume to make this function skip devices that
+	 * have been seen by it once.
+	 */
+	dev->power.needs_force_resume = false;
+
 	pm_runtime_enable(dev);
 	return ret;
 }
 EXPORT_SYMBOL_GPL(pm_runtime_force_resume);
+
+bool pm_runtime_need_not_resume(struct device *dev)
+{
+	return atomic_read(&dev->power.usage_count) <= 1 &&
+		(atomic_read(&dev->power.child_count) == 0 ||
+		 dev->power.ignore_children);
+}
+
+#endif /* CONFIG_PM_SLEEP */