iommu/amd: Add PerfOpt IOMMU performance optimization support

Add support for the AMD IOMMU Performance Optimization (PerfOpt)
feature as defined in the AMD I/O Virtualization Technology (IOMMU)
Specification, Section 3.4.9 (MMIO Offset 016Ch).

This feature allows privileged integrated I/O devices (GPUs) to bypass
the IOMMU when directly accessing system memory.  The IOMMU only
enforces the IR/IW permission bits without GPA->SPA translations.

amd_iommu_enable_perfopt() performs a detach/reattach cycle to rehome
devices already on the identity domain with ATS/PRI/PASID/GCR3
disabled (skip_caps path).  amd_iommu_disable_perfopt() restores those
capabilities.  The per-device dev_data->perfopt flag tracks state.

PERF_OPT_EN is a single control bit per IOMMU, shared by every device
behind that IOMMU, while enablement is requested per device.  It is
therefore reference counted (amd_iommu->perfopt_refcount): armed on the
first requesting device and cleared on the last, so one device's
teardown never clears the bit while a peer behind the same IOMMU still
needs it.

The per-device flag is cleared on every teardown path
(blocked_domain_attach, release_device, and amd_iommu_disable_perfopt),
dropping the reference with it, so a reused dev_data never carries stale
PerfOpt state onto its next bind.

On suspend/resume the hardware is reprogrammed from scratch:
amd_iommu_perfopt_clear() forces the bit off without touching the
reference count, and amd_iommu_perfopt_restore() re-asserts it from the
count after early_enable_iommu(), so armed devices keep the optimization
across resume without relying on each consumer driver to re-arm.

The exported amd_iommu_enable_perfopt()/amd_iommu_disable_perfopt() run
only from a consumer driver's bind/unbind path.  group->mutex is not
exposed to drivers, but a device bound to its native driver cannot have
its IOMMU domain changed concurrently by the core, which serializes the
detach/attach pair against core-driven attach.

PerfOpt is opt-in -- only enabled when explicitly requested by a driver.

Tested-by: Derek J. Clark <derekjohn.clark@gmail.com>
Co-developed-by: Jatin Kataria <jkataria@netflix.com>
Signed-off-by: Jatin Kataria <jkataria@netflix.com>
Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
Tested-by: Boqun Feng (Netflix) <boqun@kernel.org>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
[joerg.roedel@amd.com: Fixed build failure on i386]
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>

0 files changed, 0 insertions, 0 deletions