summaryrefslogtreecommitdiff
path: root/drivers/iommu/amd/init.c
AgeCommit message (Collapse)Author
13 daysMerge branches 'apple/dart', 'arm/smmu/updates', 'arm/smmu/bindings', ↵Joerg Roedel
'hyper-v', 'samsung/exynos', 'riscv', 'broadcom', 'intel/vt-d', 'amd/amd-vi' and 'core' into next
2026-10-14iommu/amd: Don't allocate kdump device table from DMA32Matt Fleming
Booting a kdump kernel with crashkernel=X,high crashkernel=0,low on an AMD system fails the per-segment device table allocation because there is no memory below 4G: swapper/0: page allocation failure: order:9, mode:0x40104(GFP_DMA32|__GFP_ZERO|__GFP_COMP) iommu_alloc_pages_node_sz+0x7f/0x150 alloc_pci_segment+0x4bd/0x650 iommu_go_to_state+0x213d/0x34e0 amd_iommu_prepare+0x39/0xa0 irq_remapping_prepare+0x83/0xd0 enable_IR_x2apic+0x61/0x370 On our E810 systems, the kdump kernel booted but ice MSI-X interrupts never fired and the TX queues timed out. Commit b336781b82cc ("iommu/amd: Allocate memory below 4G for dev table if translation pre-enabled") allocated device tables below 4G. At the time, the kdump kernel programmed a copy of the old table into the base register with the IOMMU still enabled. The 64-bit register is updated as two 32-bit halves, so the running IOMMU could see a half-written, invalid table pointer. Keeping both tables below 4G leaves the upper half zero throughout the update, so the pointer stays valid. Since commit 38e5f33ee359 ("iommu/amd: Reuse device table for kdump") the kdump kernel reuses the previous kernel's table without writing the base register, so its newly allocated table is discarded when reuse succeeds. If reuse fails, the previous patch programs the new table only after disabling the IOMMU. Either way, the kdump kernel's table doesn't need to be below 4G. The previous kernel still allocates its table from DMA32, so the kdump kernel's check that the old table is below 4G still works. Drop GFP_DMA32 for kdump kernels only. Cc: stable@vger.kernel.org # 6.18.x Reviewed-by: Ashish Kalra <ashish.kalra@amd.com> Reviewed-by: Ankit Soni <Ankit.Soni@amd.com> Signed-off-by: Matt Fleming <mfleming@cloudflare.com> Reviewed-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Matt Fleming <mfleming@cloudflare.com> Reviewed-by: Ashish Kalra <ashish.kalra@amd.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-10-14iommu/amd: Program device table when kdump reuse failsMatt Fleming
Commit 38e5f33ee359 ("iommu/amd: Reuse device table for kdump") made iommu_set_device_table() return early in every kdump kernel. That is needed when the previous kernel's device table is reused: the base register already points at it, and on SNP-enabled systems the register is locked. But if reuse isn't possible, early_enable_iommus() falls back to the freshly allocated table and calls early_enable_iommu(). Because of the early return, that table is never written to the base register. The IOMMU keeps using whatever table the register pointed at before, while the driver writes entries into the new one. Only skip the write when the previous kernel's table was reused. Fixes: 38e5f33ee359 ("iommu/amd: Reuse device table for kdump") Cc: stable@vger.kernel.org Reviewed-by: Ashish Kalra <ashish.kalra@amd.com> Reviewed-by: Ankit Soni <Ankit.Soni@amd.com> Reviewed-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Matt Fleming <mfleming@cloudflare.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-10-14Revert "iommu/amd: Remove redundant check in irq_remapping_select()"Joerg Roedel
This reverts commit 5e1afd4ea1d6a9bbaecf3e28707dac9c8b56bd45. This commit caused a regression with GPU devices on some platforms. Revert it for now. Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-09-25iommu/amd: Add PerfOpt IOMMU performance optimization supportMario Limonciello
Add support for the AMD IOMMU Performance Optimization (PerfOpt) feature as defined in the AMD I/O Virtualization Technology (IOMMU) Specification, Section 3.4.9 (MMIO Offset 016Ch). This feature allows privileged integrated I/O devices (GPUs) to bypass the IOMMU when directly accessing system memory. The IOMMU only enforces the IR/IW permission bits without GPA->SPA translations. amd_iommu_enable_perfopt() performs a detach/reattach cycle to rehome devices already on the identity domain with ATS/PRI/PASID/GCR3 disabled (skip_caps path). amd_iommu_disable_perfopt() restores those capabilities. The per-device dev_data->perfopt flag tracks state. PERF_OPT_EN is a single control bit per IOMMU, shared by every device behind that IOMMU, while enablement is requested per device. It is therefore reference counted (amd_iommu->perfopt_refcount): armed on the first requesting device and cleared on the last, so one device's teardown never clears the bit while a peer behind the same IOMMU still needs it. The per-device flag is cleared on every teardown path (blocked_domain_attach, release_device, and amd_iommu_disable_perfopt), dropping the reference with it, so a reused dev_data never carries stale PerfOpt state onto its next bind. On suspend/resume the hardware is reprogrammed from scratch: amd_iommu_perfopt_clear() forces the bit off without touching the reference count, and amd_iommu_perfopt_restore() re-asserts it from the count after early_enable_iommu(), so armed devices keep the optimization across resume without relying on each consumer driver to re-arm. The exported amd_iommu_enable_perfopt()/amd_iommu_disable_perfopt() run only from a consumer driver's bind/unbind path. group->mutex is not exposed to drivers, but a device bound to its native driver cannot have its IOMMU domain changed concurrently by the core, which serializes the detach/attach pair against core-driven attach. PerfOpt is opt-in -- only enabled when explicitly requested by a driver. Tested-by: Derek J. Clark <derekjohn.clark@gmail.com> Co-developed-by: Jatin Kataria <jkataria@netflix.com> Signed-off-by: Jatin Kataria <jkataria@netflix.com> Signed-off-by: Mario Limonciello <mario.limonciello@amd.com> Tested-by: Boqun Feng (Netflix) <boqun@kernel.org> Acked-by: Alex Deucher <alexander.deucher@amd.com> [joerg.roedel@amd.com: Fixed build failure on i386] Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-09-24iommu/amd: Move GA log allocation to initVasant Hegde
Currently GA log buffer allocation is done at the time of enabling ga log. This mixes capability checking, resource allocation, and hardware enablement into a single late stage code path. Move capability checking and buffer allocation to init path: - check_vapic_support(): Checks AVIC support - alloc_ga_log(): Rename iommu_init_ga_log() as alloc_ga_log(). It will allocate the GA log and GA log tail buffers for a given IOMMU. Remove redundant checks inside alloc_ga_log(). - enable_iommus_vapic() is simplified - it handles enabling GA log and GAM/SNPAVIC support for IOMMUs. Finally update iommu_ga_log_enable() return value from int to bool as return value is used in boolean context only. Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Reviewed-by: Ankit Soni <Ankit.Soni@amd.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-09-24iommu/amd: Remove ad-hoc checks that are never trueRong Zhang
The state transitions of the AMD IOMMU Initialization State Machine imply that some ad-hoc checks will never be true. In detail, they are only reachable if the previous state transitions have succeeded, but the conditions are only true if the previous state transitions have failed. Therefore, remove the redundant ad-hoc checks. Signed-off-by: Rong Zhang <i@rong.moe> Reviewed-by: Ankit Soni <Ankit.Soni@amd.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-09-24iommu/amd: Disallow implicit START_STATE => IVRS_DETECTED transitionRong Zhang
Xen PV intentionally skips calling amd_iommu_detect() in order to gate IOMMU initialization, as the Xen hypervisor has taken over the hardware. However, due to amd_iommu_prepare() requesting ACPI_FINISHED while setting up APIC, the state machine is still progressed, only to be stopped midway due to the `!amd_iommu_detected' check in early_amd_iommu_init(). This is worthless and fragile. It unnecessarily calls detect_ivrs(), which leads to extra overhead, i.e., getting the IVRS table and accessing the PCI config space. Disallow implicit START_STATE => IVRS_DETECTED transition, so that amd_iommu_prepare() and other paths no longer progress the state machine accidentally. This should also help prevent potential bugs if more housekeeping work is added to amd_iommu_detect() in the future. Signed-off-by: Rong Zhang <i@rong.moe> Reviewed-by: Ankit Soni <Ankit.Soni@amd.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-09-24iommu/amd: Do not request ACS when IOMMU is not going to be initializedRong Zhang
The AMD IOMMU Initialization State Machine has the following state transition diagram (only the very first states are showed, and the `IOMMU_' prefix is omitted): START_STATE | v [0] detect_ivrs() --> NOT_FOUND | ok v IVRS_DETECTED | v [1] amd_iommu_disabled? (amd_iommu=off) --> IOMMU_CMDLINE_DISABLED | no v [2] early_amd_iommu_init() | +-- [3] amd_iommu_detected? (!iommu=off && ...) -+ | yes | +-- ... --> IOMMU_INIT_ERROR <-------------------+ | ok v IOMMU_ACPI_FINISHED | v ... [0] always calls pci_request_acs() as long as there's a valid IVRS table and no Stoney Ridge graphics. This is not optimal as ACS is not required in an [amd_]iommu=off boot. In a normal boot, ACS is requested due to amd_iommu_detect() requesting IVRS_DETECTED. pci_request_acs+0x9/0x18 iommu_go_to_state+0x106/0x1a20 amd_iommu_detect+0x1c/0x50 pci_iommu_alloc+0x26/0x40 mm_core_init+0xa/0x120 start_kernel+0x527/0x7a0 x86_64_start_reservations+0x24/0x30 x86_64_start_kernel+0xd1/0xe0 common_startup_64+0x13e/0x158 This is intended to ensure ACS is requested before the PCI core initialization, or else a !CONFIG_IRQ_REMAP, nointremap or intremap=off boot would be broken. However, in an amd_iommu=off boot, the state machine still requests ACS at the exact same time, as amd_iommu_detect() has nothing to do with amd_iommu_disabled. Even worse, in an iommu=off boot, though amd_iommu_detect() bails out early, ACS is still requested due to amd_iommu_prepare() requesting IOMMU_ACPI_FINISHED, which is called by irq_remapping_prepare() thanks to CONFIG_X86_LOCAL_APIC (always set on X86_64) and CONFIG_IRQ_REMAP (enabled by defconfig), unless nointremap or intremap=off is also passed to cmdline. pci_request_acs+0x9/0x18 iommu_go_to_state+0x106/0x1a20 amd_iommu_prepare+0x15/0x40 irq_remapping_prepare+0x43/0x60 enable_IR_x2apic+0x22/0x190 x86_64_probe_apic+0xa/0x50 apic_intr_mode_init+0x70/0xd0 x86_late_time_init+0x28/0x40 start_kernel+0x6f9/0x7a0 ... In both cases, [2] is still gated due to the [1] or [3] check, so that IOMMU can be disabled per cmdline. Technically, it makes no sense to detect IVRS at all in an [amd_]iommu=off boot or if IOMMU is not supported due to platform settings. This is probably why amd_iommu_detect() bails out before requesting IVRS_DETECTED. Apparently only bailing out there is not sufficient, and the bailing-out paths should really have been parts of the state machine. Clean up the initialization routines by moving the bailing-out paths and [1] to the right place in the state machine (i.e., before [0]), and always requesting IVRS_DETECTED in amd_iommu_detect() to initialize the state machine early and properly. Note that the condition (iommu_detected && !gart_iommu_aperture) is dropped as it's dead code since the relevant code being removed. It meant to detect what commit 6631ee9d0099 ("x86, AMD IOMMU: add dma_ops initialization function") had introduced, but commit d7f077697533 ("x86/amd-iommu: Fall back to GART if initialization fails") has removed it. Later, commit 78013eaadf69 ("x86: remove the IOMMU table infrastructure") ensured AMD IOMMU being detected before Intel's and led to iommu_detected == gart_iommu_aperture in any case. Signed-off-by: Rong Zhang <i@rong.moe> Reviewed-by: Ankit Soni <Ankit.Soni@amd.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-09-11iommu/amd: Remove redundant check in irq_remapping_select()Vasant Hegde
The amd_iommu_irq_remap flag is already validated during irq remapping domain creation (before calling amd_iommu_create_irq_domain()). The duplicate check in irq_remapping_select() is unnecessary and can be removed. Additionally, mark amd_iommu_irq_remap as static. No functional changes. Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-09-07iommu/amd: Fix premature break in init_iommu_one() againVasant Hegde
Commit 283d245468a2 ("iommu/amd: Fix premature break in init_iommu_one()") unintentionally broke older platforms - such as the ASRockRack B550D4-4L - where the BIOS advertises incorrect IOMMU features. Move the HATDis check ahead of the GASup check, and re-introduce the break inside the GASup check to restore correct behavior on affected platforms. This is a short-term fix to resolve the regression. Longer term, we should rework how EFRs are tracked and prioritize the MMIO-advertised EFR over the one reported via IVRS. That requires more extensive changes and will be addressed separately. Fixes: 283d245468a2 ("iommu/amd: Fix premature break in init_iommu_one()") Reported-by: Andreas Juch <andreas@juch.cc> Closes: https://lore.kernel.org/linux-iommu/07b2d390-f7a0-47e2-bc2c-eb0853acf52e@juch.cc/ Tested-by: Andreas Juch <andreas@juch.cc> Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-09-07iommu/amd: Do not reallocate GA log buffers on resumeKarl Mehltretter
Commit c5e1a1eb9279 ("iommu/amd: Simplify and Consolidate Virtual APIC (AVIC) Enablement") moved the GA log allocation from iommu_init_pci() to enable_iommus_vapic(), which is called on every resume. iommu_init_ga_log() assigns iommu->ga_log and iommu->ga_log_tail unconditionally. Each resume therefore replaces the boot-time pointers and leaks both old allocations. The function also uses GFP_KERNEL from a syscore resume callback, where interrupts are disabled and the non-boot CPUs are offline. Return early if both buffers are already allocated. Clear the pointers in free_ga_log() so a partial allocation failure cannot leave ga_log dangling. Fixes: c5e1a1eb9279 ("iommu/amd: Simplify and Consolidate Virtual APIC (AVIC) Enablement") Assisted-by: Claude:claude-opus-5 Signed-off-by: Karl Mehltretter <kmehltretter@gmail.com> Reviewed-by: Vasant Hegde <vasant.hegde@amd.com> Reviewed-by: Ankit Soni <Ankit.Soni@amd.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-08-11Merge branches 'arm/smmu/updates', 'arm/smmu/bindings', 'mediatek', ↵Joerg Roedel
'qualcomm/msm', 'rockchip', 'ti/omap', 'riscv', 'intel/vt-d', 'amd/amd-vi', 'core' and 'typos' into next
2026-08-11iommu/amd: Add SNP page mode 0 supportVasant Hegde
Newer AMD IOMMUs supports DTE[Mode]=0 for SNP-enabled system. This is detected using new feature bit (EFR2[SNP_Page_Mode_0_Sup]). If this feature is enabled, then IOMMU supports: - Passthrough mode (i.e. iommu=pt) - amd_iommu=pgtbl_v2: Forcing Linux DMA-API to use IOMMU v2 page table - Setting up device for SVA mode in the host. Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Reviewed-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Tested-by: Amandeep Kaur Longia <AmandeepKaur.Longia@amd.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-07-21iommu/amd: Bound the early ACPI HID mapPengpeng Hou
The ivrs_acpihid command-line parser appends entries to a fixed four-element early_acpihid_map array. Unlike the sibling IOAPIC and HPET parsers, it does not reject a fifth entry before incrementing the map size. Check the capacity at the common found label before parsing the HID and UID or writing the entry. Fixes: ca3bf5d47cec ("iommu/amd: Introduces ivrs_acpihid kernel parameter") Signed-off-by: Pengpeng Hou <pengpeng@iscas.ac.cn> Reviewed-by: Ankit Soni <Ankit.Soni@amd.com> Signed-off-by: Will Deacon <will@kernel.org>
2026-07-09iommu/amd: Dynamically verify Southbridge IOAPIC via PCI config spaceWei Wang
check_ioapic_information() verifies whether the BIOS has provided a valid device ID for the Southbridge (SB) IOAPIC in the IVRS table. Currently, if the SB IOAPIC entry in the IVRS table does not match a historically hardcoded device ID (00:14.0), interrupt remapping is forcibly disabled. This hardcoded expectation does not scale to newer architectures. For example, recent Hygon Gen 4 servers use 00:0b.0 for the SB IOAPIC, which originally caused the validation to fail and interrupt remapping to be disabled until this device ID was added upstream. Instead of maintaining per-vendor/per-generation hardcoded device IDs, dynamically verify the SB IOAPIC by reading its PCI configuration space. Because the SB IOAPIC is embedded within the FCH (Fusion Controller Hub) and shares its device ID, we can inspect the PCI class code of the given device ID to confirm it is an actual FCH device (a SMBus controller or ISA Bridge). Signed-off-by: Wei Wang <wei.w.wang@hotmail.com> Tested-by: Yongwei Xu <xuyongwei@open-hieco.net> Reviewed-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-07-09iommu/amd: Fix false positive in SB IOAPIC IVRS validationWei Wang
The check_ioapic_information() function is designed to prevent boot hangs by ensuring the Southbridge (SB) IOAPIC is properly mapped in the IVRS table before enabling Interrupt Remapping. Currently, this check passes if *any* enumerated IOAPIC matches the expected SB IOAPIC device ID. If a buggy BIOS incorrectly assigns the SB IOAPIC's device ID to a secondary IOAPIC in the IVRS, while scrambling the true SB IOAPIC's mapping, the check hits a false positive and succeeds. This erroneously enables Interrupt Remapping. Consequently, the IOMMU blocks unmapped interrupts from the actual SB IOAPIC, dropping the system timer and leading to a silent kernel boot hang. Tighten the validation to verify the device ID specifically against the SB IOAPIC by matching their APIC IDs first. This prevents the validation check from being bypassed via device ID aliasing. Fixes: c2ff5cf5294b ("iommu/amd: Work around wrong IOAPIC device-id in IVRS table") Signed-off-by: Wei Wang <wei.w.wang@hotmail.com> Tested-by: Yongwei Xu <xuyongwei@open-hieco.net> Reviewed-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-07-09iommu/amd: Prevent SB IOAPIC from overriding IVRS validation errorsWei Wang
The check_ioapic_information() function validates IOAPICs against the IVRS table to safely disable Interrupt Remapping (IR) if the BIOS provides a broken topology. Currently, the validation loop contains a bug: If an unmapped secondary IOAPIC is encountered, 'ret' is set to false. But if the Southbridge (SB) IOAPIC is enumerated after it in the MADT, the loop overwrites 'ret' to true. This bypasses the validation failure and leaves IR enabled. When devices attached to the unmapped secondary IOAPIC fire interrupts, the IOMMU drops them due to the missing Requestor ID, leading to localized device hangs. Fix this by initializing 'ret' to true and only toggling it to false upon encountering a validation error, ensuring failures are never erased. Fixes: c2ff5cf5294b ("iommu/amd: Work around wrong IOAPIC device-id in IVRS table") Signed-off-by: Wei Wang <wei.w.wang@hotmail.com> Tested-by: Yongwei Xu <xuyongwei@open-hieco.net> Reviewed-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-06-12Merge branches 'apple/dart', 'arm/smmu/updates', 'arm/smmu/bindings', ↵Joerg Roedel
'rockchip', 'verisilicon', 'riscv', 'intel/vt-d', 'amd/amd-vi' and 'core' into next
2026-05-28iommu/amd: Fix premature break in init_iommu_one()Vasant Hegde
In init_iommu_one(), when processing IOMMU EFR attributes, the code checks whether GASUP is enabled. If GASUP is not enabled, the code falls back to legacy guest IR mode and then breaks out of the switch statement. This break incorrectly skips the subsequent initialization steps that follow the GASUP check. These initializations are independent of GASUP support and must always be performed. Fix this by replacing the early break with a conditional else block, ensuring that the XTSUP check is only skipped when GASUP is not available. Fixes: a44092e326d4 ("iommu/amd: Use IVHD EFR for early initialization of IOMMU features") Reported-by: Sudheer Dantuluri <dantuluris@google.com> Tested-by: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com> Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-05-19iommu/amd: Adhere to IVINFO[VASIZE] for address limitsAnkit Soni
ACPI IVRS IVHD’s IVINFO field reports the maximum virtual address size (VASIZE) supported by the IOMMU. The AMD IOMMU driver currently caps this with pagetable level reported by EFR[HATS] when configuring paging domains (hw_max_vasz_lg2). On systems where firmware or VM advertises smaller or different limits, the driver may over-advertise capabilities and create domains outside the hardware’s actual bounds. Reviewed-by: Jason Gunthorpe <jgg@nvidia.com> Reviewed-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Ankit Soni <Ankit.Soni@amd.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-05-11iommu/amd: Add support for Hygon family 18h model 4h IOAPICFu Hao
The SB IOAPIC is on the device 0xb from Hygon family 18h model 4h. Signed-off-by: Fu Hao <fuhao@open-hieco.net> Tested-by: Tingyin Duan <tingyin.duan@gmail.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-05-11iommu/amd: Remove dead code for exclusion ranges in IVMDYu Zhang
Exclusion ranges in IVMD are treated as unity mappings with r&w permissions since commit 0bbe4ced53e3 ("iommu/amd: Fix the overwritten field in IVMD header"), using 1:1 mappings. And IOMMU Exclusion Base & Range Limit Registers (0x0020/0x0028) are actually no longer used. As a result, several definitions and code paths became dead code: - exclusion_start/exclusion_length in struct amd_iommu are never assigned, - iommu_set_exclusion_range() always returns 0 now. - MMIO_EXCL_ENABLE_MASK & MMIO_EXCL_ALLOW_MASK are only used by the removed iommu_set_exclusion_range(). - DEV_ENTRY_EX is no longer set in any DTE. - IOMMU_UNITY_MAP_FLAG_EXCL_RANGE is no longer set, thus the IOMMU_RESV_RESERVED branch in amd_iommu_get_resv_regions() is no longer reachable. Just remove all of the dead code. No functional change. Signed-off-by: Yu Zhang <zhangyu1@linux.microsoft.com> Reviewed-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Reviewed-by: Jacob Pan <jacob.pan@linux.microsoft.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-05-11iommu/amd: Explicitly bail from enable_iommus_vapic() when in legacy modeSean Christopherson
Bail early from enable_iommus_vapic() if IOMMUs are configured for either of the legacy modes, as it's absurdly difficult to see that iommu_ga_log_enable() is guaranteed to fail because iommu_init_ga_log() skips allocating the ga_log. Opportunistically have iommu_ga_log_enable() WARN if it's called without IOMMUs being configured to support AVIC/vAPIC. Cc: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Signed-off-by: Sean Christopherson <seanjc@google.com> Reviewed-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-04-27iommu/amd: Use maximum PPR log buffer size when SNP is enabled on Family 0x19Vasant Hegde
Due to CVE-2023-20585, the PPR log buffer must use the maximum supported size (512K) on Genoa (Family 0x19, model >= 0x10) systems when SNP is enabled, to mitigate a potential security vulnerability. Note that Family 0x19 models below 0x10 (Milan) do not support PPR when SNP is enabled. Hence the PPR log size increase is only applied for model >= 0x10. All other systems continue to use the default PPR log buffer size (8K). Apply the errata fix by making the following changes: - Introduce global new variable (amd_iommu_pprlog_size) to have PPR log buffer size. Adjust variable size for Genoa family. - Extend 'amd_iommu_apply_erratum_snp()' to also set the PPR log buffer size to maximum for Family 0x19 model >= 0x10 when SNP is enabled. - Rename PPR_* macros to make it more readable. Link: https://www.amd.com/en/resources/product-security/bulletin/amd-sb-3016.html Cc: Borislav Petkov <bp@alien8.de> Cc: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Cc: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Tested-by: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-04-27iommu/amd: Use maximum Event log buffer size when SNP is enabled on Family 0x19Vasant Hegde
Due to CVE-2023-20585, the Event log buffer must use the maximum supported size (512K) on Milan/Genoa (Family 0x19) systems when SNP is enabled, to mitigate a potential security vulnerability. All other systems continue to use the default Event log buffer size (8K). Apply the errata fix by making the following changes: * Introduce new global variable (amd_iommu_evtlog_size) to have event log buffer size. Adjust variable size for family 0x19. * Since 'iommu_snp_enable()' must be called after the core IOMMU subsystem is initialized, it cannot be moved to the early init stage. The SNP errata must also be applied after the 'iommu_snp_enable()' check. Therefore, 'alloc_event_buffer()' and 'iommu_enable_event_buffer()' are now called in the IOMMU_ENABLED state, after the errata is applied. * Adjust alloc_event_buffer() and iommu_enable_event_buffer() to handle all IOMMU instances. * Also rename EVT_* macros to make it more readable. Link: https://www.amd.com/en/resources/product-security/bulletin/amd-sb-3016.html Cc: Borislav Petkov <bp@alien8.de> Cc: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Cc: Joerg Roedel <joerg.roedel@amd.com> Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Tested-by: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-03-17iommu/amd: Add NUMA node affinity for IOMMU log buffersLi RongQing
Currently, PPR Log and GA logs for AMD IOMMU are allocated using iommu_alloc_pages_sz(), which does not account for NUMA affinity. This can lead to remote memory access latencies if the memory is allocated on a different node than the IOMMU hardware. Switch to iommu_alloc_pages_node_sz() to ensure that these data structures are allocated on the same NUMA node as the IOMMU device. If the node information is unavailable, it defaults to NUMA_NO_NODE. Signed-off-by: Li RongQing <lirongqing@baidu.com> Reviewed-by: Vasant Hegde <vasant.hegde@amd.com> Reviewed-by: Ankit Soni <Ankit.Soni@amd.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-02-22Convert remaining multi-line kmalloc_obj/flex GFP_KERNEL usesKees Cook
Conversion performed via this Coccinelle script: // SPDX-License-Identifier: GPL-2.0-only // Options: --include-headers-for-types --all-includes --include-headers --keep-comments virtual patch @gfp depends on patch && !(file in "tools") && !(file in "samples")@ identifier ALLOC = {kmalloc_obj,kmalloc_objs,kmalloc_flex, kzalloc_obj,kzalloc_objs,kzalloc_flex, kvmalloc_obj,kvmalloc_objs,kvmalloc_flex, kvzalloc_obj,kvzalloc_objs,kvzalloc_flex}; @@ ALLOC(... - , GFP_KERNEL ) $ make coccicheck MODE=patch COCCI=gfp.cocci Build and boot tested x86_64 with Fedora 42's GCC and Clang: Linux version 6.19.0+ (user@host) (gcc (GCC) 15.2.1 20260123 (Red Hat 15.2.1-7), GNU ld version 2.44-12.fc42) #1 SMP PREEMPT_DYNAMIC 1970-01-01 Linux version 6.19.0+ (user@host) (clang version 20.1.8 (Fedora 20.1.8-4.fc42), LLD 20.1.8) #1 SMP PREEMPT_DYNAMIC 1970-01-01 Signed-off-by: Kees Cook <kees@kernel.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2026-02-21Convert 'alloc_obj' family to use the new default GFP_KERNEL argumentLinus Torvalds
This was done entirely with mindless brute force, using git grep -l '\<k[vmz]*alloc_objs*(.*, GFP_KERNEL)' | xargs sed -i 's/\(alloc_objs*(.*\), GFP_KERNEL)/\1)/' to convert the new alloc_obj() users that had a simple GFP_KERNEL argument to just drop that argument. Note that due to the extreme simplicity of the scripting, any slightly more complex cases spread over multiple lines would not be triggered: they definitely exist, but this covers the vast bulk of the cases, and the resulting diff is also then easier to check automatically. For the same reason the 'flex' versions will be done as a separate conversion. Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2026-02-21treewide: Replace kmalloc with kmalloc_obj for non-scalar typesKees Cook
This is the result of running the Coccinelle script from scripts/coccinelle/api/kmalloc_objs.cocci. The script is designed to avoid scalar types (which need careful case-by-case checking), and instead replace kmalloc-family calls that allocate struct or union object instances: Single allocations: kmalloc(sizeof(TYPE), ...) are replaced with: kmalloc_obj(TYPE, ...) Array allocations: kmalloc_array(COUNT, sizeof(TYPE), ...) are replaced with: kmalloc_objs(TYPE, COUNT, ...) Flex array allocations: kmalloc(struct_size(PTR, FAM, COUNT), ...) are replaced with: kmalloc_flex(*PTR, FAM, COUNT, ...) (where TYPE may also be *VAR) The resulting allocations no longer return "void *", instead returning "TYPE *". Signed-off-by: Kees Cook <kees@kernel.org>
2026-02-11Merge tag 'iommu-updates-v7.0' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/iommu/linux Pull iommu updates from Joerg Roedel: "Core changes: - Rust bindings for IO-pgtable code - IOMMU page allocation debugging support - Disable ATS during PCI resets Intel VT-d changes: - Skip dev-iotlb flush for inaccessible PCIe device - Flush cache for PASID table before using it - Use right invalidation method for SVA and NESTED domains - Ensure atomicity in context and PASID entry updates AMD-Vi changes: - Support for nested translations - Other minor improvements ARM-SMMU-v2 changes: - Configure SoC-specific prefetcher settings for Qualcomm's "MDSS" ARM-SMMU-v3 changes: - Improve CMDQ locking fairness for pathetically small queue sizes - Remove tracking of the IAS as this is only relevant for AArch32 and was causing C_BAD_STE errors - Add device-tree support for NVIDIA's CMDQV extension - Allow some hitless transitions for the 'MEV' and 'EATS' STE fields - Don't disable ATS for nested S1-bypass nested domains - Additions to the kunit selftests" * tag 'iommu-updates-v7.0' of git://git.kernel.org/pub/scm/linux/kernel/git/iommu/linux: (54 commits) iommupt: Always add IOVA range to iotlb_gather in gather_range_pages() iommu/amd: serialize sequence allocation under concurrent TLB invalidations iommu/amd: Fix type of type parameter to amd_iommufd_hw_info() iommu/arm-smmu-v3: Do not set disable_ats unless vSTE is Translate iommu/arm-smmu-v3-test: Add nested s1bypass/s1dssbypass coverage iommu/arm-smmu-v3: Mark EATS_TRANS safe when computing the update sequence iommu/arm-smmu-v3: Mark STE MEV safe when computing the update sequence iommu/arm-smmu-v3: Add update_safe bits to fix STE update sequence iommu/arm-smmu-v3: Add device-tree support for CMDQV driver iommu/tegra241-cmdqv: Decouple driver from ACPI iommu/arm-smmu-qcom: Restore ACTLR settings for MDSS on sa8775p iommu/vt-d: Fix race condition during PASID entry replacement iommu/vt-d: Clear Present bit before tearing down context entry iommu/vt-d: Clear Present bit before tearing down PASID entry iommu/vt-d: Flush piotlb for SVM and Nested domain iommu/vt-d: Flush cache for PASID table before using it iommu/vt-d: Flush dev-IOTLB only when PCIe device is accessible in scalable mode iommu/vt-d: Skip dev-iotlb flush for inaccessible PCIe device without scalable mode rust: iommu: fix `srctree` link warning rust: iommu: fix Rust formatting ...
2026-02-03iommu/amd: serialize sequence allocation under concurrent TLB invalidationsAnkit Soni
With concurrent TLB invalidations, completion wait randomly gets timed out because cmd_sem_val was incremented outside the IOMMU spinlock, allowing CMD_COMPL_WAIT commands to be queued out of sequence and breaking the ordering assumption in wait_on_sem(). Move the cmd_sem_val increment under iommu->lock so completion sequence allocation is serialized with command queuing. And remove the unnecessary return. Fixes: d2a0cac10597 ("iommu/amd: move wait_on_sem() out of spinlock") Tested-by: Srikanth Aithal <sraithal@amd.com> Reported-by: Srikanth Aithal <sraithal@amd.com> Signed-off-by: Ankit Soni <Ankit.Soni@amd.com> Reviewed-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-02-01iommu/amd: Use core's primary handler and set IRQF_ONESHOTSebastian Andrzej Siewior
request_threaded_irq() is invoked with a primary and a secondary handler and no flags are passed. The primary handler is the same as irq_default_primary_handler() so there is no need to have an identical copy. The lack of the IRQF_ONESHOT can be dangerous because the interrupt source is not masked while the threaded handler is active. This means, especially on LEVEL typed interrupt lines, the interrupt can fire again before the threaded handler had a chance to run. Use the default primary interrupt handler by specifying NULL and set IRQF_ONESHOT so the interrupt source is masked until the secondary handler is done. Fixes: 72fe00f01f9a3 ("x86/amd-iommu: Use threaded interupt handler") Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Signed-off-by: Thomas Gleixner <tglx@kernel.org> Link: https://patch.msgid.link/20260128095540.863589-4-bigeasy@linutronix.de
2026-01-18iommu/amd: Always enable GCR3TRPMode when supported.Suravee Suthikulpanit
The GCR3TRPMode feature allows the DTE[GCR3TRP] field to be configured with GPA (instead of SPA). This simplifies the implementation, and is a pre-requisite for nested translation support. Therefore, always enable this feature if available. Reviewed-by: Jason Gunthorpe <jgg@nvidia.com> Reviewed-by: Nicolin Chen <nicolinc@nvidia.com> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2026-01-18iommu/amd: Rename DEV_DOMID_MASK to DTE_DOMID_MASKSuravee Suthikulpanit
Also change the define to use GENMASK_ULL instead. There is no functional change. Reviewed-by: Nicolin Chen <nicolinc@nvidia.com> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com> Reviewed-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2025-12-19amd/iommu: Make protection domain ID functions non-staticSairaj Kodilkar
So that both iommu.c and init.c can utilize them. Also define a new function 'pdom_id_destroy()' to destroy 'pdom_ids' instead of directly calling ida functions. Signed-off-by: Sairaj Kodilkar <sarunkod@amd.com> Reviewed-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2025-12-19amd/iommu: Preserve domain ids inside the kdump kernelSairaj Kodilkar
Currently AMD IOMMU driver does not reserve domain ids programmed in the DTE while reusing the device table inside kdump kernel. This can cause reallocation of these domain ids for newer domains that are created by the kdump kernel, which can lead to potential IO_PAGE_FAULTs Hence reserve these ids inside pdom_ids. Fixes: 38e5f33ee359 ("iommu/amd: Reuse device table for kdump") Signed-off-by: Sairaj Kodilkar <sarunkod@amd.com> Reported-by: Jason Gunthorpe <jgg@nvidia.com> Reviewed-by: Vasant Hegde <vasant.hegde@amd.com> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2025-12-06Merge tag 'tsm-for-6.19' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/devsec/tsm Pull PCIe Link Encryption and Device Authentication from Dan Williams: "New PCI infrastructure and one architecture implementation for PCIe link encryption establishment via platform firmware services. This work is the result of multiple vendors coming to consensus on some core infrastructure (thanks Alexey, Yilun, and Aneesh!), and three vendor implementations, although only one is included in this pull. The PCI core changes have an ack from Bjorn, the crypto/ccp/ changes have an ack from Tom, and the iommu/amd/ changes have an ack from Joerg. PCIe link encryption is made possible by the soup of acronyms mentioned in the shortlog below. Link Integrity and Data Encryption (IDE) is a protocol for installing keys in the transmitter and receiver at each end of a link. That protocol is transported over Data Object Exchange (DOE) mailboxes using PCI configuration requests. The aspect that makes this a "platform firmware service" is that the key provisioning and protocol is coordinated through a Trusted Execution Envrionment (TEE) Security Manager (TSM). That is either firmware running in a coprocessor (AMD SEV-TIO), or quasi-hypervisor software (Intel TDX Connect / ARM CCA) running in a protected CPU mode. Now, the only reason to ask a TSM to run this protocol and install the keys rather than have a Linux driver do the same is so that later, a confidential VM can ask the TSM directly "can you certify this device?". That precludes host Linux from provisioning its own keys, because host Linux is outside the trust domain for the VM. It also turns out that all architectures, save for one, do not publish a mechanism for an OS to establish keys in the root port. So "TSM-established link encryption" is the only cross-architecture path for this capability for the foreseeable future. This unblocks the other arch implementations to follow in v6.20/v7.0, once they clear some other dependencies, and it unblocks the next phase of work to implement the end-to-end flow of confidential device assignment. The PCIe specification calls this end-to-end flow Trusted Execution Environment (TEE) Device Interface Security Protocol (TDISP). In the meantime, Linux gets a link encryption facility which has practical benefits along the same lines as memory encryption. It authenticates devices via certificates and may protect against interposer attacks trying to capture clear-text PCIe traffic. Summary: - Introduce the PCI/TSM core for the coordination of device authentication, link encryption and establishment (IDE), and later management of the device security operational states (TDISP). Notify the new TSM core layer of PCI device arrival and departure - Add a low level TSM driver for the link encryption establishment capabilities of the AMD SEV-TIO architecture - Add a library of helpers TSM drivers to use for IDE establishment and the DOE transport - Add skeleton support for 'bind' and 'guest_request' operations in support of TDISP" * tag 'tsm-for-6.19' of git://git.kernel.org/pub/scm/linux/kernel/git/devsec/tsm: (23 commits) crypto/ccp: Fix CONFIG_PCI=n build virt: Fix Kconfig warning when selecting TSM without VIRT_DRIVERS crypto/ccp: Implement SEV-TIO PCIe IDE (phase1) iommu/amd: Report SEV-TIO support psp-sev: Assign numbers to all status codes and add new ccp: Make snp_reclaim_pages and __sev_do_cmd_locked public PCI/TSM: Add 'dsm' and 'bound' attributes for dependent functions PCI/TSM: Add pci_tsm_guest_req() for managing TDIs PCI/TSM: Add pci_tsm_bind() helper for instantiating TDIs PCI/IDE: Initialize an ID for all IDE streams PCI/IDE: Add Address Association Register setup for downstream MMIO resource: Introduce resource_assigned() for discerning active resources PCI/TSM: Drop stub for pci_tsm_doe_transfer() drivers/virt: Drop VIRT_DRIVERS build dependency PCI/TSM: Report active IDE streams PCI/IDE: Report available IDE streams PCI/IDE: Add IDE establishment helpers PCI: Establish document for PCI host bridge sysfs attributes PCI: Add PCIe Device 3 Extended Capability enumeration PCI/TSM: Establish Secure Sessions and Link Encryption ...
2025-12-05Merge tag 'soc-drivers-6.19' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc Pull SoC driver updates from Arnd Bergmann: "This is the first half of the driver changes: - A treewide interface change to the "syscore" operations for power management, as a preparation for future Tegra specific changes - Reset controller updates with added drivers for LAN969x, eic770 and RZ/G3S SoCs - Protection of system controller registers on Renesas and Google SoCs, to prevent trivially triggering a system crash from e.g. debugfs access - soc_device identification updates on Nvidia, Exynos and Mediatek - debugfs support in the ST STM32 firewall driver - Minor updates for SoC drivers on AMD/Xilinx, Renesas, Allwinner, TI - Cleanups for memory controller support on Nvidia and Renesas" * tag 'soc-drivers-6.19' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc: (114 commits) memory: tegra186-emc: Fix missing put_bpmp Documentation: reset: Remove reset_controller_add_lookup() reset: fix BIT macro reference reset: rzg2l-usbphy-ctrl: Fix a NULL vs IS_ERR() bug in probe reset: th1520: Support reset controllers in more subsystems reset: th1520: Prepare for supporting multiple controllers dt-bindings: reset: thead,th1520-reset: Add controllers for more subsys dt-bindings: reset: thead,th1520-reset: Remove non-VO-subsystem resets reset: remove legacy reset lookup code clk: davinci: psc: drop unused reset lookup reset: rzg2l-usbphy-ctrl: Add support for RZ/G3S SoC reset: rzg2l-usbphy-ctrl: Add support for USB PWRRDY dt-bindings: reset: renesas,rzg2l-usbphy-ctrl: Document RZ/G3S support reset: eswin: Add eic7700 reset driver dt-bindings: reset: eswin: Documentation for eic7700 SoC reset: sparx5: add LAN969x support dt-bindings: reset: microchip: Add LAN969x support soc: rockchip: grf: Add select correct PWM implementation on RK3368 soc/tegra: pmc: Add USB wake events for Tegra234 amba: tegra-ahb: Fix device leak on SMMU enable ...
2025-12-02iommu/amd: Report SEV-TIO supportAlexey Kardashevskiy
The SEV-TIO switch in the AMD BIOS is reported to the OS via the IOMMU Extended Feature 2 register (EFR2), bit 1. Add helper to parse the bit and report the feature presence. Signed-off-by: Alexey Kardashevskiy <aik@amd.com> Link: https://patch.msgid.link/20251202024449.542361-4-aik@amd.com Acked-by: Joerg Roedel <joerg.roedel@amd.com> Reviewed-by: Vasant Hegde <vasant.hegde@amd.com> Acked-by: Tom Lendacky <thomas.lendacky@amd.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com>
2025-11-14syscore: Pass context data to callbacksThierry Reding
Several drivers can benefit from registering per-instance data along with the syscore operations. To achieve this, move the modifiable fields out of the syscore_ops structure and into a separate struct syscore that can be registered with the framework. Add a void * driver data field for drivers to store contextual data that will be passed to the syscore ops. Acked-by: Rafael J. Wysocki (Intel) <rafael@kernel.org> Signed-off-by: Thierry Reding <treding@nvidia.com>
2025-11-13iommu/amd: Fix pci_segment memleak in alloc_pci_segment()Jinhui Guo
Fix a memory leak of struct amd_iommu_pci_segment in alloc_pci_segment() when system memory (or contiguous memory) is insufficient. Fixes: 04230c119930 ("iommu/amd: Introduce per PCI segment device table") Fixes: eda797a27795 ("iommu/amd: Introduce per PCI segment rlookup table") Fixes: 99fc4ac3d297 ("iommu/amd: Introduce per PCI segment alias_table") Cc: stable@vger.kernel.org Signed-off-by: Jinhui Guo <guojinhui.liam@bytedance.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2025-09-26Merge branches 'apple/dart', 'ti/omap', 'riscv', 'intel/vt-d' and ↵Joerg Roedel
'amd/amd-vi' into next
2025-09-13iommu/amd: Fix alias device DTE settingVasant Hegde
Commit 7bea695ada0 restructured DTE flag handling but inadvertently changed the alias device configuration logic. This may cause incorrect DTE settings for certain devices. Add alias flag check before calling set_dev_entry_from_acpi(). Also move the device iteration loop inside the alias check to restrict execution to cases where alias devices are present. Fixes: 7bea695ada0 ("iommu/amd: Introduce struct ivhd_dte_flags to store persistent DTE flags") Cc: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Signed-off-by: Vasant Hegde <vasant.hegde@amd.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2025-09-05iommu/amd: Fix ivrs_base memleak in early_amd_iommu_init()Zhen Ni
Fix a permanent ACPI table memory leak in early_amd_iommu_init() when CMPXCHG16B feature is not supported Fixes: 82582f85ed22 ("iommu/amd: Disable AMD IOMMU if CMPXCHG16B feature is not supported") Cc: stable@vger.kernel.org Signed-off-by: Zhen Ni <zhen.ni@easystack.cn> Reviewed-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Link: https://lore.kernel.org/r/20250822024915.673427-1-zhen.ni@easystack.cn Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2025-09-05iommu/amd: Skip enabling command/event buffers for kdumpAshish Kalra
After a panic if SNP is enabled in the previous kernel then the kdump kernel boots with IOMMU SNP enforcement still enabled. IOMMU command buffers and event buffer registers remain locked and exclusive to the previous kernel. Attempts to enable command and event buffers in the kdump kernel will fail, as hardware ignores writes to the locked MMIO registers as per AMD IOMMU spec Section 2.12.2.1. Skip enabling command buffers and event buffers for kdump boot as they are already enabled in the previous kernel. Reviewed-by: Vasant Hegde <vasant.hegde@amd.com> Tested-by: Sairaj Kodilkar <sarunkod@amd.com> Signed-off-by: Ashish Kalra <ashish.kalra@amd.com> Link: https://lore.kernel.org/r/576445eb4f168b467b0fc789079b650ca7c5b037.1756157913.git.ashish.kalra@amd.com Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2025-09-05iommu/amd: Reuse device table for kdumpAshish Kalra
After a panic if SNP is enabled in the previous kernel then the kdump kernel boots with IOMMU SNP enforcement still enabled. IOMMU device table register is locked and exclusive to the previous kernel. Attempts to copy old device table from the previous kernel fails in kdump kernel as hardware ignores writes to the locked device table base address register as per AMD IOMMU spec Section 2.12.2.1. This causes the IOMMU driver (OS) and the hardware to reference different memory locations. As a result, the IOMMU hardware cannot process the command which results in repeated "Completion-Wait loop timed out" errors and a second kernel panic: "Kernel panic - not syncing: timer doesn't work through Interrupt-remapped IO-APIC". Reuse device table instead of copying device table in case of kdump boot and remove all copying device table code. Reviewed-by: Vasant Hegde <vasant.hegde@amd.com> Tested-by: Sairaj Kodilkar <sarunkod@amd.com> Signed-off-by: Ashish Kalra <ashish.kalra@amd.com> Link: https://lore.kernel.org/r/3a31036fb2f7323e6b1a1a1921ac777e9f7bdddc.1756157913.git.ashish.kalra@amd.com Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2025-09-05iommu/amd: Add support to remap/unmap IOMMU buffers for kdumpAshish Kalra
After a panic if SNP is enabled in the previous kernel then the kdump kernel boots with IOMMU SNP enforcement still enabled. IOMMU completion wait buffers (CWBs), command buffers and event buffer registers remain locked and exclusive to the previous kernel. Attempts to allocate and use new buffers in the kdump kernel fail, as hardware ignores writes to the locked MMIO registers as per AMD IOMMU spec Section 2.12.2.1. This results in repeated "Completion-Wait loop timed out" errors and a second kernel panic: "Kernel panic - not syncing: timer doesn't work through Interrupt-remapped IO-APIC" The list of MMIO registers locked and which ignore writes after failed SNP shutdown are mentioned in the AMD IOMMU specifications below: Section 2.12.2.1. https://docs.amd.com/v/u/en-US/48882_3.10_PUB Reuse the pages of the previous kernel for completion wait buffers, command buffers, event buffers and memremap them during kdump boot and essentially work with an already enabled IOMMU configuration and re-using the previous kernel’s data structures. Reusing of command buffers and event buffers is now done for kdump boot irrespective of SNP being enabled during kdump. Re-use of completion wait buffers is only done when SNP is enabled as the exclusion base register is used for the completion wait buffer (CWB) address only when SNP is enabled. Reviewed-by: Vasant Hegde <vasant.hegde@amd.com> Tested-by: Sairaj Kodilkar <sarunkod@amd.com> Signed-off-by: Ashish Kalra <ashish.kalra@amd.com> Link: https://lore.kernel.org/r/ff04b381a8fe774b175c23c1a336b28bc1396511.1756157913.git.ashish.kalra@amd.com Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2025-08-15iommu/amd: Avoid stack buffer overflow from kernel cmdlineKees Cook
While the kernel command line is considered trusted in most environments, avoid writing 1 byte past the end of "acpiid" if the "str" argument is maximum length. Reported-by: Simcha Kosman <simcha.kosman@cyberark.com> Closes: https://lore.kernel.org/all/AS8P193MB2271C4B24BCEDA31830F37AE84A52@AS8P193MB2271.EURP193.PROD.OUTLOOK.COM Fixes: b6b26d86c61c ("iommu/amd: Add a length limitation for the ivrs_acpihid command-line parameter") Signed-off-by: Kees Cook <kees@kernel.org> Reviewed-by: Ankit Soni <Ankit.Soni@amd.com> Link: https://lore.kernel.org/r/20250804154023.work.970-kees@kernel.org Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
2025-07-15iommu/amd: Refactor AMD IOMMU debugfs initial setupDheeraj Kumar Srivastava
Rearrange initial setup of AMD IOMMU debugfs to segregate per IOMMU setup and setup which is common for all IOMMUs. This ensures that common debugfs paths (introduced in subsequent patches) are created only once instead of being created for each IOMMU. With the change, there is no need to use lock as amd_iommu_debugfs_setup() will be called only once during AMD IOMMU initialization. So remove lock acquisition in amd_iommu_debugfs_setup(). Signed-off-by: Dheeraj Kumar Srivastava <dheerajkumar.srivastava@amd.com> Reviewed-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> Reviewed-by: Vasant Hegde <vasant.hegde@amd.com> Link: https://lore.kernel.org/r/20250702093804.849-2-dheerajkumar.srivastava@amd.com Signed-off-by: Will Deacon <will@kernel.org>