| Age | Commit message (Collapse) | Author |
|---|
|
git://git.kernel.org/pub/scm/linux/kernel/git/iommu/linux
Pull IOMMU updates from Joerg Roedel:
"Core:
- Detect scatterlist length overflows and finalize deferred domain
attachments before mapping MSI pages
- Consolidate x86 MSI reserved-region handling and clean up internal
DMA and SVA helpers
AMD IOMMU:
- Add IOMMU PerfOpt support, enabled by amdgpu for supported
integrated GPUs in identity domains
- Restrict forced identity mapping to GPUs that require it
- Fix kdump device-table allocation and reuse, ATS configuration
error handling, and an interrupt-remapping allocation leak
- Clean up device probing and initialization
- Add Vasant Hegde as an AMD IOMMU maintainer
ARM SMMUv2:
- Device-tree binding updates for Qualcomm Maili and Nord SoCs
ARM SMMUv3:
- Work around hardware integration issue on Google Tensor G5 SoCs
where all DMA transactions appear as instruction fetches
- Allow command queue size to be configured on the kernel
command-line
- Reduce queue sizes when running as a kdump kernel
- Fix some pre-existing issues in the driver and kunit tests reported
by Sashiko
- Fix handling of Arm hardware errata relating to the contiguous hint
when using SVA
- Rework the TLB invalidation logic in preparation for enabling
iommu-pt in future
- Minor cleanups in preparation for improving support for kdump
Apple DART:
- Support device-tree DMA aperture specifications
Broadcom:
- Add BCM2712 IOMMU support for Raspberry Pi 5 display and multimedia
devices, including the generic page-table format and shared TLB
cache management
Hyper-V:
- Add a para-virtualized guest IOMMU driver with guest-managed
stage-1 page tables and page-selective IOTLB invalidation
Intel VT-d:
- Enhanced IQE fault handling for batched multiple descriptors,
- Optimization of page-table level calculation in
compute_vasz_lg2_ss(),
- Fixes for several pre-existing issues reported by Sashiko,
- General cleanup and refinement
RISC-V:
- Report cache-coherency capability for coherent devices, enabling
VFIO/iommufd use
Samsung Exynos:
- Support SysMMUs without BLOCK mode, fixing their enable sequence
and TLB invalidation
- Improve v7 fault diagnostics with transaction information"
* tag 'iommu-updates-v7.4' of git://git.kernel.org/pub/scm/linux/kernel/git/iommu/linux: (69 commits)
MAINTAINERS: Make Vasant Hegde AMD IOMMU maintainer
iommu/hyperv: Make hv_iommu_device static
iommu/hyperv: Add missing PCI dependency for the guest IOMMU driver
iommu/amd: Don't allocate kdump device table from DMA32
iommu/amd: Program device table when kdump reuse fails
iommu/dma: Clean up redundant MSI cookie check
iommu/dma: Catch scatterlist length overflows
iommu/hyperv: Add page-selective IOTLB flush support
iommu/hyperv: Add para-virtualized IOMMU support for Hyper-V guest
iommu/x86: Add architectural MSI reserved region helper
Drivers: hv: Add logical device ID registry for vPCI devices
hyperv: Introduce new hypercall interfaces used by Hyper-V guest IOMMU
iommu: Make iommu-dma helpers private
iommu/arm-smmu-v3: Don't rb_erase() a never-inserted stream node
iommu/arm-smmu-v3: Add Q_POS() macro
iommu/arm-smmu-v3: Add arm_smmu_attach_release()
iommu/arm-smmu-v3: Disable the impl before disabling the SMMU on shutdown
iommu/arm-smmu-v3: Populate the tlbi at the top of the call chain
iommu/arm-smmu-v3: Precompute the invalidation commands
iommu/arm-smmu-v3: Keep track in arm_smmu_invs if range invalidation is used
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/mszyprowski/linux
Pull dma-mapping updates from Marek Szyprowski:
- several SWIOTLB initialization cleanups (Aneesh Kumar K.V, Francesco
Valla)
- exact page allocation for DMA pages (Qingfang Deng)
- tools/dma build fixes (Sasha Levin)
- minor fixes for various corner cases (Marek Szyprowski, Leon
Romanovsky) and code cleanup (John Garry)
* tag 'dma-mapping-7.4-2026-10-21' of git://git.kernel.org/pub/scm/linux/kernel/git/mszyprowski/linux:
dma-debug: Skip cacheline tracking for MMIO mappings
tools/dma: fix building from the top level with O=
tools/dma: fix building with O=
dma: swiotlb: Track whether the pool size was explicitly set
dma: swiotlb: Consolidate slab rounding
dma: swiotlb: Rename swiotlb_size_or_default()
dma: swiotlb: use KiB in restricted pool allocation log
dma-direct: Restore arch_dma_alloc() for the DMA_ATTR_NO_KERNEL_MAPPING case
MAINTAINERS: Add files to the DMA MAPPING HELPERS entry
dma-mapping: use exact allocation for DMA pages
dma-mapping: rename dma_opt_mapping_size()
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/jgg/iommufd
Pull iommufd updates from Jason Gunthorpe:
"Several bug fixes, probably found by AI based on the emails:
- Several edge cases in the self test
- Reject malformed invalidation commands in case the VMM let them
through
- Rework how the invalidation interface works to the drivers to be
more robust and fix several Sashiko issues
- Correct several error unwind paths
- UAF during eventq file release
- Reject more malformed usage of the dirty tracking by the VMM"
* tag 'for-linus-iommufd' of git://git.kernel.org/pub/scm/linux/kernel/git/jgg/iommufd:
iommupt: Do not allow PTEs to be split when dirty logging
iommufd/iova_bitmap: Ensure that iova_bitmap_set() handles the full IOVA
iommufd: Reject a zero-length dirty bitmap request
iommufd/iova_bitmap: Fail GET_DIRTY_BITMAP when a bitmap page cannot be pinned
iommufd: Fix event queue release use-after-free
iommufd: Keep dmabuf PFNs MMIO when filling another domain
iommufd: Keep reserved IOVA ranges when a replace within one IOAS fails
iommufd: Fix iommufd_hw_capabilities reference in kernel-doc
iommu/vt-d: Convert nested cache invalidation to the core array loop
iommu/arm-smmu-v3-iommufd: Convert cache invalidation to the core array loop
iommufd/selftest: Convert cache invalidation mocks to the core array loop
iommufd: Iterate the cache invalidation array in the core
iommu/arm-smmu-v3-iommufd: Reject unsupported bits in invalidation commands
iommufd/selftest: Skip when the mock device is not there
iommufd/selftest: Return dmabuf fd from IOMMU_TEST_OP_DMABUF_GET again
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm
Pull ACPI support updates from Rafael Wysocki:
"The most significant change here is the rework of the core ACPI device
enumeration code to avoid initializing ACPI power management
prematurely for devices with unmet dependencies. This essentially is a
problem fix, but it requires taking PCI device enumeration ordering
into account among other things, so it is far from straightforward.
There was an attempt to do this in the 7.3 time frame that had to be
reverted, so this time it is done much more carefully.
There is also an ACPI thermal control update rearranging the code to
register thermal class cooling devices under specific parents, which
is visible in sysfs. This helps to figure out what is represented by
those devices much more easily, but it also allows the binding of
cooling devices to trip points in thermal zones to be adjusted so it
need not rely on specific cooling device data usage in thermal
drivers.
From the number of commits perspective, the most visible part is the
one replacing acpi_get_first_physical_node() with
acpi_bus_get_primary_device() everywhere and dropping the former that
was inherently racy and generally unsafe to use. Fortunately, the
maintainers involved responded timely and all of that could be done
within one development cycle.
Another quite visible part is Dave Jiang's series fixing multiple
issues reported by Sashiko in the ACPI APEI support code with a PCI
AER improvement on top. It took several iterations of this to make
Sashiko happy and the code in question should be much more robust
going forward.
One other thing worth mentioning separately is that the ACPI bus type
is now going to be registered as a "companion" one and it will not be
possible to register a driver binding directly to a struct
acpi_device. All of the existing users of those drivers in the tree
were converted to platform drivers in the previous cycles, so this is
just a final step in a longer journey.
Apart from the above, there are many fixes, mostly in drivers, with a
noticeable exception of a fix in the code binding devices to their
ACPI companions, new IOVT IOMMU support in ACPICA, new _Ixx object
support for suspend-to-idle on x86, multiple new quirks, mostly
related to unused ACPI power resources and backlight handling, and
assorted cleanups.
Specifics:
- Make the core ACPI device enumeration code stop attempting to
initialize ACPI power management for devices before they are known
to be present and functional (Rafael Wysocki)
- Simplify and clean up the core ACPI device enumeration code on top
of the previous adjustment and reduce debug noise coming from it
(Rafael Wysocki)
- Fix locking in the code binding devices to their ACPI companions,
prevent it from replacing a valid ACPI companion pointer with NULL
incidentally, improve its messaging (including debug noise
reduction), and reduce its overhead by making it skip the cases in
which no ACPI companion may exist early (Rafael Wysocki)
- Reduce runtime memory footprint of struct acpi_device by
rearranging fields in it (Andy Shevchenko)
- Update the driver core to allow bus types without drivers to be
registered as "companion bus types" without driver-related sysfs
attributes and register the ACPI bus type as a companion one
(Rafael Wysocki)
- Use strscpy() instead of strcpy() in the ACPI sysfs interface
(Ruslan Vagner)
- Remove __meminitdata from hmat_adist_nb declaration preserving it
even when CONFIG_MEMORY_HOTPLUG is not set to prevent notifier
chain corruption in some cases (Alejandro Lucero)
- Prevent a NULL pointer dereference when attempting to read the
table length after a mapping bookkeeping failure during an initrd
override invocation with permanent table mappings enabled (Slavin
Liu)
- Use for_each_populated_zone() in get_node_num() in the MRRM table
pareser (Sang-Heon Jeon)
- Use vsnprintf() in acpi_os_vprintf() to prevent memory corruption
resulting from overly long diagnostic messages that may not fit
into a static buffer used by ACPICA and may be produced by a
malformed ACPI table (Vlatko Kosturjak)
- Ignore leading root scope prefix in string _UID match to allow AMD
IOMMU matching to work as expected in some cases (Mario
Limonciello)
- Adjust message printing macros for ACPI objects to use dev_fmt()
for more consistency with the analogous dev_*() macros (Rafael
Wysocki)
- Add _Ixx GPE wake indicator support to the x86 suspend-to-idle
handling code (Saranya Gopal)
- Clear driver_data before free in the ACPI PCI Link driver to
prevent a dangling pointer from being used later (Jiale Yao)
- Accumulate the queued flag across error sections of a CPER record
in ghes_do_proc() to prevent it from being cleared by a later
section or entry that does not queue memory-failure work (Breno
Leitao)
- Map a raw PCI AER Capability image field by field to prevent
information loss from occurring while processing it (Dave Jiang)
- Update a stale driver source reference in the ACPI APEI EINJ driver
documentation (Prabhakar Pujeri)
- Reject CPER records with an out-of-range error_data_length or an
error status block length wrapping a u32 in the EFI CPER driver
(Dave Jiang)
- Address multiple assorted issues in the ACPI extlog driver (Dave
Jiang)
- Address two issues related to CXL in the ACPI APEI GHES driver
(Dave Jiang)
- Read only validated fields in cper_mem_err_pack() (Dave Jiang)
- Validate memory error section length before payload access, prevent
writes past array bounds from occurring and sanitize software
metadata in the ACPI APEI GHES driver (Dave Jiang)
- Make cxl_cper_handle_prot_err() static (Dave Jiang)
- Add IOVT IOMMU support to ACPICA (Bibo Mao)
- Fix PCC OperationRegion command offsets in ACPICA (Sudeep Holla)
- Fix memory leak in acpi_ds_create_field() related to the handling
of internal_pcc_buffer allocations (Breno Leitao)
- Fix memory leak due to leftover devm_kcalloc() argument in the ACPI
fan driver (Rafael Wysocki)
- Fix backlight unregistration ordering in the ACPI video bus device
driver (Rafael Wysocki)
- Update the thermal core to allow the registration of thermal class
cooling devices under specific parents and convert the ACPI fan,
ACPI processor, and ACPI video bus device drivers to do so (Rafael
Wysocki)
- Adjust the ACPI thermal zone driver to use thermal class cooling
device parent information for binding cooling devices to trip
points in thermal zones (Rafael Wysocki)
- Use cooling device data that is a better match for a given driver
instead of a struct acpi_device pointer that had to be used for
this purpose previously in the ACPI fan and processor drivers
(Rafael Wysocki)
- Fix acpi_thermal_guess_offset() in the ACPI thermal zone driver for
thermal zone devices with a 273.0°K offset (Hans de Goede)
- Define a free for ACPI objects allocated by ACPICA helpers and use
it to simplify AML error handling in the ACPI fan driver (Rafael
Wysocki)
- Switch over the users of acpi_get_first_physical_node(), that was
inherently racy and therefore not really safe to use, to use its
replacement acpi_bus_get_primary_device(), and eliminate the former
(Rafael Wysocki)
- Add !CONFIG_ACPI stubs for acpi_find_child_device() and
acpi_device_uid() (Mario Limonciello and Robin Murphy)
- Fix bogus cycle count values on systems without _BIX in the ACPI
battery driver (Armin Wolf)
- Fix NULL pointer dereference on allocation failure and report
relative state of charge as CAPACITY in the ACPI SBS driver (Itay
Shem-tov and Guangshuo Li)
- Add lid_init_state quirk for Razer Blade Stealth 13 (Early 2020)
RZ09-0310 to the ACPI button driver (Ian Gallagher)
- Add new quirks to force the native backlight on Dell OptiPlex AIO
7410 35W and Acer Nitro AN515-58 in the ACPI video bus device
driver (Chitransh Mandhaniya and Dirga Yuza)
- Fix the shared memory signature handling in the ACPI PCC address
space handler, free the PCC mailbox channel on operation region
deactivation, and avoid unnecessary repeated computations of the
PCC operation region command timeout (Sudeep Holla)
- Fix a log message typo in the ACPI NUMA support code (Hemanth
Selam)
- Zero-initialize capability query result in the ACPI Platform
Firmware Runtime Update and Telemetry driver to avoid disclosure of
kernel stack data to user space (Karl Mehltretter)
- Replace the remaining built-in/module check for CONFIG_ACPI_WMI in
a header file with IS_ENABLED() macro (Lalit Shankar Chowdhury)
- Retry probing battery docks after hotplug in the ACPI docking
station driver to avoid missing ACPI add notifications (Vibhor
Goel)
- Detach device after resource transition failure in
pnp_device_probe() to avoid blocking later manual PNP configuration
attempts (Myeonghun Pak)"
* tag 'acpi-7.4-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm: (90 commits)
ACPI: PM: Add Acer Aspire A515-53G to dmi_leave_unused_power_resources_on[]
ACPI: video: force native backlight on Dell OptiPlex AIO 7410 35W
ACPI: PM: sleep: Drop unused sleep_no_lps0 define
ACPI: PM: Add unused power resource quirk for Casper Excalibur G900
ACPI: dock: Retry probing battery docks after hotplug
ACPI: PCI: pci_link: clear driver_data before free
ACPI: APEI: accumulate the queued flag across error sections
ACPICA: actbl2.h: Add bit constants of the Flags field in the IOVT IOMMU structure
ACPI: bus: Drop acpi_get_first_physical_node()
efi/dev-path-parser: Switch to use acpi_bus_get_primary_device()
ALSA: hda: tas2781: Switch to use acpi_bus_get_primary_device()
ALSA: hda: cs35l41: Switch to use acpi_bus_get_primary_device()
iommu/arm-smmu-v3: Switch to use acpi_bus_get_primary_device()
PCI/AER: Map a raw AER Capability image field by field
ACPI: NUMA: fix typo in log message
ACPI: Add stub for acpi_find_child_device()
driver core/ACPI: Introduce companion_bus_register()
ACPI: PCC: Cache OpRegion command timeout
ACPI: PCC: Free channel on OpRegion deactivation
ACPI: PCC: Preserve shared memory signature in OpRegion handler
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux
Pull s390 updates from Heiko Carstens:
- Add support for sclp-vt220 terminal resize events e.g. for QEMU
- Warn if kernel command line contains non-printable EBCDIC characters
- Improve show_code() disassembly output by using reliable markers for
the instruction address and faulting instruction
- Rework exception handling to require only a single exception table
entry per faulting instruction: convert all inline assemblies from
EX_TABLE to the new EX_TABLE_INSN macros. The previous exception
handling was error prone since it usually required two EX_TABLE
entries: one pointing at the instruction, and one pointing behind it
- Disallow kprobes on instructions with exception table entries to
avoid exception handlers decoding a breakpoint instead of the
original instruction
- Preemptible percpu ops changes:
- Simplify percpu_entry() and percpu_exit() by encoding the percpu
base, offset, and pointer registers into the lowcore marker,
allowing percpu_exit() to update the percpu pointer on CPU
migration without comparing the interrupted instruction
- Convert this_cpu_cmpxchg(), this_cpu_xchg(), and
this_cpu_cmpxchg128() to preemptible percpu ops; provide
this_cpu_try_cmpxchg()
- Emulate virtual timer in appldata via delayed work and remove the
s390 specific vtimer infrastructure
- Add lock contention tracepoints to rwlock slowpath
- Fix incorrect return address in signal frame setup
- Fix cpum_cf CPU hotplug handling
- Convert various __get_free_pages() / get_zeroed_page() users to
kmalloc() / kzalloc()
- PCI:
- Improve RPCIT instruction tracing
- Fix double-free, use-after-free, and resource leaks in zPCI MSI
and IRQ setup / cleanup paths
- Crypto and zcrypt:
- Improve reply message and length verification checks for CCA,
EP11, and ICA messages to prevent out-of-bounds reads
- Add missing array_index_nospec() for domain index uses in zcrypt
to prevent speculative execution bypass
- Fix out-of-bounds read in cca_query_crypto_facility()
- Various other small fixes and improvements
* tag 's390-7.4-1' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux: (102 commits)
s390/signal: Cleanup coding style
s390/signal: Fix incorrect return address in signal frame
s390/probes: Fix wrong mnemonic comment for clgib
s390/probes: Add missing fixup for bctgr and bic
s390/probes: Fix wrong opcode for cgrb fixup
s390: Select ARCH_MEMORY_ORDER_TSO
s390/mm: Cleanup pXXp_flush_lazy() routines
s390/pci: Add function handle to RPCIT insn trace
s390/pci: Use 32-bit fh outside of inline asm
s390/pci: Adjust alignment in insn trace
s390/sclp: Remove redundant memset after memblock_alloc_low()
s390/entry: Save lowcore on invalid kernel stack
s390/entry: Last breaking event vs invalid kernel stack
s390/percpu: ImplemenHeiko Carstenst this_cpu_try_cmpxchg()
s390/percpu: Drop unused output register from arch_this_cpu_to_op()
s390/percpu: Drop unused output register from arch_this_cpu_add()
s390/percpu: Use asm_inline for arch_this_cpu_add()
s390/extable: Remove EX_TABLE macros and fallback extable search
s390/bpf: Simplify exception table entry handling
s390/sclp: Convert inline assembly to EX_TABLE_INSN
...
|
|
'hyper-v', 'samsung/exynos', 'riscv', 'broadcom', 'intel/vt-d', 'amd/amd-vi' and 'core' into next
|
|
Merge changes related to ACPI support in device drivers for 7.4:
- Switch over the users of acpi_get_first_physical_node(), that
was inherently racy and therefore not really safe to use, to use
its replacement acpi_bus_get_primary_device(), and eliminate the
former (Rafael Wysocki)
- Add !CONFIG_ACPI stubs for acpi_find_child_device() and
acpi_device_uid() (Mario Limonciello and Robin Murphy)
* acpi-driver:
ACPI: bus: Drop acpi_get_first_physical_node()
efi/dev-path-parser: Switch to use acpi_bus_get_primary_device()
ALSA: hda: tas2781: Switch to use acpi_bus_get_primary_device()
ALSA: hda: cs35l41: Switch to use acpi_bus_get_primary_device()
iommu/arm-smmu-v3: Switch to use acpi_bus_get_primary_device()
ACPI: Add stub for acpi_find_child_device()
ACPI: bus: Provide !ACPI stub for acpi_device_uid()
ALSA: hda: aw88399: Switch to use acpi_bus_get_primary_device()
ASoC: loongson: Switch to use acpi_bus_get_primary_device()
ASoC: intel: boards: Switch to use acpi_bus_get_primary_device()
ASoC: amd: acp3x-es83xx: Switch to use acpi_bus_get_primary_device()
ASoC: amd: acp-es8336: Switch to use acpi_bus_get_primary_device()
thunderbolt: Switch to use acpi_bus_get_primary_device()
platform/x86: apple-gmux: Switch to use acpi_bus_get_primary_device()
platform/x86: x86-android-tablets: Use acpi_bus_get_primary_device()
platform/x86: serdev_helpers: Switch to use acpi_bus_get_primary_device()
platform/x86: yogabook: Switch to use acpi_bus_get_primary_device()
ACPI: amba: Switch to use acpi_bus_get_primary_device()
ACPI: bus: Use acpi/acpi_bus.h without CONFIG_ACPI
|
|
If a single PTE cannot be stored in the current bitmap that should be a
failure and reported back to userspace. Allowing a PTE to be split creates
a possibility that dirty memory will be reported as clean.
The gather must also cover the entire PTE that is being cleaned, not just
the part that fits within the bitmap.
Fix both issues by computing the full IOVA range of every PTE and using
that to update the bitmap and the gather. The bitmap code now rejects
ranges that don't fit in it, and the gather always combined ranges.
The selftest quietly relied on a bug where the CONT was split, that is why
the half page size for the mock domain saw alternating dirties. Increase
the dirty test page size to PAGE_SIZE so it still works as intended.
Cc: stable@vger.kernel.org
Fixes: 4a00f9434891 ("iommupt: Add read_and_clear_dirty op")
Reported-by: Samarth Narang <samarth.colleges@gmail.com>
Closes: https://lore.kernel.org/all/20261013165814.309255
Link: https://patch.msgid.link/r/0-v1-5bb926ff73ec+576-iommufd_dirty_jgg@nvidia.com
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
Generally userspace is going to step the dirty bitmap through the IOVA
space. However if the bitmap ends in the middle of a large PTE this is
problematic. The first step will clear the dirty bit and report dirty to
the end of its bitmap while the second step will see the clear'd dirty and
report clean for the start of the bitmap. If it happens this ends up
losing dirties.
Further this function is missing bounds checking so if this does happen
ther is a good chance it will write memory beyond what the userspace was
expecting.
Return an error code if this illegal condition happens.
Fixes: 58ccf0190d19 ("vfio: Add an IOVA bitmap support")
Cc: stable@vger.kernel.org
Assisted-by: LLM
Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
Link: https://patch.msgid.link/20260925161142.189813-2-parri.andrea@gmail.com
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
Make hv_iommu_device static to give it internal linkage. This
pointer is only used within hv-iommu-guest.c, but previously
has external linkage without a prior declaration. This causes
sparse to warn that the symbol was not declared.
Fixes: 2690d4156b51 ("iommu/hyperv: Add para-virtualized IOMMU support for Hyper-V guest")
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202610092126.TuSXnTYR-lkp@intel.com/
Signed-off-by: Yu Zhang <zhangyu1@linux.microsoft.com>
Reviewed-by: Jacob Pan <jacob.pan@linux.microsoft.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
Make HYPERV_PVIOMMU depend on PCI, to fix the possible link failure
of pci_request_acs(), if HYPERV_PVIOMMU is enabled yet with PCI disabled.
Such configuration shall be prevented.
Fixes: 2690d4156b51 ("iommu/hyperv: Add para-virtualized IOMMU support for Hyper-V guest")
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202610110511.Nk1sZ8SG-lkp@intel.com/
Signed-off-by: Yu Zhang <zhangyu1@linux.microsoft.com>
Reviewed-by: Jacob Pan <jacob.pan@linux.microsoft.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
iommufd_check_iova_range() subtracts one from bitmap->length before it
checks the length, so a zero-length request wraps to SIZE_MAX. With
iova = 0 the wrapped value passes the overflow and alignment checks, and
the request reaches iova_bitmap_alloc() with length 0.
iova_bitmap_alloc() then sizes the bitmap from a length - 1 of SIZE_MAX,
so mapped_total_index is effectively unbounded. The scan uses
last_iova = ULONG_MAX, which spans the whole IOAS when an area is mapped
at IOVA 0. Reject a zero length before the subtraction.
Fixes: b9a60d6f850e ("iommufd: Add IOMMU_HWPT_GET_DIRTY_BITMAP")
Cc: stable@vger.kernel.org
Assisted-by: LLM
Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
Link: https://patch.msgid.link/20260925161142.189813-4-parri.andrea@gmail.com
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
The iova_bitmap_set() function reports dirty IOVAs into a user-provided
bitmap, it does so by pinning the bitmap pages lazily one chunk at a time
as the reported IOVA advances. When advancing to the next chunk fails,
the function simply returns without recording the dirty bits.
if (iova_bitmap_advance_to(bitmap, iova))
return;
So on an IOMMU_HWPT_GET_DIRTY_BITMAP ioctl, if any bitmap chunk past the
first cannot be pinned, the dirty pages in that chunk are cleared in the
page table, not set in the bitmap, and the ioctl will still return
success. This means that the destination VM will observe stale memory.
This patch records the first such error in the iova_bitmap and propagates
it so the ioctl fails instead of silently succeeding.
Fixes: 4a00f9434891 ("iommupt: Add read_and_clear_dirty op")
Signed-off-by: Samarth Narang <samarth.colleges@gmail.com>
Link: https://patch.msgid.link/20261013171045.3104619-1-samarth.colleges@gmail.com
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
iommufd_eventq_fops_release() drops the file's users reference before
reading eventq->ictx. Once that reference is dropped, IOMMU_DESTROY in
another thread can remove and free the event queue, so the later load
accesses freed memory.
Read ictx while the users reference still pins the event queue, then
drop the reference and put the saved context. The context reference
taken by iommufd_eventq_init() keeps ictx valid until the put. Fault and
vEVENTQ file descriptors share this release path.
Fixes: 07838f7fd529 ("iommufd: Add iommufd fault object")
Cc: stable@vger.kernel.org
Assisted-by: LLM, Specula
Signed-off-by: Nirmoy Das <nirmoyd@nvidia.com>
Link: https://patch.msgid.link/20261014133632.3440297-1-nirmoyd@nvidia.com
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
A VFIO PCI dmabuf mapped into an IOAS is mapped with IOMMU_MMIO only in
the domains present when it is mapped. A domain added later, or a copy
of the area into another IOAS, maps the same BAR as cacheable CPU
memory: IOMMU_CACHE set and IOMMU_MMIO clear.
Once the area is in pages->domains_itree, pfn_reader_fill_span() reads
its PFNs back from the storage domain through batch_from_domain(), which
adds them with batch_add_pfn() as BATCH_CPU_MEMORY. Only a hole in the
span reaches pfn_reader_fill_dmabuf() and gets BATCH_MMIO, so
batch_to_domain() programs the later domain with the wrong prot.
The effect depends on the page table format. io-pgtable-arm maps the BAR
as Normal cacheable instead of Device memory, the AMDv1 and x86_64
generic_pt formats set the SME C-bit on it when the tables are encrypted,
and the RISC-V format with Svpbmt maps it as normal memory instead of IO.
Read dmabuf PFNs from the recorded phys for every span, before the
xarray and domain paths. The phys range is always available, and the
read-back from a domain only serves to avoid re-pinning user pages.
Sashiko pointed this out while reviewing an RFC that plumbs the dma-buf
memory type through pfn_reader_fill_dmabuf().
Fixes: 74014a4b55f5 ("iommufd: Have pfn_reader process DMABUF iopt_pages")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Link: https://lore.kernel.org/all/20260716154123.32DC01F000E9@smtp.kernel.org/
Cc: stable@vger.kernel.org
Assisted-by: LLM
Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
Link: https://patch.msgid.link/20261008125206.11990-2-parri.andrea@gmail.com
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
iommufd_device_do_replace() installs the group's reserved IOVA ranges in
the new hwpt's IOAS only when it differs from the old hwpt's IOAS. If the
replace fails in iommufd_device_do_replace() or
iommufd_group_do_replace_reserved_iova(), the error paths remove the
ranges from the new hwpt's IOAS whether or not the IOAS is shared
between the old and new hwpt, which removes the ranges that the old hwpt
still needs.
For example, on x86 with intel-iommu, after a replace within a shared
IOAS fails with -ENODEV, IOMMU_IOAS_IOVA_RANGES goes from
0x0-0xfedfffff
0xfef00000-0xffffffffffff
to
0x0-0xffffffffffff
and IOMMU_IOAS_MAP at 0xfee00000 succeeds.
Remove the ranges on failure only when the replace installed them, and
look up the old hwpt_paging before the replace so that the error path
can compare the IOAS.
Fixes: e88d4ec154a8 ("iommufd: Add iommufd_device_replace()")
Assisted-by: LLM
Signed-off-by: Donggeun Yoo <donggeunyoo.kernel@gmail.com>
Link: https://patch.msgid.link/20260929031402.1998253-1-donggeunyoo.kernel@gmail.com
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
|
|
zpci_refresh_trans() declared the PCI function handle as a u64 type just
because the actual RPCIT machine instruction required that in a 64bit
register, while the function handle is architected as a 32bit type and
defined as such in the zpci_dev struct.
Push the extension from u32 to u64 down in the call chain to just before
calling the RPCIT wrapper function. Since the struct of the typical
req's matches the format expected by RPCIT, too, create one of those to
pass along.
Signed-off-by: Gerd Bayer <gbayer@linux.ibm.com>
Reviewed-by: Niklas Schnelle <schnelle@linux.ibm.com>
Reviewed-by: Matthew Rosato <mjrosato@linux.ibm.com>
Signed-off-by: Heiko Carstens <hca@linux.ibm.com>
|
|
Booting a kdump kernel with crashkernel=X,high crashkernel=0,low on an
AMD system fails the per-segment device table allocation because there
is no memory below 4G:
swapper/0: page allocation failure: order:9, mode:0x40104(GFP_DMA32|__GFP_ZERO|__GFP_COMP)
iommu_alloc_pages_node_sz+0x7f/0x150
alloc_pci_segment+0x4bd/0x650
iommu_go_to_state+0x213d/0x34e0
amd_iommu_prepare+0x39/0xa0
irq_remapping_prepare+0x83/0xd0
enable_IR_x2apic+0x61/0x370
On our E810 systems, the kdump kernel booted but ice MSI-X interrupts
never fired and the TX queues timed out.
Commit b336781b82cc ("iommu/amd: Allocate memory below 4G for dev table
if translation pre-enabled") allocated device tables below 4G. At the
time, the kdump kernel programmed a copy of the old table into the base
register with the IOMMU still enabled. The 64-bit register is updated
as two 32-bit halves, so the running IOMMU could see a half-written,
invalid table pointer. Keeping both tables below 4G leaves the upper
half zero throughout the update, so the pointer stays valid.
Since commit 38e5f33ee359 ("iommu/amd: Reuse device table for kdump")
the kdump kernel reuses the previous kernel's table without writing the
base register, so its newly allocated table is discarded when reuse
succeeds. If reuse fails, the previous patch programs the new table only
after disabling the IOMMU. Either way, the kdump kernel's table doesn't
need to be below 4G.
The previous kernel still allocates its table from DMA32, so the kdump
kernel's check that the old table is below 4G still works. Drop
GFP_DMA32 for kdump kernels only.
Cc: stable@vger.kernel.org # 6.18.x
Reviewed-by: Ashish Kalra <ashish.kalra@amd.com>
Reviewed-by: Ankit Soni <Ankit.Soni@amd.com>
Signed-off-by: Matt Fleming <mfleming@cloudflare.com>
Reviewed-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Matt Fleming <mfleming@cloudflare.com>
Reviewed-by: Ashish Kalra <ashish.kalra@amd.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
Commit 38e5f33ee359 ("iommu/amd: Reuse device table for kdump") made
iommu_set_device_table() return early in every kdump kernel. That is
needed when the previous kernel's device table is reused: the base
register already points at it, and on SNP-enabled systems the register
is locked.
But if reuse isn't possible, early_enable_iommus() falls back to the
freshly allocated table and calls early_enable_iommu(). Because of the
early return, that table is never written to the base register. The
IOMMU keeps using whatever table the register pointed at before, while
the driver writes entries into the new one.
Only skip the write when the previous kernel's table was reused.
Fixes: 38e5f33ee359 ("iommu/amd: Reuse device table for kdump")
Cc: stable@vger.kernel.org
Reviewed-by: Ashish Kalra <ashish.kalra@amd.com>
Reviewed-by: Ankit Soni <Ankit.Soni@amd.com>
Reviewed-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Matt Fleming <mfleming@cloudflare.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
Use if_changed_rule so rule_cc_o_c runs cmd_save_c_flags for the kunit
objects. Without the saved flags CONFIG_LTO_CLANG_THIN_DIST runs the
ThinLTO backend with no --target and generates x86_64 code from arm64
bitcode.
Fixes: 1dd4187f53c3 ("iommupt: Add a kunit test for Generic Page Table")
Reported-by: kernel test robot <lkp@intel.com>
Closes: https://lore.kernel.org/oe-kbuild-all/202610110841.120JzAJG-lkp@intel.com/
Assisted-by: LLM
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
This reverts commit 5e1afd4ea1d6a9bbaecf3e28707dac9c8b56bd45.
This commit caused a regression with GPU devices on some platforms.
Revert it for now.
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/mszyprowski/linux
Pull dma-mapping fixes from Marek Szyprowski:
"Two more fixes for the corner cases in the DMA-mapping SWIOTLB code
(Peng Fan and Marek Szyprowski)"
* tag 'dma-mapping-7.3-2026-10-09' of git://git.kernel.org/pub/scm/linux/kernel/git/mszyprowski/linux:
swiotlb: fix default_swiotlb_limit() for non-growable default pool
iommu/dma: skip swiotlb bounce for DMA_ATTR_MMIO in iommu_dma_map_phys
|
|
The has_msi_cookie() condition only mattered transiently as an artefact
of the cookie rework being done in parallel with the sw_msi rework. Once
combined, calls to iommu_dma_sw_msi() are themselves dispatched based on
domain->cookie_type, so checking again is clearly redundant.
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
It is possible, when a very large mapping uses only one
scatterlist, that padding overflows scatterlist's length field.
This results in:
1) silently wrapping the value
2) smaller than desired mappings produced by iommu_map_sg
3) leaving mapped bytes in memory (no iommu_unmap)
Address this issue by adding overflow detection for scatterlist
length field.
Fixes: 809eac54cdd6 ("iommu/dma: Implement scatterlist segment merging")
Cc: stable@vger.kernel.org
Assisted-by: LLM
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
Add page-selective IOTLB flush using HVCALL_FLUSH_DEVICE_DOMAIN_LIST.
This hypercall accepts a list of (page_number, page_mask_shift) entries,
enabling finer-grained IOTLB invalidation compared to the domain-wide
HVCALL_FLUSH_DEVICE_DOMAIN used by hv_iommu_flush_iotlb_all().
hv_iommu_calc_flush_range() computes the smallest power-of-two aligned
range that covers the target IOVA region, producing a single flush
descriptor. This may over-flush when the range is not naturally aligned,
matching the approach used by Intel VT-d PSI. If the page-selective
flush fails, the code falls back to a full domain flush.
Signed-off-by: Easwar Hariharan <easwar.hariharan@linux.microsoft.com>
Signed-off-by: Yu Zhang <zhangyu1@linux.microsoft.com>
Reviewed-by: Jacob Pan <jacob.pan@linux.microsoft.com>
Reviewed-by: Michael Kelley <mhklinux@outlook.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
Add a para-virtualized IOMMU driver for Linux guests running on Hyper-V.
This driver implements stage-1 IO translation within the guest OS.
It integrates with the Linux IOMMU core, utilizing Hyper-V hypercalls
for:
- Capability discovery
- Domain allocation, configuration, and deallocation
- Device attachment and detachment
- IOTLB invalidation
The driver constructs x86-compatible stage-1 IO page tables in the
guest memory using consolidated IO page table helpers. This allows
the guest to manage stage-1 translations independently of vendor-
specific drivers (like Intel VT-d or AMD IOMMU).
Hyper-V consumes this stage-1 IO page table when a device domain is
created and configured, and nests it with the host's stage-2 IO page
tables, therefore eliminating the VM exits for guest IOMMU mapping
operations. For unmapping operations, VM exits to perform the IOTLB
flush are still unavoidable.
Guest hibernation and resume are not supported by this initial
implementation. The guest-owned stage-1 page tables reside in guest
memory and are preserved in the hibernation image. However, the image
does not by itself preserve the Hyper-V device-domain objects or their
device attachments, and the current pvIOMMU interface does not define
whether or how Hyper-V restores that state.
To identify a device in its hypercall interface, the driver looks up the
logical device ID prefix registered for the device's PCI domain (see the
logical device ID registry in hv_common.c) and combines it with the PCI
function number of the endpoint device.
Co-developed-by: Wei Liu <wei.liu@kernel.org>
Signed-off-by: Wei Liu <wei.liu@kernel.org>
Co-developed-by: Easwar Hariharan <easwar.hariharan@linux.microsoft.com>
Signed-off-by: Easwar Hariharan <easwar.hariharan@linux.microsoft.com>
Signed-off-by: Yu Zhang <zhangyu1@linux.microsoft.com>
Reviewed-by: Jacob Pan <jacob.pan@linux.microsoft.com>
Reviewed-by: Michael Kelley <mhklinux@outlook.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
Intel and AMD IOMMU drivers both reserve the x86 architectural MSI
address window from normal IOVA mappings, but each open-codes the range,
allocation attributes, and insertion into the reserved-region list.
Add iommu_add_resv_x86_msi_region() to encapsulate those operations, and
convert the Intel and AMD drivers to use it. Return an error on allocation
failure so callers can preserve their existing error paths.
Signed-off-by: Yu Zhang <zhangyu1@linux.microsoft.com>
Reviewed-by: Michael Kelley <mhklinux@outlook.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Vasant Hegde <vasant.hegde@amd.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
Most of iommu-dma's own internals were privatised a while ago, but
there are still a couple of DMA-domain-specific helpers provided
by the core code. These now definitely have no reason to be exposed
publicly, so move their declarations into the private header.
Signed-off-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
arm_smmu_insert_master() skips inserting a stream whose StreamID duplicates
one the same master already owns (bridged PCI devices can present duplicate
IDs), leaving that master->streams[i].node zeroed and unlinked from the
smmu->streams rb-tree.
Both the insert error-rollback loop and arm_smmu_remove_master() then call
rb_erase() on every master->streams[i].node unconditionally. rb_erase() on
a zeroed node sees a NULL parent, treats the node as the tree root and sets
root->rb_node = NULL, silently emptying the whole SID tree and breaking SID
lookups (and DMA) for every other master on the SMMU.
Mark each node with RB_CLEAR_NODE() after sort_nonatomic() reorders the
array, since sorting relocates the entries and would leave the earlier
self-referential RB_CLEAR_NODE() pointer stale. An un-inserted node then
stays RB_EMPTY_NODE() and is skipped in both erase loops; inserted nodes
are linked by rb_find_add() and erased as before.
Fixes: b00d24997a11 ("iommu/arm-smmu-v3: Fix iommu_device_probe bug due to duplicated stream ids")
Assisted-by: LLM
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Mostafa Saleh <smostafa@google.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
A queue position, the wrap bit combined with the index, is Q_WRP | Q_IDX.
It is a (max_n_shift + 1)-bit value that wraps at twice the queue capacity.
queue_inc_cons(), queue_sync_cons_ovf(), and queue_inc_prod_n() currently
compute such positions by open-coding the two macros at each call site.
Add a Q_POS() macro and switch the open-coded sites to it.
A subsequent change will build a Q_DIFF() on it, to count the entries that
a queue pointer moved past.
No functional change intended.
Assisted-by: LLM
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Jonathan Cameron <jonathan.cameron@oss.qualcomm.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
The IOPF teardown is done in arm_smmu_remove_master_domain() when releasing
the master_domain on detach, under the global arm_smmu_asid_lock mutex.
A later change will add an IOPF workqueue flush to that teardown, which can
block on a user-faulting page-fault handler. Holding the arm_smmu_asid_lock
across it would stall every unrelated attachment in the system.
Split the teardown out of arm_smmu_remove_master_domain(), to a new helper
arm_smmu_attach_release() that runs after arm_smmu_asid_lock is released.
No functional change: the old master_domain belongs to no other device, so
freeing it outside the lock stays safe, still under iommu_group->mutex.
Reviewed-by: Jonathan Cameron <jonathan.cameron@oss.qualcomm.com>
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
arm_smmu_device_shutdown() only calls arm_smmu_device_disable(), and never
the impl_ops->device_disable() hook. On tegra241-cmdqv, that hook quiesces
every VINTF through tegra241_vintf_hw_deinit(), which issues a CMD_SYNC on
the shared CMDQ while it is still enabled.
arm_smmu_disable_action() already runs this hook before disabling the SMMU,
to cover an unbind or a probe failure, and no devres release ever follows a
reboot or a kexec to run it again. Have arm_smmu_device_shutdown() call it
directly, instead of reimplementing part of what it already does.
Fixes: 9ff145a25c5c ("iommu/tegra241-cmdqv: Fix CMD_SYNC use-after-free on teardown")
Cc: stable@vger.kernel.org
Reviewed-by: Jonathan Cameron <jonathan.cameron@oss.qualcomm.com>
Assisted-by: LLM
Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
Each of these have their own unique situation, populate the tlbi right at
the top and pass it into arm_smmu_domain_inv_range(). They will diverge
further when the iommupt invalidation scheme is introduced.
Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Pranjal Shrivastava <praan@google.com>
Tested-by: Nicolin Chen <nicolinc@nvidia.com>
Tested-by: Pranjal Shrivastava <praan@google.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
Store the required cmd data in the tlbi and just copy it out when
processing each item in the invs list. The cmd form only depends on if the
instance supports range invalidation or not, otherwise it is always the
same.
This avoids redundant calculations for each invs entry.
Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Pranjal Shrivastava <praan@google.com>
Tested-by: Nicolin Chen <nicolinc@nvidia.com>
Tested-by: Pranjal Shrivastava <praan@google.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
Summarize if any of the inv entries will use range invalidation and if any
need ARM_SMMU_OPT_FULL_CONT_RANGE_INV. The next patch will use this to
avoid range invalidation pre-calculations unless the range commands will
be used.
Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Pranjal Shrivastava <praan@google.com>
Tested-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
The server IOMMU drivers focus on invalidation latency by default,
over-invalidating if necessary, to round the invalidation range up to a
single command. I think this represents a trade off for DMA non-FQ and SVA
where stalling the operation is overall worse than re-loading the IOTLB.
For instance AMD and VT-d both round the range up to the largest aligned
power of two and invalidate that. This causes over-invalidation but that
is preferred on real HW over trying to issue a number of smaller range
invalidations.
SMMUv3 on the other hand will try to do up to 512 single invalidations,
otherwise falls back to full invalidation, or it will try to issue an
string of range invalidations for every 5 bits of IOVA range to perfectly
cover it.
While the range-invalidation optimization is pretty good we can still do
better and reduce it to only 2 invalidations at maximum using the idea
Robin came up with to overlap two range invalidations in the middle. This
preserves the exact coverage of the current range invalidation and still
caps the number of range invalidations at 2.
This works because the range invalidation can start at any IOVA, so we can
place a range invalidation forwards from the start and backwards from the
end, overlapping in the middle if the scale isn't precise enough.
In the cases where this produces 2 range invalidations it continues to
produce range invalidations that don't overlap, but the exact split point
is different than the original algorithm due to the new calculation
method. In cases where the original algorithm would produce 3 or more
range invalidations this will produce 2 range invalidations with an
overlap.
The SVA under invalidation errata work around is maintained by "rounding
up" to generate a single range invalidation that over-covers the entire
range.
Since the normal path is now the only one with a loop, split them into two
functions and fold a simplified version of arm_smmu_inv_size_too_big()
directly into the normal flow in a way that directly limits the number of
single invalidation commands generated, again focusing on controlling
latency.
The end result is any gather is converted into either:
- One invalidate all
- One or two range invalidation operations
- At most 512 single invalidation ops
The current code switches at 511, but this turns every table removal
into a full invalidation. Change to 512 to try to minimize the amount
of full invalidation when using a 4k translation granule.
arm_smmu_domain_inv() is modified to always pass in the tgsz because the
new logic relies on it being valid.
Reviewed-by: Pranjal Shrivastava <praan@google.com>
Tested-by: Nicolin Chen <nicolinc@nvidia.com>
Tested-by: Pranjal Shrivastava <praan@google.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
pgsize is a constant property of the domain, it is the base translation
granule of the page table (4k, 16k, 64k) in log2.
Store it to the struct arm_smmu_domain based on how the page table was
created.
Pass it around in the tlbi.
Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Pranjal Shrivastava <praan@google.com>
Tested-by: Nicolin Chen <nicolinc@nvidia.com>
Tested-by: Pranjal Shrivastava <praan@google.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
These parameters go to a lot of different functions and the next patches
will add more. Put them into a struct to keep things tidy.
Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Pranjal Shrivastava <praan@google.com>
Tested-by: Nicolin Chen <nicolinc@nvidia.com>
Tested-by: Pranjal Shrivastava <praan@google.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
The erratum (MMU-700: #3777127, S3: #3673557) deals with under
invalidation of a CONT PTE grouping in the SMMU. The recommended work
around is to use a range invalidation that spans the entire CONT. The only
user of CONT in the kernel right now is through SVA sharing a CPU page
table that contains a CONT created by the mm.
Previously it was thought that this errata was dealt with because the
driver always uses range invalidation. However, there is a subtle detail
in the errata that the invalidation range must fully enclose the entire
CONT for it to work.
It seems that two sequential range invalidations, with a split point
falling inside a CONT grouping, will not prevent the errata.
The SMMU's range invalidation generation algorithm does not produce a
single invalidation for a single SVA invalidation request, nor does the mm
carefully align the SVA invalidation ranges to accommodate the splitting
of the invalidation into several ranges.
Thus, when processing a SVA invalidation, the range invalidation splitting
routine can generate a range invalidation that is split in the middle of
the CONT and risk under invalidation from this errata. This condition
could be triggered by a malicious userspace manipulating the TLB gathers
via mmap/mprotect/munmap.
Update the errata list to the include the S3 variation, detect the IOMMUs
that have it, and then have SVA invalidations use a simplified version of
the over invalidation algorithm from the tlbi rework series. This ensures
that a single range invalidation is issued for a single MMU notifier
callback and now the range invalidation is guarenteed to cover any posible
CONT.
Future work to add CONT to iommu_domain page tables should either use this
one-invalidate/one-range invalidation algorithm or disable CONT support in
the iommu_domain.
Cc: stable@vger.kernel.org
Cc: Vijayanand Jitta <vijayanand.jitta@oss.qualcomm.com>
Fixes: 3f1ce8e85ee0 ("iommu/arm-smmu-v3: Share process page tables")
Reviewed-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Pranjal Shrivastava <peaan@google.com>
Tested-by: Nicolin Chen <nicolinc@nvidia.com>
Tested-by: Pranjal Shrivastava <praan@google.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
struct arm_smmu_invs marks its flexible array member inv[] with the
__counted_by(max_invs).
arm_smmu_v3_invs_test() uses invs1 to invs5 which has num_invs = 3
but omit max_invs.
This causes the following UBSAN error:
[ 2.333240] UBSAN: array-index-out-of-bounds in drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c:1116:21
[ 2.333604] # arm_smmu_v3_invs_test: lib/ubsan.c:228: array-index-out-of-bounds in drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
[ 2.334129] index 1 is out of range for type 'struct arm_smmu_inv[] __counted_by(max_invs)' (aka 'struct arm_smmu_inv[]')
[ 2.335375] CPU: 6 UID: 0 PID: 155 Comm: kunit_try_catch Tainted: G N 7.2.0-g8dcb331fbb72 #2 PREEMPT
[ 2.335759] Tainted: [N]=TEST
[ 2.335791] Hardware name: linux,dummy-virt (DT)
[ 2.336254] Call trace:
[ 2.336932] show_stack+0x18/0x24 (C)
[ 2.338443] __dump_stack+0x28/0x38
[ 2.338494] dump_stack_lvl+0x54/0x6c
[ 2.338518] dump_stack+0x18/0x24
[ 2.338541] ubsan_epilogue+0x10/0x44
[ 2.338565] __ubsan_handle_out_of_bounds+0xb8/0xbc
[ 2.338834] arm_smmu_invs_merge+0x688/0x8ac
[ 2.338862] arm_smmu_v3_invs_test+0xd4/0x598
[ 2.338890] kunit_try_run_case+0x64/0x160
[ 2.338914] kunit_generic_run_threadfn_adapter+0x28/0x4c
[ 2.338955] kthread+0x10c/0x12c
[ 2.338984] ret_from_fork+0x10/0x20
[ 2.339433] ---[ end trace ]---
Explicitly set max_invs = 3 on the test arrays to match the number of
elements.
Fixes: 15a2a5645ad7 ("iommu/arm-smmu-v3: Introduce a per-domain arm_smmu_invs array")
Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Pranjal Shrivastava <praan@google.com>
Tested-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
arm_smmu_v3_invs_test_verify() validates the array bounds using
KUNIT_EXPECT_EQ(), which triggers a failure and continues execution.
If invs->num_invs was smaller than expected, the next loop over
num_invs would blindly read past the end of the invs->inv[] array.
Switch to KUNIT_ASSERT_EQ() to fail and stop the test on bound errors.
Fixes: 15a2a5645ad7 ("iommu/arm-smmu-v3: Introduce a per-domain arm_smmu_invs array")
Reported-by: Sashiko <>
Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Pranjal Shrivastava <praan@google.com>
Tested-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
arm_smmu_invs_merge() and arm_smmu_invs_alloc() can return NULL or
errors which are checked by the driver but not the test.
Add KUNIT_ASSERT_NOT_ERR_OR_NULL() after calling them to fail the
test instead of accessing NULL or ERR pointers.
Fixes: 15a2a5645ad7 ("iommu/arm-smmu-v3: Introduce a per-domain arm_smmu_invs array")
Reported-by: Sashiko <>
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Pranjal Shrivastava <praan@google.com>
Tested-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
arm_smmu_v3_test_debug_print_used_bits() hardcodes arm_smmu_get_ste_used()
instead of using the provided writer->ops->get_used() callback.
This caused Context Descriptors to be incorrectly parsed as STEs.
Fixes: 56e1a4cc2588 ("iommu/arm-smmu-v3: Add unit tests for arm_smmu_write_entry")
Reported-by: Sashiko <>
Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Pranjal Shrivastava <praan@google.com>
Tested-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
Commit 6fabce53f6b9 ("iommu/arm-smmu-v3: Add a missing dma_wmb() for hitless STE update")
adds a dma_wmb() to arm_smmu_write_entry() to make sure stream tables
and context descriptors are observed first.
However, STE L1 table descriptors are configured directly
via WRITE_ONCE(), where before that they were zeroed with memset()
inside dma_direct_alloc() then written to abort in via memset() also
in arm_smmu_init_initial_stes() without a barrier in both cases which
means that the SMMUv3 can observe the allocated table before the
written descriptors causing it to fetch random data.
Similarly in arm_smmu_write_cd_l1_desc() where the L1 CD is written
after dma_alloc_coherent() with no barriers.
Add dma_wmb() in both cases.
Fixes: 48ec83bcbcf5 ("iommu/arm-smmu: Add initial driver support for ARM SMMUv3 devices")
Reported-by: Sashiko <>
Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Pranjal Shrivastava <praan@google.com>
Tested-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
All three queues are sized from the maxima the hardware advertises in IDR1
and allocated at probe, up to 4 MB each on a 4K-page kernel. The capture
kernel already disables two of them: arm_smmu_device_reset() drops
CR0_EVTQEN and CR0_PRIQEN. It still allocates both at full size.
A kdump capture kernel runs from a small crashkernel reservation, and every
SMMUv3 instance pays that cost again, up to 12 MB apiece. It goes to queues
that either serve the handful of devices used to save the dump or are
switched off outright, and it is memory the dump itself needs.
Size all three queues at one page worth of entries when is_kdump_kernel().
The queues carry commands and fault records rather than DMA data, so dump
throughput is unaffected. A shallower command queue only bounds how many
commands may be in flight before a sync, which does not matter for the few
devices that save the dump. The cmdq_max_n_shift parameter therefore does
not apply in a capture kernel.
Suggested-by: Kyle McMartin <jkkm@meta.com>
Assisted-by: LLM
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Tested-by: Yuanhe Shu <xiangzao@linux.alibaba.com>
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
Tested-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Breno Leitao <leitao@debian.org>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
The command queue depth comes straight from the maximum the hardware
advertises in IDR1, which reaches megabytes of coherent DMA per queue.
A system with several SMMUv3 instances pays that per instance, and the
Tegra241 CMDQV pays it again for every VCMDQ it preallocates.
Queue depth only bounds how many commands may be in flight before a sync.
A machine driving a handful of devices, or one with a tight memory budget,
has no use for the maximum, and no way to say so.
Add cmdq_max_n_shift, a cap on the depth given as the log2 of the entry
count, the form the hardware itself takes in the LOG2SIZE field of
CMDQ_BASE. It defaults to the largest depth the driver allocates, so an
unset parameter changes nothing. Decide the depth in
arm_smmu_cmdq_max_n_shift(), which applies the parameter to the IDR1
value, so the queue is allocated at the requested size. The Tegra241 CMDQV
sizes its VCMDQs from IDR1 itself, so route that through the same helper.
Floor the request at one page worth of entries. Without the floor, a small
request trips the CMDQ_BATCH_ENTRIES check in arm_smmu_device_hw_probe()
and the SMMU fails to probe. The floor also costs nothing: coherent DMA is
page granular, so a shallower queue occupies the same memory as one that
fills the page.
Assisted-by: LLM
Reviewed-by: Breno Leitao <leitao@debian.org>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
Tested-by: Nicolin Chen <nicolinc@nvidia.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
The Google Tensor G5 SoC (Laguna) has a silicon errata with the SMMU
integration where the AxPROT[2] signal is always 1, which makes all
transactions appear as instructions.
To avoid F_PERMISSION faults (due to the VMSAv8-64 requirement that
unprivilieged writable buffers are treated as Privileged Execute-Never
(PXN)), as a software workaround we make use of the INSTCFG override
feature to force all incoming traffic to data.
To enable this workaround, the "arm,instdata-override" DT property can be
provided in the device tree entry for the SMMU, similar to how existing
errata workarounds are handled by this driver.
Note this is added for "stage-1" and "stage-2" but we intentionally leave
out "bypass" as nothing is downstream of the SMMU that makes use of the
Instruction/Data attribute.
[peter.griffin: ported to upstream, updated commit message, remove
feature flag]
Signed-off-by: Pranjal Shrivastava <praan@google.com>
Signed-off-by: Peter Griffin <peter.griffin@linaro.org>
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Will Deacon <will@kernel.org>
|
|
Replace acpi_get_first_physical_node() that is slated for removal
with acpi_bus_get_primary_device() that takes a reference to the
device it is about to return.
This addresses a potential use-after-free that may occur if the
device returned by acpi_get_first_physical_node() is removed right
after dropping its ACPI companion's physical_node_lock in that
function.
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com>
Link: https://patch.msgid.link/3636226.QJadu78ljV@rafael.j.wysocki
|
|
This IOMMU translates memory access requests for the VC6 display
pipeline and various multimedia devices in the Broadcom BCM2712 SoC used
on Raspberry Pi 5.
The driver uses the generic_pt framework to manage the 2-level page
tables. It also provides support for the centralized L2 TLB (IOMMUC),
which is shared across the SoC's IOMMU instances and must be flushed
when modifying or unmapping page table entries.
Adapted from Raspberry Pi's downstream bcm2712-iommu driver
(original author Nick Hollinghurst).
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Daniel Drake <dan@reactivated.net>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
|
Implement generic_pt support for the 2-level page table format used by
the BCM2712 IOMMUs.
Adapted from Raspberry Pi's downstream bcm2712-iommu driver (original
author Nick Hollinghurst).
Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Daniel Drake <dan@reactivated.net>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>
|
