diff options
| author | Uros Bizjak <ubizjak@gmail.com> | 2022-05-25 16:54:16 +0200 |
|---|---|---|
| committer | Joerg Roedel <jroedel@suse.de> | 2022-06-23 09:43:31 +0200 |
| commit | 0d10fe7591178713695d7d8d77284b2fc2e6629f (patch) | |
| tree | 76e5587ab57d764a92540345c19b2800387fdc48 /drivers/iommu/amd/io_pgtable.c | |
| parent | a111daf0c53ae91e71fd2bfe7497862d14132e3e (diff) | |
| download | lwn-0d10fe7591178713695d7d8d77284b2fc2e6629f.tar.gz lwn-0d10fe7591178713695d7d8d77284b2fc2e6629f.zip | |
iommu/amd: Use try_cmpxchg64 in alloc_pte and free_clear_pte
Use try_cmpxchg64 instead of cmpxchg64 (*ptr, old, new) != old in
alloc_pte and free_clear_pte. cmpxchg returns success in ZF flag, so this
change saves a compare after cmpxchg (and related move instruction
in front of cmpxchg). Also, remove racy explicit assignment to pteval
when cmpxchg fails, this is what try_cmpxchg does implicitly from
*pte in an atomic way.
Signed-off-by: Uros Bizjak <ubizjak@gmail.com>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Cc: Will Deacon <will@kernel.org>
Link: https://lore.kernel.org/r/20220525145416.10816-1-ubizjak@gmail.com
Signed-off-by: Joerg Roedel <jroedel@suse.de>
Diffstat (limited to 'drivers/iommu/amd/io_pgtable.c')
| -rw-r--r-- | drivers/iommu/amd/io_pgtable.c | 6 |
1 files changed, 2 insertions, 4 deletions
diff --git a/drivers/iommu/amd/io_pgtable.c b/drivers/iommu/amd/io_pgtable.c index 6608d1717574..7d4b61e5db47 100644 --- a/drivers/iommu/amd/io_pgtable.c +++ b/drivers/iommu/amd/io_pgtable.c @@ -258,7 +258,7 @@ static u64 *alloc_pte(struct protection_domain *domain, __npte = PM_LEVEL_PDE(level, iommu_virt_to_phys(page)); /* pte could have been changed somewhere. */ - if (cmpxchg64(pte, __pte, __npte) != __pte) + if (!try_cmpxchg64(pte, &__pte, __npte)) free_page((unsigned long)page); else if (IOMMU_PTE_PRESENT(__pte)) *updated = true; @@ -341,10 +341,8 @@ static void free_clear_pte(u64 *pte, u64 pteval, struct list_head *freelist) u64 *pt; int mode; - while (cmpxchg64(pte, pteval, 0) != pteval) { + while (!try_cmpxchg64(pte, &pteval, 0)) pr_warn("AMD-Vi: IOMMU pte changed since we read it\n"); - pteval = *pte; - } if (!IOMMU_PTE_PRESENT(pteval)) return; |