summaryrefslogtreecommitdiff
path: root/include/linux
diff options
context:
space:
mode:
authorLi Zhe <lizhe.67@bytedance.com>2026-08-31 19:16:35 +0800
committerAndrew Morton <akpm@linux-foundation.org>2026-10-06 17:39:17 -0700
commita43a0ceff082382d943d8de13cec3ef0ff133a88 (patch)
tree1f8a29e38d93e26fe0b0dc826aac62d6ae932f26 /include/linux
parent72dcfdc2ce47c84d248ab43490ec91a0d5098253 (diff)
downloadlinux-next-a43a0ceff082382d943d8de13cec3ef0ff133a88.tar.gz
linux-next-a43a0ceff082382d943d8de13cec3ef0ff133a88.zip
mm: extend the template fast path to zone-device compound tails
The template fast path from the previous patch only accelerates head pages. Compound tails in memmap_init_compound() still go through the normal initialization path one by one. Build separate head and tail templates and reuse one prepared tail template across the tail pages in a compound range. Head pages preserve the existing refcount policy, while compound tails always start with a refcount of 0 after prep_compound_tail(). This extends the template-copy fast path to pfns_per_compound > 1. Tail-page PFN-dependent fields are refreshed in the reusable tail template before each copy. Do not keep a separate non-template fallback for compound tails either. These pages are still under memmap initialization, and the initialization-time refcount updates are not part of the observable lifetime of pages handed out later. The impact is controlled for the same reason as for head pages. The first tail page still seeds the reusable tail template through the normal tail initialization sequence, and the copied tail pages have the same final initialized state except for the PFN-dependent fields refreshed before each copy. Tested in a VM with a 100 GB devdax namespace (align=2097152) on Intel Ice Lake server. This test exercises the dax_pmem rebind path and measures memmap initialization latency. Test procedure: Unbind and rebind the dax_pmem driver 30 times, collect memmap initialization time from the pr_debug() output of memmap_init_zone_device(). Base(v7.3-rc1): Average of rebinds for dax_pmem driver: 191.20 ms With this patch and its prerequisites applied: Average of rebinds for dax_pmem driver: 176.87 ms This reduces the average memmap initialization time measured during rebind from 191.20 ms to 176.87 ms, or about 7.5%. Link: https://lore.kernel.org/20260831111638.76012-5-lizhe.67@bytedance.com Signed-off-by: Li Zhe <lizhe.67@bytedance.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Reviewed-by: Mike Rapoport (Microsoft) <rppt@kernel.org> Cc: Alistair Popple <apopple@nvidia.com> Cc: Arnd Bergmann <arnd@arndb.de> Cc: Balbir Singh <balbirs@nvidia.com> Cc: "Borislav Petkov (AMD)" <bp@alien8.de> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: David Hildenbrand (Arm) <david@kernel.org> Cc: Ingo Molnar <mingo@redhat.com> Cc: Kees Cook <kees@kernel.org> Cc: Muchun Song <muchun.song@linux.dev>
Diffstat (limited to 'include/linux')
0 files changed, 0 insertions, 0 deletions