diff options
| author | Björn Töpel <bjorn@rivosinc.com> | 2024-06-05 13:40:45 +0200 |
|---|---|---|
| committer | Palmer Dabbelt <palmer@rivosinc.com> | 2024-06-26 08:42:39 -0700 |
| commit | 66673099f734fd6512055fee353b5c81dafec216 (patch) | |
| tree | 9a7ce8a27f3a6e83febcfe02231813375a9bec4c /arch/riscv/mm/init.c | |
| parent | e3ecf2fdc8f39a898f9e06481e935b460a097e10 (diff) | |
| download | lwn-66673099f734fd6512055fee353b5c81dafec216.tar.gz lwn-66673099f734fd6512055fee353b5c81dafec216.zip | |
riscv: mm: Pre-allocate vmemmap/direct map/kasan PGD entries
The RISC-V port copies the PGD table from init_mm/swapper_pg_dir to
all userland page tables, which means that if the PGD level table is
changed, other page tables has to be updated as well.
Instead of having the PGD changes ripple out to all tables, the
synchronization can be avoided by pre-allocating the PGD entries/pages
at boot, avoiding the synchronization all together.
This is currently done for the bpf/modules, and vmalloc PGD regions.
Extend this scheme for the PGD regions touched by memory hotplugging.
Prepare the RISC-V port for memory hotplug by pre-allocate
vmemmap/direct map/kasan entries at the PGD level. This will roughly
waste ~128 (plus 32 if KASAN is enabled) worth of 4K pages when memory
hotplugging is enabled in the kernel configuration.
Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Signed-off-by: Björn Töpel <bjorn@rivosinc.com>
Link: https://lore.kernel.org/r/20240605114100.315918-3-bjorn@kernel.org
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
Diffstat (limited to 'arch/riscv/mm/init.c')
| -rw-r--r-- | arch/riscv/mm/init.c | 9 |
1 files changed, 9 insertions, 0 deletions
diff --git a/arch/riscv/mm/init.c b/arch/riscv/mm/init.c index ecd4e716c995..85f076b80a12 100644 --- a/arch/riscv/mm/init.c +++ b/arch/riscv/mm/init.c @@ -28,6 +28,7 @@ #include <asm/fixmap.h> #include <asm/io.h> +#include <asm/kasan.h> #include <asm/numa.h> #include <asm/pgtable.h> #include <asm/sections.h> @@ -1492,11 +1493,19 @@ failed: panic("Failed to pre-allocate %s pages for %s area\n", lvl, area); } +#define PAGE_END KASAN_SHADOW_START + void __init pgtable_cache_init(void) { preallocate_pgd_pages_range(VMALLOC_START, VMALLOC_END, "vmalloc"); if (IS_ENABLED(CONFIG_MODULES)) preallocate_pgd_pages_range(MODULES_VADDR, MODULES_END, "bpf/modules"); + if (IS_ENABLED(CONFIG_MEMORY_HOTPLUG)) { + preallocate_pgd_pages_range(VMEMMAP_START, VMEMMAP_END, "vmemmap"); + preallocate_pgd_pages_range(PAGE_OFFSET, PAGE_END, "direct map"); + if (IS_ENABLED(CONFIG_KASAN)) + preallocate_pgd_pages_range(KASAN_SHADOW_START, KASAN_SHADOW_END, "kasan"); + } } #endif |