lwn.git/arch/arm/mm/cache-v7.S, branch docs-next

kcfi: Rename CONFIG_CFI_CLANG to CONFIG_CFI

2025-09-24T21:29:14+00:00

The kernel's CFI implementation uses the KCFI ABI specifically, and is not strictly tied to a particular compiler. In preparation for GCC supporting KCFI, rename CONFIG_CFI_CLANG to CONFIG_CFI (along with associated options). Use new "transitional" Kconfig option for old CONFIG_CFI_CLANG that will enable CONFIG_CFI during olddefconfig. Reviewed-by: Linus Walleij Reviewed-by: Nathan Chancellor Link: https://lore.kernel.org/r/20250923213422.1105654-3-kees@kernel.org Signed-off-by: Kees Cook

ARM: 9393/1: mm: Use conditionals for CFI branches

2024-05-07T09:30:24+00:00

Commit 9385/2 introduced a few branches inside function prototypes when using CFI in order to deal with the situation where CFI inserts a few bytes of function information in front of the symbol. This is not good for older CPUs where every cycle counts. Commit 9386/2 alleviated the situation a bit by using aliases for the cache functions with identical signatures. This leaves the coherent cache flush functions *_coherent_kern_range() with these branches to the corresponing *_coherent_user_range() around, since their return type differ and they therefore cannot be aliased. Solve this by a simple ifdef so at least we can use fallthroughs when compiling without CFI enabled. Link: https://lore.kernel.org/linux-arm-kernel/Zi+e9M%2Ff5b%2FSto9H@shell.armlinux.org.uk/ Suggested-by: Ard Biesheuvel Signed-off-by: Linus Walleij Signed-off-by: Russell King (Oracle)

ARM: 9387/2: mm: Rewrite cacheflush vtables in CFI safe C

2024-04-29T13:14:18+00:00

Instead of defining all cache flush operations with an assembly macro in proc-macros.S, provide an explicit struct cpu_cache_fns for each CPU cache type in mm/cache.c. As a side effect from rewriting the vtables in C, we can avoid the aliasing for the "louis" cache callback, instead we can just assign the NN_flush_kern_cache_all() function to the louis callback in the C vtable. As the louis cache callback is called explicitly (not through the vtable) if we only have one type of cache support compiled in, we need an ifdef quirk for this in the !MULTI_CACHE case. Feroceon and XScale have some dma mapping quirk, in this case we can just define two structs and assign all but one callback to the main implementation; since each of them invoked define_cache_functions twice they require MULTI_CACHE by definition so the compiled-in shortcut is not used on these variants. Tested-by: Kees Cook Reviewed-by: Sami Tolvanen Signed-off-by: Linus Walleij Signed-off-by: Russell King (Oracle)

ARM: 9385/2: mm: Type-annotate all cache assembly routines

2024-04-29T13:14:16+00:00

Tag all references to assembly functions with SYM_TYPED_FUNC_START() and SYM_FUNC_END() so they also become CFI-safe. When we add SYM_TYPED_FUNC_START() to assembly calls, a function prototype signature will be emitted into the object file at (pc-4) at the call site, so that the KCFI runtime check can compare this to the expected call. Example: 8011ae38: a540670c .word 0xa540670c 8011ae3c : 8011ae3c: e3a00000 mov r0, #0 8011ae40: ee070f11 mcr 15, 0, r0, cr7, cr1, {0} 8011ae44: e12fff1e bx lr This means no "fallthrough" code can enter a SYM_TYPED_FUNC_START() call from above it: there will be a function prototype signature there, so those are consistently converted to a branch or ret lr depending on context. Tested-by: Kees Cook Reviewed-by: Sami Tolvanen Signed-off-by: Linus Walleij Signed-off-by: Russell King (Oracle)

ARM: 9263/1: use .arch directives instead of assembler command line flags

2022-11-08T18:36:17+00:00

Similar to commit a6c30873ee4a ("ARM: 8989/1: use .fpu assembler directives instead of assembler arguments"). GCC and GNU binutils support setting the "sub arch" via -march=, -Wa,-march, target function attribute, and .arch assembler directive. Clang was missing support for -Wa,-march=, but this was implemented in clang-13. The behavior of both GCC and Clang is to prefer -Wa,-march= over -march= for assembler and assembler-with-cpp sources, but Clang will warn about the -march= being unused. clang: warning: argument unused during compilation: '-march=armv6k' [-Wunused-command-line-argument] Since most assembler is non-conditionally assembled with one sub arch (modulo arch/arm/delay-loop.S which conditionally is assembled as armv4 based on CONFIG_ARCH_RPC, and arch/arm/mach-at91/pm-suspend.S which is conditionally assembled as armv7-a based on CONFIG_CPU_V7), prefer the .arch assembler directive. Add a few more instances found in compile testing as found by Arnd and Nathan. Link: https://github.com/llvm/llvm-project/commit/1d51c699b9e2ebc5bcfdbe85c74cc871426333d4 Link: https://bugs.llvm.org/show_bug.cgi?id=48894 Link: https://github.com/ClangBuiltLinux/linux/issues/1195 Link: https://github.com/ClangBuiltLinux/linux/issues/1315 Suggested-by: Arnd Bergmann Suggested-by: Nathan Chancellor Signed-off-by: Arnd Bergmann Tested-by: Nathan Chancellor Signed-off-by: Nick Desaulniers Signed-off-by: Russell King (Oracle)

ARM: cacheflush: avoid clobbering the frame pointer

2022-02-09T09:13:10+00:00

Thumb2 uses R7 rather than R11 as the frame pointer, and even if we rarely use a frame pointer to begin with when building in Thumb2 mode, there are cases where it is required by the compiler (Clang when inserting profiling hooks via -pg) However, preserving and restoring the frame pointer is risky, as any unhandled exceptions raised in the mean time will produce a bogus backtrace, and it would be better not to touch the frame pointer at all. This is the case even when CONFIG_FRAME_POINTER is not set, as the unwind directive used by the unwinder may also use R7 or R11 as the unwind anchor, even if the frame pointer is not managed strictly according to the frame pointer ABI. So let's tweak the cacheflush asm code not to clobber R7 or R11 at all, so that we can drop R7 from the clobber lists of the inline asm blocks that call these routines, and remove the code that preserves/restores R11. Signed-off-by: Ard Biesheuvel Reviewed-by: Nick Desaulniers

ARM: 9059/1: cache-v7: get rid of mini-stack

2021-03-09T10:25:18+00:00

Now that we have reduced the number of registers that we need to preserve when calling v7_invalidate_l1 from the boot code, we can use scratch registers to preserve the remaining ones, and get rid of the mini stack entirely. This works around any issues regarding cache behavior in relation to the uncached accesses to this memory, which is hard to get right in the general case (i.e., both bare metal and under virtualization) While at it, switch v7_invalidate_l1 to using ip as a scratch register instead of r4. This makes the function AAPCS compliant, and removes the need to stash r4 in ip across the call. Acked-by: Nicolas Pitre Signed-off-by: Ard Biesheuvel Signed-off-by: Russell King

ARM: 9058/1: cache-v7: refactor v7_invalidate_l1 to avoid clobbering r5/r6

2021-03-09T10:25:18+00:00

The cache invalidation code in v7_invalidate_l1 can be tweaked to re-read the associativity from CCSIDR, and keep the way identifier component in a single register that is assigned in the outer loop. This way, we need 2 registers less. Given that the number of sets is typically much larger than the associativity, rearrange the code so that the outer loop has the fewer number of iterations, ensuring that the re-read of CCSIDR only occurs a handful of times in practice. Fix the whitespace while at it, and update the comment to indicate that this code is no longer a clone of anything else. Acked-by: Nicolas Pitre Signed-off-by: Ard Biesheuvel Signed-off-by: Russell King

ARM: 9057/1: cache-v7: add missing ISB after cache level selection

2021-03-09T10:25:17+00:00

A write to CSSELR needs to complete before its results can be observed via CCSIDR. So add a ISB to ensure that this is the case. Acked-by: Nicolas Pitre Signed-off-by: Ard Biesheuvel Signed-off-by: Russell King

sched/rt, ARM: Use CONFIG_PREEMPTION

2019-12-08T13:37:32+00:00

CONFIG_PREEMPTION is selected by CONFIG_PREEMPT and by CONFIG_PREEMPT_RT. Both PREEMPT and PREEMPT_RT require the same functionality which today depends on CONFIG_PREEMPT. Switch the entry code, cache over to use CONFIG_PREEMPTION and add output in show_stack() for PREEMPT_RT. [bigeasy: +traps.c] Signed-off-by: Thomas Gleixner Signed-off-by: Sebastian Andrzej Siewior Signed-off-by: Thomas Gleixner Cc: Linus Torvalds Cc: Peter Zijlstra Cc: Russell King Cc: linux-arm-kernel@lists.infradead.org Link: https://lore.kernel.org/r/20191015191821.11479-2-bigeasy@linutronix.de Signed-off-by: Ingo Molnar