lwn.git/arch/s390/include/asm/cmpxchg.h, branch docs-next

s390: Remove superfluous newlines from inline assemblies

2025-09-29T11:52:08+00:00

Remove superfluous newlines from inline assemblies. Compilers use the number of lines of inline assemblies as heuristic for the complexity and inline decisions. Therefore inline assemblies should only contain as many lines as required. A lot of inline assemblies contain a superfluous newline for the last line. Remove such newlines to improve compiler inlining decisions. Suggested-by: Juergen Christ Signed-off-by: Heiko Carstens Reviewed-by: Alexander Gordeev Reviewed-by: Juergen Christ Signed-off-by: Alexander Gordeev

s390/cmpxchg: Provide arch_try_cmpxchg128()

2024-11-12T13:01:28+00:00

Since gcc 14 flag output operands are supported also for s390. Provide an arch_try_cmpxchg128() implementation so that all existing try_cmpxchg128() variants provide slightly better code, if compiled with gcc 14 or newer. Reviewed-by: Juergen Christ Signed-off-by: Heiko Carstens

s390/cmpxchg: Provide arch_cmpxchg128_local()

2024-11-12T13:01:28+00:00

Just like x86 and arm64 provide a trivial arch_cmpxchg128_local() implementation by mapping it to arch_cmpxchg128(). Reviewed-by: Juergen Christ Signed-off-by: Heiko Carstens

s390/cmpxchg: Implement arch_xchg() with arch_try_cmpxchg()

2024-11-12T13:01:28+00:00

Get rid of the arch_xchg() inline assemblies by converting the inline assemblies to C functions which make use of arch_try_cmpxchg(). With flag output operand support the generated code is at least as good as the previous version. Without it is slightly worse, however getting rid of all the inline assembly code is worth it. Signed-off-by: Heiko Carstens

s390/cmpxchg: Provide arch_try_cmpxchg()

2024-11-12T13:01:28+00:00

Since gcc 14 flag output operands are supported also for s390. Provide an arch_try_cmpxchg() implementation so that all existing try_cmpxchg() variants provide slightly better code, if compiled with gcc 14 or newer. Reviewed-by: Juergen Christ Signed-off-by: Heiko Carstens

s390/cmpxchg: Convert one and two byte case inline assemblies to C

2024-11-12T13:01:28+00:00

Rewrite __cmpxchg() in order to get rid of the large inline assemblies. Convert the one and two byte inline assemblies to C functions. The generated code of the new implementation is nearly as good or bad as the old variant, but easier to read. Note that the new variants are quite close to the generic cmpxchg_emu_u8() implementation, however a conversion to the generic variant will not follow since with mm/vmstat.c there is heavy user of one byte cmpxchg(). A not inlined variant would have a negative performance impact. Also note that the calls within __arch_cmpxchg() come with rather pointless "& 0xff..." operations. They exist only to avoid false positive sparse warnings like "warning: cast truncates bits from constant value ...". Reviewed-by: Juergen Christ Signed-off-by: Heiko Carstens

arch: Remove cmpxchg_double

2023-06-05T07:36:39+00:00

No moar users, remove the monster. Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Arnd Bergmann Reviewed-by: Mark Rutland Acked-by: Heiko Carstens Tested-by: Mark Rutland Link: https://lore.kernel.org/r/20230531132323.991907085@infradead.org

arch: Introduce arch_{,try_}_cmpxchg128{,_local}()

2023-06-05T07:36:35+00:00

For all architectures that currently support cmpxchg_double() implement the cmpxchg128() family of functions that is basically the same but with a saner interface. Signed-off-by: Peter Zijlstra (Intel) Reviewed-by: Arnd Bergmann Reviewed-by: Mark Rutland Acked-by: Heiko Carstens Acked-by: Mark Rutland Tested-by: Mark Rutland Link: https://lore.kernel.org/r/20230531132323.452120708@infradead.org

locking/arch: Rename all internal xchg() names to arch_xchg()

2023-04-29T07:08:44+00:00

Decrease the probability of this internal facility to be used by driver code. Signed-off-by: Andrzej Hajda Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Ingo Molnar Reviewed-by: Arnd Bergmann Reviewed-by: Andi Shyti Acked-by: Geert Uytterhoeven [m68k] Acked-by: Palmer Dabbelt [riscv] Link: https://lore.kernel.org/r/20230118154450.73842-1-andrzej.hajda@intel.com Cc: Linus Torvalds

s390/cmpxchg: make loop condition for 1,2 byte cases precise

2022-11-21T12:36:16+00:00

The cmpxchg implementation for 1 and 2 bytes consists of a 4 byte cmpxchg loop. Currently, the decision to retry is imprecise, looping if bits outside the target byte(s) change instead of retrying until the target byte(s) differ from the old value. E.g. if an attempt to exchange (prev_left_0 old_bytes prev_right_0) is made and it fails because the word at the address is (prev_left_1 x prev_right_1) where both x != old_bytes and one of the prev_*_1 values differs from the respective prev_*_0 value, the cmpxchg is retried, even if by a semantic equivalent to a normal cmpxchg, the exchange would fail. Instead exit the loop if x != old_bytes and retry otherwise. Signed-off-by: Janis Schoetterl-Glausch Link: https://lore.kernel.org/r/20221116144711.3811011-1-scgl@linux.ibm.com Signed-off-by: Heiko Carstens

lwn.git/arch/s390/include/asm/cmpxchg.h, branch docs-next

s390: Remove superfluous newlines from inline assemblies

s390/cmpxchg: Provide arch_try_cmpxchg128()

s390/cmpxchg: Provide arch_cmpxchg128_local()

s390/cmpxchg: Implement arch_xchg() with arch_try_cmpxchg()

s390/cmpxchg: Provide arch_try_cmpxchg()

s390/cmpxchg: Convert one and two byte case inline assemblies to C

arch: Remove cmpxchg_double

arch: Introduce arch_{,try_}_cmpxchg128{,_local}()

locking/arch: Rename all internal __xchg() names to __arch_xchg()

s390/cmpxchg: make loop condition for 1,2 byte cases precise

locking/arch: Rename all internal xchg() names to arch_xchg()