summaryrefslogtreecommitdiff
path: root/arch/riscv/kernel/smp.c
AgeCommit message (Collapse)Author
2025-01-20riscv: export __cpuid_to_hartid_mapValentina Fernandez
EXPORT_SYMBOL_GPL() is missing for __cpuid_to_hartid_map array. Export this symbol to allow drivers compiled as modules to use cpuid_to_hartid_map(). Signed-off-by: Valentina Fernandez <valentina.fernandezalanis@microchip.com> Acked-by: Conor Dooley <conor.dooley@microchip.com> Signed-off-by: Jassi Brar <jassisinghbrar@gmail.com>
2024-09-17RISC-V: Implement kgdb_roundup_cpus() to enable future NMI RoundupJinjie Ruan
Until now, the generic weak kgdb_roundup_cpus() has been used for kgdb on RISCV. A custom one allows to debug CPUs that are stuck with interrupts disabled with NMI support in the future. And using an IPI is better than the generic one since it avoids the potential situation described in the generic kgdb_call_nmi_hook(). As Andrew pointed out, once there is NMI support, we can easily extend this and the CPU backtrace support to use NMIs. After this patch, the kgdb test show that: # echo g > /proc/sysrq-trigger [2]kdb> btc btc: cpu status: Currently on cpu 2 Available cpus: 0-1(-), 2, 3(-) Stack traceback for pid 0 0xffffffff81c13a40 0 0 1 0 - 0xffffffff81c14510 swapper/0 CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.10.0-g3120273055b6-dirty #51 Hardware name: riscv-virtio,qemu (DT) Call Trace: [<ffffffff80006c48>] dump_backtrace+0x28/0x30 [<ffffffff80fceb38>] show_stack+0x38/0x44 [<ffffffff80fe6a04>] dump_stack_lvl+0x58/0x7a [<ffffffff80fe6a3e>] dump_stack+0x18/0x20 [<ffffffff801143fa>] kgdb_cpu_enter+0x682/0x6b2 [<ffffffff801144ca>] kgdb_nmicallback+0xa0/0xac [<ffffffff8000a392>] handle_IPI+0x9c/0x120 [<ffffffff800a2baa>] handle_percpu_devid_irq+0xa4/0x1e4 [<ffffffff8009cca8>] generic_handle_domain_irq+0x28/0x36 [<ffffffff800a9e5c>] ipi_mux_process+0xe8/0x110 [<ffffffff806e1e30>] imsic_handle_irq+0xf8/0x13a [<ffffffff8009cca8>] generic_handle_domain_irq+0x28/0x36 [<ffffffff806dff12>] riscv_intc_aia_irq+0x2e/0x40 [<ffffffff80fe6ab0>] handle_riscv_irq+0x54/0x86 [<ffffffff80ff2e4a>] call_on_irq_stack+0x32/0x40 Rebased on Ryo Takakura's "RISC-V: Enable IPI CPU Backtrace" patch. Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Link: https://lore.kernel.org/r/20240727063438.886155-1-ruanjinjie@huawei.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2024-08-07RISC-V: Enable IPI CPU BacktraceRyo Takakura
Add arch_trigger_cpumask_backtrace() which is a generic infrastructure for sampling other CPUs' backtrace using IPI. The feature is used when lockups are detected or in case of oops/panic if parameters are set accordingly. Below is the case of oops with the oops_all_cpu_backtrace enabled. $ sysctl kernel.oops_all_cpu_backtrace=1 triggering oops shows: [ 212.214237] NMI backtrace for cpu 1 [ 212.214390] CPU: 1 PID: 610 Comm: in:imklog Tainted: G OE 6.10.0-rc6 #1 [ 212.214570] Hardware name: riscv-virtio,qemu (DT) [ 212.214690] epc : fallback_scalar_usercopy+0x8/0xdc [ 212.214809] ra : _copy_to_user+0x20/0x40 [ 212.214913] epc : ffffffff80c3a930 ra : ffffffff8059ba7e sp : ff20000000eabb50 [ 212.215061] gp : ffffffff82066f90 tp : ff6000008e958000 t0 : 3463303866660000 [ 212.215210] t1 : 000000000000005b t2 : 3463303866666666 s0 : ff20000000eabb60 [ 212.215358] s1 : 0000000000000386 a0 : 00007ff6e81df926 a1 : ff600000824df800 [ 212.215505] a2 : 000000000000003f a3 : 7fffffffffffffc0 a4 : 0000000000000000 [ 212.215651] a5 : 000000000000003f a6 : 0000000000000000 a7 : 0000000000000000 [ 212.215857] s2 : ff600000824df800 s3 : ffffffff82066cc0 s4 : 0000000000001c1a [ 212.216074] s5 : ffffffff8206a5a8 s6 : 00007ff6e81df926 s7 : ffffffff8206a5a0 [ 212.216278] s8 : ff600000824df800 s9 : ffffffff81e25de0 s10: 000000000000003f [ 212.216471] s11: ffffffff8206a59d t3 : ff600000824df812 t4 : ff600000824df812 [ 212.216651] t5 : ff600000824df818 t6 : 0000000000040000 [ 212.216796] status: 0000000000040120 badaddr: 0000000000000000 cause: 8000000000000001 [ 212.217035] [<ffffffff80c3a930>] fallback_scalar_usercopy+0x8/0xdc [ 212.217207] [<ffffffff80095f56>] syslog_print+0x1f4/0x2b2 [ 212.217362] [<ffffffff80096e5c>] do_syslog.part.0+0x94/0x2d8 [ 212.217502] [<ffffffff800979e8>] do_syslog+0x66/0x88 [ 212.217636] [<ffffffff803a5dda>] kmsg_read+0x44/0x5c [ 212.217764] [<ffffffff80392dbe>] proc_reg_read+0x7a/0xa8 [ 212.217952] [<ffffffff802ff726>] vfs_read+0xb0/0x24e [ 212.218090] [<ffffffff803001ba>] ksys_read+0x64/0xe4 [ 212.218264] [<ffffffff8030025a>] __riscv_sys_read+0x20/0x2c [ 212.218453] [<ffffffff80c4af9a>] do_trap_ecall_u+0x60/0x1d4 [ 212.218664] [<ffffffff80c56998>] ret_from_exception+0x0/0x64 Signed-off-by: Ryo Takakura <takakura@valinux.co.jp> Link: https://lore.kernel.org/r/20240718093659.158912-1-takakura@valinux.co.jp Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2024-04-29riscv: Use IPIs for remote cache/TLB flushes by defaultSamuel Holland
An IPI backend is always required in an SMP configuration, but an SBI implementation is not. For example, SBI will be unavailable when the kernel runs in M mode. For this reason, consider IPI delivery of cache and TLB flushes to be the base case, and any other implementation (such as the SBI remote fence extension) to be an optimization. Generally, if IPIs can be delivered without firmware assistance, they are assumed to be faster than SBI calls due to the SBI context switch overhead. However, when SBI is used as the IPI backend, then the context switch cost must be paid anyway, and performing the cache/TLB flush directly in the SBI implementation is more efficient than injecting an interrupt to S-mode. This is the only existing scenario where riscv_ipi_set_virq_range() is called with use_for_rfence set to false. sbi_ipi_init() already checks riscv_ipi_have_virq_range(), so it only calls riscv_ipi_set_virq_range() when no other IPI device is available. This allows moving the static key and dropping the use_for_rfence parameter. This decouples the static key from the irqchip driver probe order. Furthermore, the static branch only makes sense when CONFIG_RISCV_SBI is enabled. Optherwise, IPIs must be used. Add a fallback definition of riscv_use_sbi_for_rfence() which handles this case and removes the need to check CONFIG_RISCV_SBI elsewhere, such as in cacheflush.c. Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Samuel Holland <samuel.holland@sifive.com> Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Link: https://lore.kernel.org/r/20240327045035.368512-4-samuel.holland@sifive.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2024-01-04riscv: Use the same CPU operations for all CPUsSamuel Holland
RISC-V provides no binding (ACPI or DT) to describe per-cpu start/stop operations, so cpu_set_ops() will always detect the same operations for every CPU. Replace the cpu_ops array with a single pointer to save space and reduce boot time. Signed-off-by: Samuel Holland <samuel.holland@sifive.com> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Link: https://lore.kernel.org/r/20231121234736.3489608-4-samuel.holland@sifive.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-08-08riscv: Fix CPU feature detection with SMP disabledSamuel Holland
commit 914d6f44fc50 ("RISC-V: only iterate over possible CPUs in ISA string parser") changed riscv_fill_hwcap() from iterating over CPU DT nodes to iterating over logical CPU IDs. Since this function runs long before cpu_dev_init() creates CPU devices, it hits the fallback path in of_cpu_device_node_get(), which itself iterates over the DT nodes, searching for a node with the requested CPU ID. (Incidentally, this makes riscv_fill_hwcap() now take quadratic time.) riscv_fill_hwcap() passes a logical CPU ID to of_cpu_device_node_get(), which uses the arch_match_cpu_phys_id() hook to translate the logical ID to a physical ID as found in the DT. arch_match_cpu_phys_id() has a generic weak definition, and RISC-V provides a strong definition using cpuid_to_hartid_map(). However, the RISC-V specific implementation is located in arch/riscv/kernel/smp.c, and that file is only compiled when SMP is enabled. As a result, when SMP is disabled, the generic definition is used, and riscv_isa gets initialized based on the ISA string of hart 0, not the boot hart. On FU740, this means has_fpu() returns false, and userspace crashes when trying to use floating-point instructions. Fix this by moving arch_match_cpu_phys_id() to a file which is always compiled. Fixes: 70114560b285 ("RISC-V: Add RISC-V specific arch_match_cpu_phys_id") Fixes: 914d6f44fc50 ("RISC-V: only iterate over possible CPUs in ISA string parser") Reported-by: Palmer Dabbelt <palmer@rivosinc.com> Signed-off-by: Samuel Holland <samuel.holland@sifive.com> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Link: https://lore.kernel.org/r/20230803012608.3540081-1-samuel.holland@sifive.com Cc: stable@vger.kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-07-04RISC-V: drop error print from riscv_hartid_to_cpuid()Conor Dooley
As of commit 2ac874343749 ("RISC-V: split early & late of_node to hartid mapping") my CI complains about newly added pr_err() messages during boot, for example: [ 0.000000] Couldn't find cpu id for hartid [0] [ 0.000000] riscv-intc: unable to find hart id for /cpus/cpu@0/interrupt-controller Before the split, riscv_of_processor_hartid() contained a check for whether the cpu was "available", before calling riscv_hartid_to_cpuid(), but after the split riscv_of_processor_hartid() can be called for cpus that are disabled. Most callers of riscv_hartid_to_cpuid() already report custom errors where it falls, making this print superfluous in those case. In other places, the print adds nothing - see riscv_intc_init() for example. Fixes: 2ac874343749 ("RISC-V: split early & late of_node to hartid mapping") Signed-off-by: Conor Dooley <conor.dooley@microchip.com> Link: https://lore.kernel.org/r/20230629-paternity-grafted-b901b76d04a0@wendy Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-04-28Merge tag 'smp-core-2023-04-27' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip Pull SMP cross-CPU function-call updates from Ingo Molnar: - Remove diagnostics and adjust config for CSD lock diagnostics - Add a generic IPI-sending tracepoint, as currently there's no easy way to instrument IPI origins: it's arch dependent and for some major architectures it's not even consistently available. * tag 'smp-core-2023-04-27' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: trace,smp: Trace all smp_function_call*() invocations trace: Add trace_ipi_send_cpu() sched, smp: Trace smp callback causing an IPI smp: reword smp call IPI comment treewide: Trace IPIs sent via smp_send_reschedule() irq_work: Trace self-IPIs sent via arch_irq_work_raise() smp: Trace IPIs sent via arch_send_call_function_ipi_mask() sched, smp: Trace IPIs sent via send_call_function_single_ipi() trace: Add trace_ipi_send_cpumask() kernel/smp: Make csdlock_debug= resettable locking/csd_lock: Remove per-CPU data indirection from CSD lock debugging locking/csd_lock: Remove added data from CSD lock debugging locking/csd_lock: Add Kconfig option for csd_debug default
2023-04-08RISC-V: Allow marking IPIs as suitable for remote FENCEsAnup Patel
To do remote FENCEs (i.e. remote TLB flushes) using IPI calls on the RISC-V kernel, we need hardware mechanism to directly inject IPI from the supervisor mode (i.e. RISC-V kernel) instead of using SBI calls. The upcoming AIA IMSIC devices allow direct IPI injection from the supervisor mode (i.e. RISC-V kernel). To support this, we extend the riscv_ipi_set_virq_range() function so that IPI provider (i.e. irqchip drivers can mark IPIs as suitable for remote FENCEs. Signed-off-by: Anup Patel <apatel@ventanamicro.com> Reviewed-by: Atish Patra <atishp@rivosinc.com> Acked-by: Palmer Dabbelt <palmer@rivosinc.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20230328035223.1480939-5-apatel@ventanamicro.com
2023-04-08RISC-V: Treat IPIs as normal Linux IRQsAnup Patel
Currently, the RISC-V kernel provides arch specific hooks (i.e. struct riscv_ipi_ops) to register IPI handling methods. The stats gathering of IPIs is also arch specific in the RISC-V kernel. Other architectures (such as ARM, ARM64, and MIPS) have moved away from custom arch specific IPI handling methods. Currently, these architectures have Linux irqchip drivers providing a range of Linux IRQ numbers to be used as IPIs and IPI triggering is done using generic IPI APIs. This approach allows architectures to treat IPIs as normal Linux IRQs and IPI stats gathering is done by the generic Linux IRQ subsystem. We extend the RISC-V IPI handling as-per above approach so that arch specific IPI handling methods (struct riscv_ipi_ops) can be removed and the IPI handling is done through the Linux IRQ subsystem. Signed-off-by: Anup Patel <apatel@ventanamicro.com> Acked-by: Palmer Dabbelt <palmer@rivosinc.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20230328035223.1480939-4-apatel@ventanamicro.com
2023-04-08RISC-V: Clear SIP bit only when using SBI IPI operationsAnup Patel
The software interrupt pending (i.e. [M|S]SIP) bit is writeable for S-mode but read-only for M-mode so we clear this bit only when using SBI IPI operations. Signed-off-by: Anup Patel <apatel@ventanamicro.com> Reviewed-by: Bin Meng <bmeng.cn@gmail.com> Reviewed-by: Atish Patra <atishp@rivosinc.com> Acked-by: Palmer Dabbelt <palmer@rivosinc.com> Signed-off-by: Marc Zyngier <maz@kernel.org> Link: https://lore.kernel.org/r/20230328035223.1480939-2-apatel@ventanamicro.com
2023-03-24treewide: Trace IPIs sent via smp_send_reschedule()Valentin Schneider
To be able to trace invocations of smp_send_reschedule(), rename the arch-specific definitions of it to arch_smp_send_reschedule() and wrap it into an smp_send_reschedule() that contains a tracepoint. Changes to include the declaration of the tracepoint were driven by the following coccinelle script: @func_use@ @@ smp_send_reschedule(...); @include@ @@ #include <trace/events/ipi.h> @no_include depends on func_use && !include@ @@ #include <...> + + #include <trace/events/ipi.h> [csky bits] [riscv bits] Signed-off-by: Valentin Schneider <vschneid@redhat.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Acked-by: Guo Ren <guoren@kernel.org> Acked-by: Palmer Dabbelt <palmer@rivosinc.com> Link: https://lore.kernel.org/r/20230307143558.294354-6-vschneid@redhat.com
2022-11-29riscv: kexec: Fixup crash_smp_send_stop without multi coresGuo Ren
Current crash_smp_send_stop is the same as the generic one in kernel/panic and misses crash_save_cpu in percpu. This patch is inspired by 78fd584cdec0 ("arm64: kdump: implement machine_crash_shutdown()") and adds the same mechanism for riscv. Before this patch, test result: crash> help -r CPU 0: [OFFLINE] CPU 1: epc : ffffffff80009ff0 ra : ffffffff800b789a sp : ff2000001098bb40 gp : ffffffff815fca60 tp : ff60000004680000 t0 : 6666666666663c5b t1 : 0000000000000000 t2 : 666666666666663c s0 : ff2000001098bc90 s1 : ffffffff81600798 a0 : ff2000001098bb48 a1 : 0000000000000000 a2 : 0000000000000000 a3 : 0000000000000001 a4 : 0000000000000000 a5 : ff60000004690800 a6 : 0000000000000000 a7 : 0000000000000000 s2 : ff2000001098bb48 s3 : ffffffff81093ec8 s4 : ffffffff816004ac s5 : 0000000000000000 s6 : 0000000000000007 s7 : ffffffff80e7f720 s8 : 00fffffffffff3f0 s9 : 0000000000000007 s10: 00aaaaaaaab98700 s11: 0000000000000001 t3 : ffffffff819a8097 t4 : ffffffff819a8097 t5 : ffffffff819a8098 t6 : ff2000001098b9a8 CPU 2: [OFFLINE] CPU 3: [OFFLINE] After this patch, test result: crash> help -r CPU 0: epc : ffffffff80003f34 ra : ffffffff808caa7c sp : ffffffff81403eb0 gp : ffffffff815fcb48 tp : ffffffff81413400 t0 : 0000000000000000 t1 : 0000000000000000 t2 : 0000000000000000 s0 : ffffffff81403ec0 s1 : 0000000000000000 a0 : 0000000000000000 a1 : 0000000000000000 a2 : 0000000000000000 a3 : 0000000000000000 a4 : 0000000000000000 a5 : 0000000000000000 a6 : 0000000000000000 a7 : 0000000000000000 s2 : ffffffff816001c8 s3 : ffffffff81600370 s4 : ffffffff80c32e18 s5 : ffffffff819d3018 s6 : ffffffff810e2110 s7 : 0000000000000000 s8 : 0000000000000000 s9 : 0000000080039eac s10: 0000000000000000 s11: 0000000000000000 t3 : 0000000000000000 t4 : 0000000000000000 t5 : 0000000000000000 t6 : 0000000000000000 CPU 1: epc : ffffffff80003f34 ra : ffffffff808caa7c sp : ff2000000068bf30 gp : ffffffff815fcb48 tp : ff6000000240d400 t0 : 0000000000000000 t1 : 0000000000000000 t2 : 0000000000000000 s0 : ff2000000068bf40 s1 : 0000000000000001 a0 : 0000000000000000 a1 : 0000000000000000 a2 : 0000000000000000 a3 : 0000000000000000 a4 : 0000000000000000 a5 : 0000000000000000 a6 : 0000000000000000 a7 : 0000000000000000 s2 : ffffffff816001c8 s3 : ffffffff81600370 s4 : ffffffff80c32e18 s5 : ffffffff819d3018 s6 : ffffffff810e2110 s7 : 0000000000000000 s8 : 0000000000000000 s9 : 0000000080039ea8 s10: 0000000000000000 s11: 0000000000000000 t3 : 0000000000000000 t4 : 0000000000000000 t5 : 0000000000000000 t6 : 0000000000000000 CPU 2: epc : ffffffff80003f34 ra : ffffffff808caa7c sp : ff20000000693f30 gp : ffffffff815fcb48 tp : ff6000000240e900 t0 : 0000000000000000 t1 : 0000000000000000 t2 : 0000000000000000 s0 : ff20000000693f40 s1 : 0000000000000002 a0 : 0000000000000000 a1 : 0000000000000000 a2 : 0000000000000000 a3 : 0000000000000000 a4 : 0000000000000000 a5 : 0000000000000000 a6 : 0000000000000000 a7 : 0000000000000000 s2 : ffffffff816001c8 s3 : ffffffff81600370 s4 : ffffffff80c32e18 s5 : ffffffff819d3018 s6 : ffffffff810e2110 s7 : 0000000000000000 s8 : 0000000000000000 s9 : 0000000080039eb0 s10: 0000000000000000 s11: 0000000000000000 t3 : 0000000000000000 t4 : 0000000000000000 t5 : 0000000000000000 t6 : 0000000000000000 CPU 3: epc : ffffffff8000a1e4 ra : ffffffff800b7bba sp : ff200000109bbb40 gp : ffffffff815fcb48 tp : ff6000000373aa00 t0 : 6666666666663c5b t1 : 0000000000000000 t2 : 666666666666663c s0 : ff200000109bbc90 s1 : ffffffff816007a0 a0 : ff200000109bbb48 a1 : 0000000000000000 a2 : 0000000000000000 a3 : 0000000000000001 a4 : 0000000000000000 a5 : ff60000002c61c00 a6 : 0000000000000000 a7 : 0000000000000000 s2 : ff200000109bbb48 s3 : ffffffff810941a8 s4 : ffffffff816004b4 s5 : 0000000000000000 s6 : 0000000000000007 s7 : ffffffff80e7f7a0 s8 : 00fffffffffff3f0 s9 : 0000000000000007 s10: 00aaaaaaaab98700 s11: 0000000000000001 t3 : ffffffff819a8097 t4 : ffffffff819a8097 t5 : ffffffff819a8098 t6 : ff200000109bb9a8 Fixes: ad943893d5f1 ("RISC-V: Fixup schedule out issue in machine_crash_shutdown()") Reviewed-by: Xianting Tian <xianting.tian@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Cc: Nick Kossifidis <mick@ics.forth.gr> Link: https://lore.kernel.org/r/20221020141603.2856206-3-guoren@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-08-07Merge tag 'mm-nonmm-stable-2022-08-06-2' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm Pull misc updates from Andrew Morton: "Updates to various subsystems which I help look after. lib, ocfs2, fatfs, autofs, squashfs, procfs, etc. A relatively small amount of material this time" * tag 'mm-nonmm-stable-2022-08-06-2' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (72 commits) scripts/gdb: ensure the absolute path is generated on initial source MAINTAINERS: kunit: add David Gow as a maintainer of KUnit mailmap: add linux.dev alias for Brendan Higgins mailmap: update Kirill's email profile: setup_profiling_timer() is moslty not implemented ocfs2: fix a typo in a comment ocfs2: use the bitmap API to simplify code ocfs2: remove some useless functions lib/mpi: fix typo 'the the' in comment proc: add some (hopefully) insightful comments bdi: remove enum wb_congested_state kernel/hung_task: fix address space of proc_dohung_task_timeout_secs lib/lzo/lzo1x_compress.c: replace ternary operator with min() and min_t() squashfs: support reading fragments in readahead call squashfs: implement readahead squashfs: always build "file direct" version of page actor Revert "squashfs: provide backing_dev_info in order to disable read-ahead" fs/ocfs2: Fix spelling typo in comment ia64: old_rr4 added under CONFIG_HUGETLB_PAGE proc: fix test for "vsyscall=xonly" boot option ...
2022-07-29profile: setup_profiling_timer() is moslty not implementedBen Dooks
The setup_profiling_timer() is mostly un-implemented by many architectures. In many places it isn't guarded by CONFIG_PROFILE which is needed for it to be used. Make it a weak symbol in kernel/profile.c and remove the 'return -EINVAL' implementations from the kenrel. There are a couple of architectures which do return 0 from the setup_profiling_timer() function but they don't seem to do anything else with it. To keep the /proc compatibility for now, leave these for a future update or removal. On ARM, this fixes the following sparse warning: arch/arm/kernel/smp.c:793:5: warning: symbol 'setup_profiling_timer' was not declared. Should it be static? Link: https://lkml.kernel.org/r/20220721195509.418205-1-ben-linux@fluff.org Signed-off-by: Ben Dooks <ben-linux@fluff.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2022-07-19riscv: smp: Add 64bit hartid support on RV64Sunil V L
The hartid can be a 64bit value on RV64 platforms. Modify the hartid parameter in riscv_hartid_to_cpuid() as unsigned long so that it can hold 64bit value on RV64 platforms. Signed-off-by: Sunil V L <sunilvl@ventanamicro.com> Reviewed-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com> Reviewed-by: Atish Patra <atishp@rivosinc.com> Link: https://lore.kernel.org/r/20220527051743.2829940-4-sunilvl@ventanamicro.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-01-09RISC-V: Use common riscv_cpuid_to_hartid_mask() for both SMP=y and SMP=nSean Christopherson
Use what is currently the SMP=y version of riscv_cpuid_to_hartid_mask() for both SMP=y and SMP=n to fix a build failure with KVM=m and SMP=n due to boot_cpu_hartid not being exported. This also fixes a second bug where the SMP=n version assumes the sole CPU in the system is in the incoming mask, which may not hold true in kvm_riscv_vcpu_sbi_ecall() if the KVM guest VM has multiple vCPUs (on a SMP=n system). Fixes: 1ef46c231df4 ("RISC-V: Implement new SBI v0.2 extensions") Reported-by: Adam Borowski <kilobyte@angband.pl> Reviewed-by: Anup Patel <anup.patel@wdc.com> Signed-off-by: Sean Christopherson <seanjc@google.com> Cc: stable@vger.kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2021-10-26irq: riscv: perform irqentry in entry codeMark Rutland
In preparation for removing HANDLE_DOMAIN_IRQ_IRQENTRY, have arch/riscv perform all the irqentry accounting in its entry code. As arch/riscv uses GENERIC_IRQ_MULTI_HANDLER, we can use generic_handle_arch_irq() to do so. Since generic_handle_arch_irq() handles the irq entry and setting the irq regs, and happens before the irqchip code calls handle_IPI(), we can remove the redundant irq entry and irq regs manipulation from handle_IPI(). There should be no functional change as a result of this patch. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reviewed-by: Guo Ren <guoren@kernel.org> Reviewed-by: Marc Zyngier <maz@kernel.org> Cc: Albert Ou <aou@eecs.berkeley.edu> Cc: Palmer Dabbelt <palmer@dabbelt.com> Cc: Paul Walmsley <paul.walmsley@sifive.com> Cc: Thomas Gleixner <tglx@linutronix.de>
2021-05-01RISC-V: Fix error code returned by riscv_hartid_to_cpuid()Anup Patel
We should return a negative error code upon failure in riscv_hartid_to_cpuid() instead of NR_CPUS. This is also aligned with all uses of riscv_hartid_to_cpuid() which expect negative error code upon failure. Fixes: 6825c7a80f18 ("RISC-V: Add logical CPU indexing for RISC-V") Fixes: f99fb607fb2b ("RISC-V: Use Linux logical CPU number instead of hartid") Signed-off-by: Anup Patel <anup.patel@wdc.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-26riscv: Constify sbi_ipi_opsJisheng Zhang
Constify the sbi_ipi_ops so that it will be placed in the .rodata section. This will cause attempts to modify it to fail when strict page permissions are in place. Signed-off-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-04-26riscv: Mark some global variables __ro_after_initJisheng Zhang
All of these are never modified after init, so they can be __ro_after_init. Signed-off-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-03-16riscv: Enable generic clockevent broadcastGuo Ren
When percpu-timers are stopped by deep power saving mode, we need system timer help to broadcast IPI_TIMER. This is first introduced by broken x86 hardware, where the local apic timer stops in C3 state. But many other architectures(powerpc, mips, arm, hexagon, openrisc, sh) have supported the infrastructure to deal with Power Management issues. Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-08-20RISC-V: Remove CLINT related code from timer and archAnup Patel
Right now the RISC-V timer driver is convoluted to support: 1. Linux RISC-V S-mode (with MMU) where it will use TIME CSR for clocksource and SBI timer calls for clockevent device. 2. Linux RISC-V M-mode (without MMU) where it will use CLINT MMIO counter register for clocksource and CLINT MMIO compare register for clockevent device. We now have a separate CLINT timer driver which also provide CLINT based IPI operations so let's remove CLINT MMIO related code from arch/riscv directory and RISC-V timer driver. Signed-off-by: Anup Patel <anup.patel@wdc.com> Tested-by: Emil Renner Berhing <kernel@esmil.dk> Acked-by: Daniel Lezcano <daniel.lezcano@linaro.org> Reviewed-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-08-20RISC-V: Add mechanism to provide custom IPI operationsAnup Patel
We add mechanism to set custom IPI operations so that CLINT driver from drivers directory can provide custom IPI operations. Signed-off-by: Anup Patel <anup.patel@wdc.com> Tested-by: Emil Renner Berhing <kernel@esmil.dk> Reviewed-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-07-30riscv: Support irq_work via self IPIsGreentime Hu
Support for arch_irq_work_raise() and arch_irq_work_has_interrupt() was missing from riscv (a prerequisite for FULL_NOHZ). Signed-off-by: Greentime Hu <greentime.hu@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-06-09RISC-V: self-contained IPI handling routineAnup Patel
Currently, the IPI handling routine riscv_software_interrupt() does not take any argument and also does not perform irq_enter()/irq_exit(). This patch makes IPI handling routine more self-contained by: 1. Passing "pt_regs *" argument 2. Explicitly doing irq_enter()/irq_exit() 3. Explicitly save/restore "pt_regs *" using set_irq_regs() With above changes, IPI handling routine does not depend on caller function to perform irq_enter()/irq_exit() and save/restore of "pt_regs *" hence its more self-contained. This also enables us to call IPI handling routine from IRQCHIP drivers. Signed-off-by: Anup Patel <anup.patel@wdc.com> Reviewed-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Acked-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-05-04RISC-V: Export riscv_cpuid_to_hartid_mask() APIAnup Patel
The riscv_cpuid_to_hartid_mask() API should be exported to allow building KVM RISC-V as loadable module. Signed-off-by: Anup Patel <anup.patel@wdc.com> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-18riscv: fix the IPI missing issue in nommu modeGreentime Hu
This patch fixes the IPI(inner processor interrupt) missing issue. It failed because it used hartid_mask to iterate for_each_cpu(), however the cpu_mask and hartid_mask may not be always the same. It will never send the IPI to hartid 4 because it will be skipped in for_each_cpu loop in my case. We can reproduce this case in Qemu sifive_u machine by this command. qemu-system-riscv64 -nographic -smp 5 -m 1G -M sifive_u -kernel \ arch/riscv/boot/loader It will hang in csd_lock_wait(csd) because the csd_unlock(csd) is not called. It is not called because hartid 4 doesn't receive the IPI to release this lock. The caller hart doesn't send the IPI to hartid 4 is because of hartid 4 is skipped in for_each_cpu(). It will be skipped is because "(cpu) < nr_cpu_ids" is not true. The hartid is 4 and nr_cpu_ids is 4. Therefore it should use cpumask in for_each_cpu() instead of hartid_mask. /* Send a message to all CPUs in the map */ arch_send_call_function_ipi_mask(cfd->cpumask_ipi); if (wait) { for_each_cpu(cpu, cfd->cpumask) { call_single_data_t *csd; csd = per_cpu_ptr(cfd->csd, cpu); csd_lock_wait(csd); } } for ((cpu) = -1; \ (cpu) = cpumask_next((cpu), (mask)), \ (cpu) < nr_cpu_ids;) It could boot to login console after this patch applied. Fixes: b2d36b5668f6 ("riscv: provide native clint access for M-mode") Signed-off-by: Greentime Hu <greentime.hu@sifive.com> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2019-11-17riscv: provide native clint access for M-modeChristoph Hellwig
RISC-V has the concept of a cpu level interrupt controller. The interface for it is split between a standardized part that is exposed as bits in the mstatus/sstatus register and the mie/mip/sie/sip CRS. But the bit to actually trigger IPIs is not standardized and just mentioned as implementable using MMIO. Add support for IPIs using MMIO using the SiFive clint layout (which is also shared by Ariane, Kendryte and the Qemu virt platform). Additionally the MMIO block also supports the time value and timer compare registers, so they are also set up using the same OF node. Support for other layouts should also be relatively easy to add in the future. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> [paul.walmsley@sifive.com: update include guard format; fix checkpatch issues; minor commit message cleanup] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-11-05riscv: abstract out CSR names for supervisor vs machine modeChristoph Hellwig
Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28riscv: add missing header file includesPaul Walmsley
sparse identifies several missing prototypes caused by missing preprocessor include directives: arch/riscv/kernel/cpufeature.c:16:6: warning: symbol 'has_fpu' was not declared. Should it be static? arch/riscv/kernel/process.c:26:6: warning: symbol 'arch_cpu_idle' was not declared. Should it be static? arch/riscv/kernel/reset.c:15:6: warning: symbol 'pm_power_off' was not declared. Should it be static? arch/riscv/kernel/syscall_table.c:15:6: warning: symbol 'sys_call_table' was not declared. Should it be static? arch/riscv/kernel/traps.c:149:13: warning: symbol 'trap_init' was not declared. Should it be static? arch/riscv/kernel/vdso.c:54:5: warning: symbol 'arch_setup_additional_pages' was not declared. Should it be static? arch/riscv/kernel/smp.c:64:6: warning: symbol 'arch_match_cpu_phys_id' was not declared. Should it be static? arch/riscv/kernel/module-sections.c:89:5: warning: symbol 'module_frob_arch_sections' was not declared. Should it be static? arch/riscv/mm/context.c:42:6: warning: symbol 'switch_mm' was not declared. Should it be static? Fix by including the appropriate header files in the appropriate source files. This patch should have no functional impact. Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com> Reviewed-by: Christoph Hellwig <hch@lst.de>
2019-09-20RISC-V: Export kernel symbols for kvmAtish Patra
Export a few symbols used by kvm module. Without this, kvm cannot be compiled as a module. Signed-off-by: Atish Patra <atish.patra@wdc.com> Signed-off-by: Anup Patel <anup.patel@wdc.com> Acked-by: Paolo Bonzini <pbonzini@redhat.com> Reviewed-by: Paolo Bonzini <pbonzini@redhat.com> Reviewed-by: Alexander Graf <graf@amazon.com> [paul.walmsley@sifive.com: updated to apply; clarified short patch description] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-09-05riscv: cleanup riscv_cpuid_to_hartid_maskChristoph Hellwig
Move the initial clearing of the mask from the callers to riscv_cpuid_to_hartid_mask, and remove the unused !CONFIG_SMP stub. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Atish Patra <atish.patra@wdc.com> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-09-05riscv: optimize send_ipi_singleChristoph Hellwig
Don't go through send_ipi_mask, but just set the op bit and then pass a simple generated hartid mask directly to sbi_send_ipi. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Atish Patra <atish.patra@wdc.com> [paul.walmsley@sifive.com: minor patch description fixes] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-09-05riscv: cleanup send_ipi_maskChristoph Hellwig
Use the special barriers for atomic bitops to make the intention a little more clear, and use riscv_cpuid_to_hartid_mask instead of open coding it. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Atish Patra <atish.patra@wdc.com> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-09-05riscv: refactor the IPI codeChristoph Hellwig
This prepares for adding native non-SBI IPI code. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Atish Patra <atish.patra@wdc.com> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-06-19treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 234Thomas Gleixner
Based on 1 normalized pattern(s): this program is free software you can redistribute it and or modify it under the terms of the gnu general public license version 2 as published by the free software foundation this program is distributed in the hope that it will be useful but without any warranty without even the implied warranty of merchantability or fitness for a particular purpose see the gnu general public license for more details you should have received a copy of the gnu general public license along with this program if not see http www gnu org licenses extracted by the scancode license scanner the SPDX license identifier GPL-2.0-only has been chosen to replace the boilerplate/reference in 503 file(s). Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Alexios Zavras <alexios.zavras@intel.com> Reviewed-by: Allison Randal <allison@lohutok.net> Reviewed-by: Enrico Weigelt <info@metux.net> Cc: linux-spdx@vger.kernel.org Link: https://lkml.kernel.org/r/20190602204653.811534538@linutronix.de Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2019-05-16riscv: move flush_icache_{all,mm} to cacheflush.cGary Guo
Currently, flush_icache_all is macro-expanded into a SBI call, yet no asm/sbi.h is included in asm/cacheflush.h. This could be moved to mm/cacheflush.c instead (SBI call will dominate performance-wise and there is no worry to not have it inlined. Currently, flush_icache_mm stays in kernel/smp.c, which looks like a hack to prevent it from being compiled when CONFIG_SMP=n. It should also be in mm/cacheflush.c. Signed-off-by: Gary Guo <gary@garyguo.net> Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2019-05-16RISC-V: Access CSRs using CSR numbersAnup Patel
We should prefer accessing CSRs using their CSR numbers because: 1. It compiles fine with older toolchains. 2. We can use latest CSR names in #define macro names of CSR numbers as-per RISC-V spec. 3. We can access newly added CSRs even if toolchain does not recognize newly addes CSRs by name. Signed-off-by: Anup Patel <anup.patel@wdc.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2019-05-16RISC-V: Fix minor checkpatch issues.Atish Patra
While working on the patches, I found some minor checkpatch issues. Signed-off-by: Atish Patra <atish.patra@wdc.com> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2019-04-30RISC-V: Add RISC-V specific arch_match_cpu_phys_idAtish Patra
OF/DT core has a hook for architecture specific logical cpuid to hartid mapping. By implementing this, we can pass the logical cpu id to cpu node parsing functions. Fix the instances where logical cpuid is expected as an argument in of_get_cpu_node. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Sudeep Holla <sudeep.holla@arm.com> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2019-03-04RISC-V: Fixmap support and MM cleanupsPalmer Dabbelt
This patchset does: 1. Moves MM related code from kernel/setup.c to mm/init.c 2. Implements compile-time fixed mappings Using fixed mappings, we get earlyprints even without SBI calls. For example, we can now use kernel parameter "earlycon=uart8250,mmio,0x10000000" to get early prints on QEMU virt machine without using SBI calls. The patchset is tested on QEMU virt machine. Palmer: It looks like some of the code movement here conflicted with the patches to move hartid handling around. As far as I can tell the only changed code was in smp_setup_processor_id(), and I've kept the one in smp.c.
2019-03-04RISC-V: Allow hartid-to-cpuid function to fail.Atish Patra
It is perfectly okay to call riscv_hartid_to_cpuid for a hartid that is not mapped with an CPU id. It can happen if the calling functions retrieves the hartid from DT. However, that hartid was never brought online by the firmware or kernel for any reasons. No need to BUG() in the above case. A negative error return is sufficient and the calling function should check for the return value always. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2019-03-04RISC-V: Move cpuid to hartid mapping to SMP.Atish Patra
Currently, logical CPU id to physical hartid mapping is defined for both smp and non-smp configurations. This is not required as we need this only for smp configuration. The mapping function can define directly boot_cpu_hartid for non-smp use case. The reverse mapping function i.e. hartid to cpuid can be called for any valid but not booted harts. So it should return default cpu 0 only if it is a boot hartid. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2019-01-07riscv: don't stop itself in smp_send_stopAndreas Schwab
Add IPI_CPU_STOP message and use it in smp_send_stop to stop other cpus, but not itself. Mark cpu offline on reception of IPI_CPU_STOP. Signed-off-by: Andreas Schwab <schwab@suse.de> Reviewed-by: Atish Patra <atish.patra@wdc.com> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2018-10-22RISC-V: Show IPI statsAnup Patel
This patch provides arch_show_interrupts() implementation to show IPI stats via /proc/interrupts. Now the contents of /proc/interrupts" will look like below: CPU0 CPU1 CPU2 CPU3 8: 17 7 6 14 SiFive PLIC 8 virtio0 10: 10 10 9 11 SiFive PLIC 10 ttyS0 IPI0: 170 673 251 79 Rescheduling interrupts IPI1: 1 12 27 1 Function call interrupts Signed-off-by: Anup Patel <anup@brainfault.org> [Atish - Fixed checkpatch errors] Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Palmer Dabbelt <palmer@sifive.com> Changes since v2: - Remove use of IPI_CALL_WAKEUP because it's being removed Changes since v1: - Add stub inline show_ipi_stats() function for !CONFIG_SMP - Make ipi_names[] dynamically sized at compile time - Minor beautification of ipi_names[] using tabs Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2018-10-22RISC-V: Use Linux logical CPU number instead of hartidAtish Patra
Setup the cpu_logical_map during boot. Moreover, every SBI call and PLIC context are based on the physical hartid. Use the logical CPU to hartid mapping to pass correct hartid to respective functions. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2018-10-22RISC-V: Add logical CPU indexing for RISC-VAtish Patra
Currently, both Linux CPU id and hart id are same. This is not recommended as it will lead to discontinuous CPU indexing in Linux. Moreover, kdump kernel will run from CPU0 which would be absent if we follow existing scheme. Implement a logical mapping between Linux CPU id and hart id to decouple these two. Always mark the boot processor as CPU0 and all other CPUs get the logical CPU id based on their booting order. Signed-off-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Anup Patel <anup@brainfault.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2018-08-13RISC-V: simplify software interrupt / IPI codeChristoph Hellwig
Rename handle_ipi to riscv_software_interrupt, drop the unused return value and move the prototype to irq.h together with riscv_timer_interupt. This allows simplifying the upcoming interrupt handling support. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2017-12-01RISC-V: Fixes for clean allmodconfig buildPalmer Dabbelt
Olaf said: Here's a short series of patches that produces a working allmodconfig. Would be nice to see them go in so we can add build coverage. I've dropped patches 8 and 10 from the original set: * [PATCH 08/10] (RISC-V: Set __ARCH_WANT_RENAMEAT to pick up generic version) has a better fix that I've sent out for review, we don't want renameat. * [PATCH 10/10] (input: joystick: riscv has get_cycles) has already been taken into Dmitry Torokhov's tree.