summaryrefslogtreecommitdiff
path: root/arch/riscv/include/asm/processor.h
AgeCommit message (Collapse)Author
2026-10-08riscv: Add a getter for user PMLEN supportAndrew Jones
Querying whether a given user PMLEN is supported is needed for RVA23U64 base detection from outside arch/riscv/kernel/process.c. Add riscv_have_user_pmlen() to expose this. Link: https://lore.kernel.org/linux-riscv/rfuwa7a3ebe76udmnwyrssjy7shkkgxntvhwzn6oquysj4tuyp@xzvpylcfhz53/ Signed-off-by: Andrew Jones <andrew.jones@oss.qualcomm.com> [Guodong: replace exported booleans with getter per Andrew's suggestion] Signed-off-by: Guodong Xu <guodong.xu@oss.qualcomm.com> Link: https://patch.msgid.link/20260920-rva23u64-hwprobe-v2-v8-8-5f14bc5c23d3@oss.qualcomm.com Signed-off-by: Paul Walmsley <pjw@kernel.org>
2026-10-08riscv: Add ERRATA_MIPS_P8700_WFI to replace WFI with mips.pauseDjordje Todorovic
The MIPS P8700 has bugs with the WFI instruction. A WFI instruction after a mispredicted branch can result in erroneous address translation on the MIPS P8700. MIPS has not published an erratum for this. The workaround uses the RISC-V alternatives framework to patch all WFI instructions with the MIPS P8700 pause opcode (0x00501013) at runtime when running on P8700 hardware. Two call sites are patched: - arch/riscv/kernel/head.S: secondary hart parking loop - arch/riscv/include/asm/processor.h: wait_for_interrupt() Signed-off-by: Djordje Todorovic <djordje.todorovic@htecgroup.com> Signed-off-by: Aleksa Paunovic <aleksa.paunovic@htecgroup.com> Link: https://patch.msgid.link/20260917-p8700-wfi-v3-1-ada4b0584712@htecgroup.com [pjw@kernel.org: updated to apply] Signed-off-by: Paul Walmsley <pjw@kernel.org>
2026-08-25Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvmLinus Torvalds
Pull kvm updates from Paolo Bonzini: "ARM64: - Add support for 'slot' based PMU events, paired with new UAPI that compels the user to select a specific PMU implementation - Lazy save/restore of vCPU state for pKVM, along with various fixes and cleanups to the management of vCPU state between the untrusted host and pKVM hypervisor - Disable traps of EL1 registers for nested hypervisors when FEAT_NV2p1 is present, guaranteeing that EL2-specific register bits are stateful in the EL1 counterpart - Leverage FEAT_NV3 to avoid unnecessary ERET/TLBI traps when the scope of those instructions remains 'in host' (i.e. L1 kernel/userspace) - Pile of fixes for the management of the VNCR pseudo-TLB, such as under-invalidations and races with concurrent TLBIs on other vCPUs - Consolidate the non-protected and pKVM view of ICH_VTR_EL2 to a runtime-patched constant, allowing the same data to be shared with pKVM prior to dropping host privileges - Considerable pile of LLM-assisted fixes around the shop but mostly in the VGIC, our in-kernel generator of bugs (and sometimes interrupts) LoongArch: - Advertise already-supported capabilities - Some bug fixes about timer and MMIO - Some hardening about interrupt injection - Replace kvm_err() with kvm_pr_unimpl() - Add FPU/LSX/LASX test cases for selftests RISC-V: - Svadu/Zicfiss/Zicfilp FWFT support for Guest - Use try_cmpxchg for IMSIC MRIF RMW - More arch-specific tracepoints in KVM RISC-V - Eager page splitting when enabling dirty logging - Optimize hfence request handling for SMP Guests - Improve dirty log clearing by skipping zero bits in mask - Guard HFENCE range loops against overflow - CPU PM notifiers in KVM RISC-V for non-retentive idle states - Fix kernel-mode vector context save/restore for Guest s390: - Fixes for vfio-ap - Fixes for the gmap rework - Fixes for vsie - AI triggered fixes all over - diag9c tracing - code move preparation for the additional arm64 support - enable CONTEXT_ANALYSIS x86: - Perform spring cleaning on x86.{c,h} and asm/kvm_host.h, by adding regs.c (the kvm_cache_regs.h => regs.h is already applied) and msrs.{c,h}, and moving relevant code out of x86.c - Split kvm_mmu in three parts, respectively to describe the format of page tables, walking the guest page tables and building the page tables. Always use the same page table walker kvm->arch.gva_walk as the entry point to convert a guest's virtual address, where the previous code used two different kvm_mmu structs depending on whether the walk included nested EPT/NPT or not. Make page fault vmexits reuse the permission checking machinery that is used for guest page faults. This is both a cleanup and a baby step towards supporting XS/XU memory permissions - Document some of the "fun" gotchas with the APIC base when creating IRQCHIPs on x86 - Remove a defunct masterclock update from kvm_xen_shared_info_init(). It could result in incorrect kvmclock due to triggering an unnecessary switch to/from masterclock mode - Skip Xen runstate time updates if time has effectively gone backwards, so that the guest doesn't report 100% steal time for a very, very long time - Drop KVM's runtime updates of the Xen PV timing CPUID leaf, as KVM was updating the wrong sub-leaf, and upstream KVM will soon provide all the information needed by userspace to populate the CPUID field itself - Fix a bug where KVM would walk a newly created rmap without holding the rmap lock (or mmu_lock) during aging - Fix a bug where aging TDP MMU SPTEs could clobber FROZEN SPTEs - Fix a variety of #DB priority bugs - Fix a class of races related to enabling Hyper-V emulation on a vCPU after the vCPU is visible to the rest of KVM - Use static calls for nested virtualization ops - Move more KVM-internal code out of x86's kvm_host.h - Enumerate support for a variety of Zhaoxin instructions that don't require explicit virtualization - Fix missing EFER validation bugs, including in the KVM_SET_SREGS* path - Harden kvm_vcpu_map() against double-mapping and thus leaking references - Misc fixes and cleanups, e.g. for largely benign syzkaller splats x86 (Intel): - Zero a vCPU's entry in VMX's Posted Interrupt Descriptor table used for IPI virtualization when the vCPU is freed, to fix a use-after-free where hardware will write to a freed vCPU's PID - Service local TLB flushes on a failed nested VM-Enter to fix a bug where KVM could miss a TLB on a future, successful VM-Enter with the same L2 VPID - Cap the maximum value shoved into the VMX Preemption Timer to workaround an erratum that affects all existing Intel CPUs that support CPUID 0x15 - Fix VPID virtualization bugs where KVM would fail to flush hardware TLBs - Harden the TDX "populate" ioctls against bad input, and to prepare for supporting in-place private<=>shared conversion x86 (AMD): - Forcefully invalidate SNP VMSA pages if their backing guest_memfd page is zapped/invalidated, e.g. due to a PUNCH_HOLE in response to a Page-State Change request - Remove a dying VM from the GA Log notifier list before the VM is actually destroyed, to fix a potential use-after-free - While FOLL_WRITE was needed in the past to trigger CoW unsharing, nowadays FOLL_LONGTERM does that already even without FOLL_WRITE, and in fact, get_user_pages() actually disallows FOLL_WRITE together with FOLL_LONGTERM. So don't pass FOLL_WRITE when registering encrypted memory regions, i.e. when pinning SEV/SEV-ES guest memory, to fix a regression with file-backed memory introduced by KVM's (correct) usage of long-term pins (This was reviewed by mm maintainers; for more information, see commit ee1a586dd1fa "KVM: SEV: Drop FOLL_WRITE for encrypted region registration") - Allocate full pages for SEV/SEV-ES {DE,EN}CRYPT ops on SNP-enabled hosts to fix a data corruption issue due to the PSP driver assigning to-be-written pages to firmware (as required by the SNP specs) - Unconditionally intercept ICBEP so that KVM generates the correct guest RIP when handling an ICEBP-induced TASK_SWITCH #VMEXIT - Harden the SNP "populate" ioctls against bad input, and to prepare for supporting in-place private<=>shared conversion Generic: - Remove kvm_debugfs_dir if kvm_init() fails after creating KVM's debugfs - Add a per-VM bitmap to track which vCPU IDs have been "claimed" but for which the vCPU isn't yet online, and use the bitmap to reject duplicate IDs before calling into arch code. This allows arch code to consume vcpu_id without having to worry about cross-vCPU clobbering (at least s390 and x86 have had related bugs) - Rework the so called "prepare" and "invalidate" guest_memfd hooks to prepare for in-place private<=>shared conversion, and clean up a few warts along the way Selftests: - Automatically allocate a full page for L2 guest stacks on x86 instead of requiring test-specific L1 guest code to carve out a portion of the L1 stack for L2 usage, and to ensure the L2 stack also adheres to the x86-64 calling convention ABI - Add a selftest to verify {Guest,Host}-Only behavior in x86's mediated PMU - Clean up nested SVM's handling of GPRs on L2<=>L1 transitions, reuse the functionality for nested VMX, and drop the ucall hack that was fudging around the lack of GPR switching on nVMX - Add a stress test to verify KVM doesn't clobber/drop #PF state, e.g. CR2, across save/restore, including when L2 is active - Add a test to verify KVM_CREATE_VM accepts exactly what is reported by KVM_CAP_VM_TYPES - Misc selftests fixes and cleanups - Fix several issues with seeding the pRNG, and rework the pRNG APIs to that the pRNG can be sanely used in host code, not just guest code - Add an IRQ test to validate virtual IRQ deliverty for IRQs wired up via KVM_IRQFD + KVM_SET_GSI_ROUTING, with optional support for triggering IRQs via writes to an assigned VFIO device - Add syscall wrappers to assert success on a variety of pthreads and CPU affinity APIs - Set vCPU pthread affinity as early as possible to reduce contention issues that were surfaced by PREEMPT_LAZY, which result in runtimes of over a minute on large hosts, versus the expected ~5 seconds - Rework the PMU counters test to run each testcase using a single VM with many vCPUs for each sub-testcase, instead of using a unique VM for each sub-testcase. This cuts the runtime by ~20x Miscellaneous: - MAINTAINERS updates for vfio-ap, guest_memfd, kvm-x86. Mostly representing the status quo more accurately, but also... welcome David Hildenbrand as guest_memfd reviewer!" * tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (413 commits) KVM: arm64: Validate GICv5 timer PPIs before claiming ownership KVM: arm64: vgic: Reject out-of-range GICv5 PPI IDs KVM: arm64: vgic: Prevent speculative SPI array underflow KVM: arm64: vgic: Free gic_kvm_info on initialization failure KVM: arm64: Avoid mismatched accesses to 'struct kvm_nvhe_init_params' s390/vfio-ap: Fix NULL deref in status_show() during queue probe s390/vfio-ap: Fix hot-unplug skipped when last AP adapter or domain removed s390/vfio-ap: fix potential use of uninitialized apm_filtered bitmap s390/vfio-ap: Fix control domain removal in vfio_ap_mdev_cfg_remove s390/vfio-ap: Fix required lock not held during update of ap_matrix_mdev object s390/vfio-ap: Fix missing lock required to access list of ap_matrix_mdev objects s390/vfio-ap: Fix dereference matrix_mdev->kvm without checking for NULL s390/vfio-ap: Fix stale do_remove flag across iterations in vfio_ap_mdev_cfg_remove RISC-V: KVM: fix vcpu vector context handling for kernel-mode vector riscv: vector: allow non-preemptible kernel-mode vector with IRQs off riscv: vector: refactor riscv_v_start_kernel_context KVM: s390: gmap: Make prefix handling optional KVM: s390: gmap: Make CMMA optional KVM: s390: gmap: Make storage keys optional KVM: s390: Prepare gmap for a second KVM implementation ...
2026-08-13RISC-V: KVM: fix vcpu vector context handling for kernel-mode vectorAndy Chiu
Running vector workloads like perf + mcf on KVM can result in an unexpected termination due to a vtype corruption. This happens because the kernel-mode vector (KMV) misattributes the guest's vcpu context as the user's context and source from a wrong status.VS. The simplified call chain that results in this problem is shown as follow: __riscv_sys_ioctl() kvm_arch_vcpu_ioctl_run() kvm_riscv_vcpu_exit() kvm_riscv_vcpu_sbi_ecall() kvm_riscv_vcpu_pmu_ctr_stop() kvm_vcpu_write_guest() __copy_to_user() enter_vector_usercopy() kernel_vector_begin() kernel_vector_begin() should use the sstatus.VS from guest's vcpu context instead of task_pt_reg(current). Also, it should not save guest's v-reg into the user's context memory. To resolve this, the vcpu context must be correctly saved when KMV is serving a guest. However, invoking KVM functions directly from generic RISC-V architecture code introduces a reverse dependency, breaking builds when KVM is configured as N or M. Address this by registering an RCU-protected callback for context flushing. KVM registers this callback at module initialization and unregisters it on exit. When KMV starts a kernel context, it can now safely flush the vector context via the callback. Fixes: ecd2ada8a5e0 ("riscv: Add support for kernel mode vector") Signed-off-by: Andy Chiu <tchiu@tenstorrent.com> Reviewed-by: Yong-Xuan Wang <yongxuan.wang@sifive.com> Reviewed-by: Anup Patel <anup@brainfault.org> Link: https://lore.kernel.org/r/20260803215250.824417-4-tchiu@tenstorrent.com Signed-off-by: Anup Patel <anup@brainfault.org>
2026-08-07riscv: Add support for srmcfg CSR from Ssqosid extensionDrew Fustini
Add support for the srmcfg CSR defined in the Ssqosid ISA extension. The CSR contains two fields: - Resource Control ID (RCID) for resource allocation - Monitoring Counter ID (MCID) for tracking resource usage Requests from a hart to shared resources are tagged with these IDs, allowing resource usage to be associated with the running task. Add a srmcfg field to thread_struct with the same format as the CSR. The context-switch path writes the field to the CSR, and resctrl_arch_set_closid_rmid() updates it when a task is assigned to a resctrl control or monitoring group. A per-cpu cpu_srmcfg_default holds the default srmcfg for each CPU, set by resctrl_arch_set_cpu_default_closid_rmid() on CPU group assignment. On context switch, RCID and MCID inherit from the CPU default independently: a task whose thread RCID field is zero takes the CPU default's RCID, and likewise for MCID. A per-cpu cpu_srmcfg variable mirrors the CSR state to avoid redundant writes. L1D-hot memory access is faster than a CSR read and avoids traps under virtualization. Link: https://github.com/riscv/riscv-ssqosid/releases/tag/v1.0 Assisted-by: Claude:claude-opus-4-7 Co-developed-by: Kornel Dulęba <mindal@semihalf.com> Signed-off-by: Kornel Dulęba <mindal@semihalf.com> Signed-off-by: Drew Fustini <fustini@kernel.org> Link: https://patch.msgid.link/20260729-dfustini-atl-sc-cbqri-dt-v6-3-7c22b05d461b@kernel.org Signed-off-by: Paul Walmsley <pjw@kernel.org>
2026-04-04riscv: fix various typos in comments and codeSean Chang
Fix various typos in RISC-V architecture code and comments. The following changes are included: - arch/riscv/errata/thead/errata.c: "futher" → "further" - arch/riscv/include/asm/atomic.h: "therefor" → "therefore", "arithmatic" → "arithmetic" - arch/riscv/include/asm/elf.h: "availiable" → "available", "coorespends" → "corresponds" - arch/riscv/include/asm/processor.h: "requries" → "is required" - arch/riscv/include/asm/thread_info.h: "returing" → "returning" - arch/riscv/kernel/acpi.c: "compliancy" → "compliance" - arch/riscv/kernel/ftrace.c: "therefor" → "therefore" - arch/riscv/kernel/head.S: "intruction" → "instruction" - arch/riscv/kernel/mcount-dyn.S: "localtion → "location" - arch/riscv/kernel/module-sections.c: "maxinum" → "maximum" - arch/riscv/kernel/probes/kprobes.c: "reenabled" → "re-enabled" - arch/riscv/kernel/probes/uprobes.c: "probbed" → "probed" - arch/riscv/kernel/soc.c: "extremly" → "extremely" - arch/riscv/kernel/suspend.c: "incosistent" → "inconsistent" - arch/riscv/kvm/tlb.c: "cahce" → "cache" - arch/riscv/kvm/vcpu_pmu.c: "indicies" → "indices" - arch/riscv/lib/csum.c: "implmentations" → "implementations" - arch/riscv/lib/memmove.S: "ammount" → "amount" - arch/riscv/mm/cacheflush.c: "visable" → "visible" - arch/riscv/mm/physaddr.c: "aginst" → "against" Signed-off-by: Sean Chang <seanwascoding@gmail.com> Acked-by: Conor Dooley <conor.dooley@microchip.com> Link: https://patch.msgid.link/20260212163325.60389-1-seanwascoding@gmail.com Signed-off-by: Paul Walmsley <pjw@kernel.org>
2026-01-25riscv: Add usercfi state for task and save/restore of CSR_SSP on trap entry/exitDeepak Gupta
Carve out space in the RISC-V architecture-specific thread struct for cfi status and shadow stack in usermode. This patch: - defines a new structure cfi_status with status bit for cfi feature - defines shadow stack pointer, base and size in cfi_status structure - defines offsets to new member fields in thread in asm-offsets.c - saves and restores shadow stack pointer on trap entry (U --> S) and exit (S --> U) Shadow stack save/restore is gated on feature availability and is implemented using alternatives. CSR_SSP can be context-switched in 'switch_to' as well, but as soon as kernel shadow stack support gets rolled in, the shadow stack pointer will need to be switched at trap entry/exit point (much like 'sp'). It can be argued that a kernel using a shadow stack deployment scenario may not be as prevalent as user mode using this feature. But even if there is some minimal deployment of kernel shadow stack, that means that it needs to be supported. Thus save/restore of shadow stack pointer is implemented in entry.S instead of in 'switch_to.h'. Reviewed-by: Charlie Jenkins <charlie@rivosinc.com> Reviewed-by: Zong Li <zong.li@sifive.com> Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Deepak Gupta <debug@rivosinc.com> Tested-by: Andreas Korb <andreas.korb@aisec.fraunhofer.de> # QEMU, custom CVA6 Tested-by: Valentin Haudiquet <valentin.haudiquet@canonical.com> Link: https://patch.msgid.link/20251112-v5_user_cfi_series-v23-5-b55691eacf4f@rivosinc.com [pjw@kernel.org: cleaned up patch description] Signed-off-by: Paul Walmsley <pjw@kernel.org>
2025-09-16riscv: Replace __ASSEMBLY__ with __ASSEMBLER__ in non-uapi headersThomas Huth
While the GCC and Clang compilers already define __ASSEMBLER__ automatically when compiling assembly code, __ASSEMBLY__ is a macro that only gets defined by the Makefiles in the kernel. This can be very confusing when switching between userspace and kernelspace coding, or when dealing with uapi headers that rather should use __ASSEMBLER__ instead. So let's standardize on the __ASSEMBLER__ macro that is provided by the compilers now. This originally was a completely mechanical patch (done with a simple "sed -i" statement), with some manual fixups during rebasing of the patch later. Cc: Paul Walmsley <paul.walmsley@sifive.com> Cc: Palmer Dabbelt <palmer@dabbelt.com> Cc: Albert Ou <aou@eecs.berkeley.edu> Cc: Alexandre Ghiti <alex@ghiti.fr> Cc: linux-riscv@lists.infradead.org Signed-off-by: Thomas Huth <thuth@redhat.com> Link: https://lore.kernel.org/r/20250606070952.498274-3-thuth@redhat.com Signed-off-by: Paul Walmsley <pjw@kernel.org>
2025-06-05riscv: uaccess: Only restore the CSR_STATUS SUM bitCyril Bur
During switch to csrs will OR the value of the register into the corresponding csr. In this case we're only interested in restoring the SUM bit not the entire register. Signed-off-by: Cyril Bur <cyrilbur@tenstorrent.com> Link: https://lore.kernel.org/r/20250522160954.429333-1-cyrilbur@tenstorrent.com Co-developed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com> Fixes: 788aa64c01f1 ("riscv: save the SR_SUM status over switches") Link: https://lore.kernel.org/r/20250602121543.1544278-1-alexghiti@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2025-06-05Merge tag 'riscv-mw1-6.16-rc1' of ↵Palmer Dabbelt
ssh://gitolite.kernel.org/pub/scm/linux/kernel/git/alexghiti/linux into for-next riscv patches for 6.16-rc1 * Implement atomic patching support for ftrace which finally allows to get rid of stop_machine(). * Support for kexec_file_load() syscall * Improve module loading time by changing the algorithm that counts the number of plt/got entries in a module. * Zicbop is now used in the kernel to prefetch instructions [Palmer: There's been two rounds of surgery on this one, so as a result it's a bit different than the PR.] * alex-pr: (734 commits) riscv: Improve Kconfig help for RISCV_ISA_V_PREEMPTIVE MAINTAINERS: Update Atish's email address riscv: hwprobe: export Zabha extension riscv: Make regs_irqs_disabled() more clear perf symbols: Ignore mapping symbols on riscv RISC-V: Kconfig: Fix help text of CMDLINE_EXTEND riscv: module: Optimize PLT/GOT entry counting riscv: Add support for PUD THP riscv: xchg: Prefetch the destination word for sc.w riscv: Add ARCH_HAS_PREFETCH[W] support with Zicbop riscv: Add support for Zicbop riscv: Introduce Zicbop instructions riscv/kexec_file: Fix comment in purgatory relocator riscv: kexec_file: Support loading Image binary file riscv: kexec_file: Split the loading of kernel and others riscv: Documentation: add a description about dynamic ftrace riscv: ftrace: support direct call using call_ops riscv: Implement HAVE_DYNAMIC_FTRACE_WITH_CALL_OPS riscv: ftrace: support PREEMPT riscv: add a data fence for CMODX in the kernel mode ... Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2025-06-05Merge patch series "riscv: Add Zicbop & prefetchw support"Alexandre Ghiti
Alexandre Ghiti <alexghiti@rivosinc.com> says: I found this lost series developed by Guo so here is a respin with the comments on v2 applied. This patch series adds Zicbop support and then enables the Linux prefetch features. * patches from https://lore.kernel.org/r/20250421142441.395849-1-alexghiti@rivosinc.com: riscv: xchg: Prefetch the destination word for sc.w riscv: Add ARCH_HAS_PREFETCH[W] support with Zicbop riscv: Add support for Zicbop riscv: Introduce Zicbop instructions Link: https://lore.kernel.org/r/20250421142441.395849-1-alexghiti@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2025-06-05riscv: Add ARCH_HAS_PREFETCH[W] support with ZicbopGuo Ren
Enable Linux prefetch and prefetchw primitives using Zicbop. Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Link: https://lore.kernel.org/r/20231231082955.16516-3-guoren@kernel.org Tested-by: Andrea Parri <parri.andrea@gmail.com> Link: https://lore.kernel.org/r/20250421142441.395849-4-alexghiti@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2025-06-05riscv: Add support for ZicbopAlexandre Ghiti
Zicbop introduces cache blocks prefetching instructions, add the necessary support for the kernel to use it in the coming commits. Co-developed-by: Guo Ren <guoren@kernel.org> Signed-off-by: Guo Ren <guoren@kernel.org> Tested-by: Andrea Parri <parri.andrea@gmail.com> Link: https://lore.kernel.org/r/20250421142441.395849-3-alexghiti@rivosinc.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2025-06-05riscv: vector: Support calling schedule() for preemptible VectorAndy Chiu
Each function entry implies a call to ftrace infrastructure. And it may call into schedule in some cases. So, it is possible for preemptible kernel-mode Vector to implicitly call into schedule. Since all V-regs are caller-saved, it is possible to drop all V context when a thread voluntarily call schedule(). Besides, we currently don't pass argument through vector register, so we don't have to save/restore V-regs in ftrace trampoline. Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Link: https://lore.kernel.org/r/20250407180838.42877-7-andybnac@gmail.com Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com> Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2025-05-08riscv: save the SR_SUM status over switchesBen Dooks
When threads/tasks are switched we need to ensure the old execution's SR_SUM state is saved and the new thread has the old SR_SUM state restored. The issue was seen under heavy load especially with the syz-stress tool running, with crashes as follows in schedule_tail: Unable to handle kernel access to user memory without uaccess routines at virtual address 000000002749f0d0 Oops [#1] Modules linked in: CPU: 1 PID: 4875 Comm: syz-executor.0 Not tainted 5.12.0-rc2-syzkaller-00467-g0d7588ab9ef9 #0 Hardware name: riscv-virtio,qemu (DT) epc : schedule_tail+0x72/0xb2 kernel/sched/core.c:4264 ra : task_pid_vnr include/linux/sched.h:1421 [inline] ra : schedule_tail+0x70/0xb2 kernel/sched/core.c:4264 epc : ffffffe00008c8b0 ra : ffffffe00008c8ae sp : ffffffe025d17ec0 gp : ffffffe005d25378 tp : ffffffe00f0d0000 t0 : 0000000000000000 t1 : 0000000000000001 t2 : 00000000000f4240 s0 : ffffffe025d17ee0 s1 : 000000002749f0d0 a0 : 000000000000002a a1 : 0000000000000003 a2 : 1ffffffc0cfac500 a3 : ffffffe0000c80cc a4 : 5ae9db91c19bbe00 a5 : 0000000000000000 a6 : 0000000000f00000 a7 : ffffffe000082eba s2 : 0000000000040000 s3 : ffffffe00eef96c0 s4 : ffffffe022c77fe0 s5 : 0000000000004000 s6 : ffffffe067d74e00 s7 : ffffffe067d74850 s8 : ffffffe067d73e18 s9 : ffffffe067d74e00 s10: ffffffe00eef96e8 s11: 000000ae6cdf8368 t3 : 5ae9db91c19bbe00 t4 : ffffffc4043cafb2 t5 : ffffffc4043cafba t6 : 0000000000040000 status: 0000000000000120 badaddr: 000000002749f0d0 cause: 000000000000000f Call Trace: [<ffffffe00008c8b0>] schedule_tail+0x72/0xb2 kernel/sched/core.c:4264 [<ffffffe000005570>] ret_from_exception+0x0/0x14 Dumping ftrace buffer: (ftrace buffer empty) ---[ end trace b5f8f9231dc87dda ]--- The issue comes from the put_user() in schedule_tail (kernel/sched/core.c) doing the following: asmlinkage __visible void schedule_tail(struct task_struct *prev) { ... if (current->set_child_tid) put_user(task_pid_vnr(current), current->set_child_tid); ... } the put_user() macro causes the code sequence to come out as follows: 1: __enable_user_access() 2: reg = task_pid_vnr(current); 3: *current->set_child_tid = reg; 4: __disable_user_access() The problem is that we may have a sleeping function as argument which could clear SR_SUM causing the panic above. This was fixed by evaluating the argument of the put_user() macro outside the user-enabled section in commit 285a76bb2cf5 ("riscv: evaluate put_user() arg before enabling user access")" In order for riscv to take advantage of unsafe_get/put_XXX() macros and to avoid the same issue we had with put_user() and sleeping functions we must ensure code flow can go through switch_to() from within a region of code with SR_SUM enabled and come back with SR_SUM still enabled. This patch addresses the problem allowing future work to enable full use of unsafe_get/put_XXX() macros without needing to take a CSR bit flip cost on every access. Make switch_to() save and restore SR_SUM. Reported-by: syzbot+e74b94fe601ab9552d69@syzkaller.appspotmail.com Signed-off-by: Ben Dooks <ben.dooks@codethink.co.uk> Signed-off-by: Cyril Bur <cyrilbur@tenstorrent.com> Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Reviewed-by: Deepak Gupta <debug@rivosinc.com> Link: https://lore.kernel.org/r/20250410070526.3160847-2-cyrilbur@tenstorrent.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2024-10-24riscv: Add support for userspace pointer maskingSamuel Holland
RISC-V supports pointer masking with a variable number of tag bits (which is called "PMLEN" in the specification) and which is configured at the next higher privilege level. Wire up the PR_SET_TAGGED_ADDR_CTRL and PR_GET_TAGGED_ADDR_CTRL prctls so userspace can request a lower bound on the number of tag bits and determine the actual number of tag bits. As with arm64's PR_TAGGED_ADDR_ENABLE, the pointer masking configuration is thread-scoped, inherited on clone() and fork() and cleared on execve(). Reviewed-by: Charlie Jenkins <charlie@rivosinc.com> Tested-by: Charlie Jenkins <charlie@rivosinc.com> Signed-off-by: Samuel Holland <samuel.holland@sifive.com> Link: https://lore.kernel.org/r/20241016202814.4061541-5-samuel.holland@sifive.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2024-10-05riscv: Add support for per-thread envcfg CSR valuesSamuel Holland
Some bits in the [ms]envcfg CSR, such as the CFI state and pointer masking mode, need to be controlled on a per-thread basis. Support this by keeping a copy of the CSR value in struct thread_struct and writing it during context switches. It is safe to discard the old CSR value during the context switch because the CSR is modified only by software, so the CSR will remain in sync with the copy in thread_struct. Use ALTERNATIVE directly instead of riscv_has_extension_unlikely() to minimize branchiness in the context switching code. Since thread_struct is copied during fork(), setting the value for the init task sets the default value for all other threads. Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Reviewed-by: Deepak Gupta <debug@rivosinc.com> Signed-off-by: Samuel Holland <samuel.holland@sifive.com> Reviewed-by: Charlie Jenkins <charlie@rivosinc.com> Link: https://lore.kernel.org/r/20240814081126.956287-3-samuel.holland@sifive.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2024-08-29riscv: mm: Do not restrict mmap address based on hintCharlie Jenkins
The hint address should not forcefully restrict the addresses returned by mmap as this causes mmap to report ENOMEM when there is memory still available. Signed-off-by: Charlie Jenkins <charlie@rivosinc.com> Fixes: b5b4287accd7 ("riscv: mm: Use hint address in mmap if available") Fixes: add2cc6b6515 ("RISC-V: mm: Restrict address space for sv39,sv48,sv57") Closes: https://lore.kernel.org/linux-kernel/ZbxTNjQPFKBatMq+@ghost/T/#mccb1890466bf5a488c9ce7441e57e42271895765 Link: https://lore.kernel.org/r/20240826-riscv_mmap-v1-3-cd8962afe47f@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2024-07-11riscv: hwprobe: export highest virtual userspace addressClément Léger
Some userspace applications (OpenJDK for instance) uses the free MSBs in pointers to insert additional information for their own logic and need to get this information from somewhere. Currently they rely on parsing /proc/cpuinfo "mmu=svxx" string to obtain the current value of virtual address usable bits [1]. Since this reflect the raw supported MMU mode, it might differ from the logical one used internally which is why arch_get_mmap_end() is used. Exporting the highest mmapable address through hwprobe will allow a more stable interface to be used. For that purpose, add a new hwprobe key named RISCV_HWPROBE_KEY_HIGHEST_VIRT_ADDRESS which will export the highest userspace virtual address. Link: https://github.com/openjdk/jdk/blob/master/src/hotspot/os_cpu/linux_riscv/vm_version_linux_riscv.cpp#L171 [1] Signed-off-by: Clément Léger <cleger@rivosinc.com> Reviewed-by: Charlie Jenkins <charlie@rivosinc.com> Link: https://lore.kernel.org/r/20240410144558.1104006-1-cleger@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2024-04-30Merge patch series "riscv: Create and document PR_RISCV_SET_ICACHE_FLUSH_CTX ↵Palmer Dabbelt
prctl" Charlie Jenkins <charlie@rivosinc.com> says: Improve the performance of icache flushing by creating a new prctl flag PR_RISCV_SET_ICACHE_FLUSH_CTX. The interface is left generic to allow for future expansions such as with the proposed J extension [1]. Documentation is also provided to explain the use case. Patch sent to add PR_RISCV_SET_ICACHE_FLUSH_CTX to man-pages [2]. [1] https://github.com/riscv/riscv-j-extension [2] https://lore.kernel.org/linux-man/20240124-fencei_prctl-v1-1-0bddafcef331@rivosinc.com * b4-shazam-merge: cpumask: Add assign cpu documentation: Document PR_RISCV_SET_ICACHE_FLUSH_CTX prctl riscv: Include riscv_set_icache_flush_ctx prctl riscv: Remove unnecessary irqflags processor.h include Link: https://lore.kernel.org/r/20240312-fencei-v13-0-4b6bdc2bbf32@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2024-04-18riscv: Include riscv_set_icache_flush_ctx prctlCharlie Jenkins
Support new prctl with key PR_RISCV_SET_ICACHE_FLUSH_CTX to enable optimization of cross modifying code. This prctl enables userspace code to use icache flushing instructions such as fence.i with the guarantee that the icache will continue to be clean after thread migration. Signed-off-by: Charlie Jenkins <charlie@rivosinc.com> Reviewed-by: Atish Patra <atishp@rivosinc.com> Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com> Reviewed-by: Samuel Holland <samuel.holland@sifive.com> Link: https://lore.kernel.org/r/20240312-fencei-v13-2-4b6bdc2bbf32@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2024-03-20Merge patch series "riscv: Introduce compat-mode helpers & improve ↵Palmer Dabbelt
arch_get_mmap_end()" Leonardo Bras <leobras@redhat.com> says: I just saw the opportunity of optimizing the helper is_compat_task() by introducing a compile-time test, and it made possible to remove some #ifdef's without any loss of performance. I also saw the possibility of removing the direct check of task flags from general code, and concentrated it in asm/compat.h by creating a few more helpers, which in the end helped optimize code. arch_get_mmap_end() just got a simple improvement and some extra docs. * b4-shazam-merge: riscv: Introduce set_compat_task() in asm/compat.h riscv: Introduce is_compat_thread() into compat.h riscv: add compile-time test into is_compat_task() riscv: Replace direct thread flag check with is_compat_task() riscv: Improve arch_get_mmap_end() macro Link: https://lore.kernel.org/r/20240103160024.70305-2-leobras@redhat.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2024-03-19riscv: add compile-time test into is_compat_task()Leonardo Bras
Currently several places will test for CONFIG_COMPAT before testing is_compat_task(), probably in order to avoid a run-time test into the task structure. Since is_compat_task() is an inlined function, it would be helpful to add a compile-time test of CONFIG_COMPAT, making sure it always returns zero when the option is not enabled during the kernel build. With this, the compiler is able to understand in build-time that is_compat_task() will always return 0, and optimize-out some of the extra code introduced by the option. This will also allow removing a lot #ifdefs that were introduced, and make the code more clean. Signed-off-by: Leonardo Bras <leobras@redhat.com> Reviewed-by: Guo Ren <guoren@kernel.org> Reviewed-by: Andy Chiu <andy.chiu@sifive.com> Link: https://lore.kernel.org/r/20240103160024.70305-5-leobras@redhat.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2024-03-19riscv: Improve arch_get_mmap_end() macroLeonardo Bras
This macro caused me some confusion, which took some reviewer's time to make it clear, so I propose adding a short comment in code to avoid confusion in the future. Also, added some improvements to the macro, such as removing the assumption of VA_USER_SV57 being the largest address space. Signed-off-by: Leonardo Bras <leobras@redhat.com> Reviewed-by: Guo Ren <guoren@kernel.org> Link: https://lore.kernel.org/r/20240103160024.70305-3-leobras@redhat.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2024-03-15Merge patch series "riscv: mm: Extend mappable memory up to hint address"Palmer Dabbelt
Charlie Jenkins <charlie@rivosinc.com> says: On riscv, mmap currently returns an address from the largest address space that can fit entirely inside of the hint address. This makes it such that the hint address is almost never returned. This patch raises the mappable area up to and including the hint address. This allows mmap to often return the hint address, which allows a performance improvement over searching for a valid address as well as making the behavior more similar to other architectures. Note that a previous patch introduced stronger semantics compared to other architectures for riscv mmap. On riscv, mmap will not use bits in the upper bits of the virtual address depending on the hint address. On other architectures, a random address is returned in the address space requested. On all architectures the hint address will be returned if it is available. This allows riscv applications to configure how many bits in the virtual address should be left empty. This has the two benefits of being able to request address spaces that are smaller than the default and doesn't require the application to know the page table layout of riscv. * b4-shazam-merge: docs: riscv: Define behavior of mmap selftests: riscv: Generalize mm selftests riscv: mm: Use hint address in mmap if available Link: https://lore.kernel.org/r/20240130-use_mmap_hint_address-v3-0-8a655cfa8bcb@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2024-03-14riscv: mm: Use hint address in mmap if availableCharlie Jenkins
On riscv it is guaranteed that the address returned by mmap is less than the hint address. Allow mmap to return an address all the way up to addr, if provided, rather than just up to the lower address space. This provides a performance benefit as well, allowing mmap to exit after checking that the address is in range rather than searching for a valid address. It is possible to provide an address that uses at most the same number of bits, however it is significantly more computationally expensive to provide that number rather than setting the max to be the hint address. There is the instruction clz/clzw in Zbb that returns the highest set bit which could be used to performantly implement this, but it would still be slower than the current implementation. At worst case, half of the address would not be able to be allocated when a hint address is provided. Signed-off-by: Charlie Jenkins <charlie@rivosinc.com> Link: https://lore.kernel.org/r/20240130-use_mmap_hint_address-v3-1-8a655cfa8bcb@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2024-01-16Merge patch series "riscv: support kernel-mode Vector"Palmer Dabbelt
Andy Chiu <andy.chiu@sifive.com> says: This series provides support running Vector in kernel mode. Additionally, kernel-mode Vector can be configured to run without turnning off preemption on a CONFIG_PREEMPT kernel. Along with the suport, we add Vector optimized copy_{to,from}_user. And provide a simple threshold to decide when to run the vectorized functions. We decided to drop vectorized memcpy/memset/memmove for the moment due to the concern of memory side-effect in kernel_vector_begin(). The detailed description can be found at v9[0] This series is composed by 4 parts: patch 1-4: adds basic support for kernel-mode Vector patch 5: includes vectorized copy_{to,from}_user into the kernel patch 6: refactor context switch code in fpu [1] patch 7-10: provides some code refactors and support for preemptible kernel-mode Vector. This series can be merged if we feel any part of {1~4, 5, 6, 7~10} is mature enough. This patch is tested on a QEMU with V and verified that booting, normal userspace operations all work as usual with thresholds set to 0. Also, we test by launching multiple kernel threads which continuously executes and verifies Vector operations in the background. The module that tests these operation is expected to be upstream later. * b4-shazam-merge: riscv: vector: allow kernel-mode Vector with preemption riscv: vector: use kmem_cache to manage vector context riscv: vector: use a mask to write vstate_ctrl riscv: vector: do not pass task_struct into riscv_v_vstate_{save,restore}() riscv: fpu: drop SR_SD bit checking riscv: lib: vectorize copy_to_user/copy_from_user riscv: sched: defer restoring Vector context for user riscv: Add vector extension XOR implementation riscv: vector: make Vector always available for softirq context riscv: Add support for kernel mode vector Link: https://lore.kernel.org/r/20240115055929.4736-1-andy.chiu@sifive.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2024-01-16riscv: vector: allow kernel-mode Vector with preemptionAndy Chiu
Add kernel_vstate to keep track of kernel-mode Vector registers when trap introduced context switch happens. Also, provide riscv_v_flags to let context save/restore routine track context status. Context tracking happens whenever the core starts its in-kernel Vector executions. An active (dirty) kernel task's V contexts will be saved to memory whenever a trap-introduced context switch happens. Or, when a softirq, which happens to nest on top of it, uses Vector. Context retoring happens when the execution transfer back to the original Kernel context where it first enable preempt_v. Also, provide a config CONFIG_RISCV_ISA_V_PREEMPTIVE to give users an option to disable preemptible kernel-mode Vector at build time. Users with constraint memory may want to disable this config as preemptible kernel-mode Vector needs extra space for tracking of per thread's kernel-mode V context. Or, users might as well want to disable it if all kernel-mode Vector code is time sensitive and cannot tolerate context switch overhead. Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Tested-by: Björn Töpel <bjorn@rivosinc.com> Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com> Link: https://lore.kernel.org/r/20240115055929.4736-11-andy.chiu@sifive.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2024-01-16riscv: vector: make Vector always available for softirq contextAndy Chiu
The goal of this patch is to provide full support of Vector in kernel softirq context. So that some of the crypto alogrithms won't need scalar fallbacks. By disabling bottom halves in active kernel-mode Vector, softirq will not be able to nest on top of any kernel-mode Vector. So, softirq context is able to use Vector whenever it runs. After this patch, Vector context cannot start with irqs disabled. Otherwise local_bh_enable() may run in a wrong context. Disabling bh is not enough for RT-kernel to prevent preeemption. So we must disable preemption, which also implies disabling bh on RT. Related-to: commit 696207d4258b ("arm64/sve: Make kernel FPU protection RT friendly") Related-to: commit 66c3ec5a7120 ("arm64: neon: Forbid when irqs are disabled") Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Reviewed-by: Eric Biggers <ebiggers@google.com> Tested-by: Björn Töpel <bjorn@rivosinc.com> Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com> Link: https://lore.kernel.org/r/20240115055929.4736-3-andy.chiu@sifive.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2024-01-16riscv: Add support for kernel mode vectorGreentime Hu
Add kernel_vector_begin() and kernel_vector_end() function declarations and corresponding definitions in kernel_mode_vector.c These are needed to wrap uses of vector in kernel mode. Co-developed-by: Vincent Chen <vincent.chen@sifive.com> Signed-off-by: Vincent Chen <vincent.chen@sifive.com> Signed-off-by: Greentime Hu <greentime.hu@sifive.com> Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Reviewed-by: Eric Biggers <ebiggers@google.com> Tested-by: Björn Töpel <bjorn@rivosinc.com> Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com> Link: https://lore.kernel.org/r/20240115055929.4736-2-andy.chiu@sifive.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2024-01-11riscv: mm: Fixup compat arch_get_mmap_endGuo Ren
When the task is in COMPAT mode, the arch_get_mmap_end should be 2GB, not TASK_SIZE_64. The TASK_SIZE has contained is_compat_mode() detection, so change the definition of STACK_TOP_MAX to TASK_SIZE directly. Cc: stable@vger.kernel.org Fixes: add2cc6b6515 ("RISC-V: mm: Restrict address space for sv39,sv48,sv57") Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Reviewed-by: Leonardo Bras <leobras@redhat.com> Reviewed-by: Charlie Jenkins <charlie@rivosinc.com> Link: https://lore.kernel.org/r/20231222115703.2404036-3-guoren@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-11-10Merge tag 'riscv-for-linus-6.7-mw2' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux Pull more RISC-V updates from Palmer Dabbelt: - Support for handling misaligned accesses in S-mode - Probing for misaligned access support is now properly cached and handled in parallel - PTDUMP now reflects the SW reserved bits, as well as the PBMT and NAPOT extensions - Performance improvements for TLB flushing - Support for many new relocations in the module loader - Various bug fixes and cleanups * tag 'riscv-for-linus-6.7-mw2' of git://git.kernel.org/pub/scm/linux/kernel/git/riscv/linux: (51 commits) riscv: Optimize bitops with Zbb extension riscv: Rearrange hwcap.h and cpufeature.h drivers: perf: Do not broadcast to other cpus when starting a counter drivers: perf: Check find_first_bit() return value of: property: Add fw_devlink support for msi-parent RISC-V: Don't fail in riscv_of_parent_hartid() for disabled HARTs riscv: Fix set_memory_XX() and set_direct_map_XX() by splitting huge linear mappings riscv: Don't use PGD entries for the linear mapping RISC-V: Probe misaligned access speed in parallel RISC-V: Remove __init on unaligned_emulation_finish() RISC-V: Show accurate per-hart isa in /proc/cpuinfo RISC-V: Don't rely on positional structure initialization riscv: Add tests for riscv module loading riscv: Add remaining module relocations riscv: Avoid unaligned access when relocating modules riscv: split cache ops out of dma-noncoherent.c riscv: Improve flush_tlb_kernel_range() riscv: Make __flush_tlb_range() loop over pte instead of flushing the whole tlb riscv: Improve flush_tlb_range() for hugetlb pages riscv: Improve tlb_flush() ...
2023-11-01riscv: add support for PR_SET_UNALIGN and PR_GET_UNALIGNClément Léger
Now that trap support is ready to handle misalignment errors in S-mode, allow the user to control the behavior of misaligned accesses using prctl(PR_SET_UNALIGN). Add an align_ctl flag in thread_struct which will be used to determine if we should SIGBUS the process or not on such fault. Signed-off-by: Clément Léger <cleger@rivosinc.com> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Link: https://lore.kernel.org/r/20231004151405.521596-9-cleger@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-10-04riscv: kdump: use generic interface to simplify crashkernel reservationBaoquan He
With the help of newly changed function parse_crashkernel() and generic reserve_crashkernel_generic(), crashkernel reservation can be simplified by steps: 1) Add a new header file <asm/crash_core.h>, and define CRASH_ALIGN, CRASH_ADDR_LOW_MAX, CRASH_ADDR_HIGH_MAX and DEFAULT_CRASH_KERNEL_LOW_SIZE in <asm/crash_core.h>; 2) Add arch_reserve_crashkernel() to call parse_crashkernel() and reserve_crashkernel_generic(); 3) Add ARCH_HAS_GENERIC_CRASHKERNEL_RESERVATION Kconfig in arch/riscv/Kconfig. The old reserve_crashkernel_low() and reserve_crashkernel() can be removed. [chenjiahao16@huawei.com: fix crashkernel reserving problem on RISC-V] Link: https://lkml.kernel.org/r/20230925024333.730964-1-chenjiahao16@huawei.com Link: https://lkml.kernel.org/r/20230914033142.676708-9-bhe@redhat.com Signed-off-by: Baoquan He <bhe@redhat.com> Signed-off-by: Chen Jiahao <chenjiahao16@huawei.com> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Chen Jiahao <chenjiahao16@huawei.com> Cc: Zhen Lei <thunder.leizhen@huawei.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2023-08-23RISC-V: mm: Restrict address space for sv39,sv48,sv57Charlie Jenkins
Make sv48 the default address space for mmap as some applications currently depend on this assumption. A hint address passed to mmap will cause the largest address space that fits entirely into the hint to be used. If the hint is less than or equal to 1<<38, an sv39 address will be used. An exception is that if the hint address is 0, then a sv48 address will be used. After an address space is completely full, the next smallest address space will be used. Signed-off-by: Charlie Jenkins <charlie@rivosinc.com> Link: https://lore.kernel.org/r/20230809232218.849726-2-charlie@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-06-23Merge patch series "ISA string parser cleanups"Palmer Dabbelt
Conor Dooley <conor@kernel.org> says: From: Conor Dooley <conor.dooley@microchip.com> Here are some bits that were discussed with Drew on the "should we allow caps" threads that I have now created patches for: - splitting of riscv_of_processor_hartid() into two distinct functions, one for use purely during early boot, prior to the establishment of the possible-cpus mask & another to fit the other current use-cases - that then allows us to then completely skip some validation of the hartid in the parser - the biggest diff in the series is a rework of the comments in the parser, as I have mostly found the existing (sparse) ones to not be all that helpful whenever I have to go back and look at it - from writing the comments, I found a conditional doing a bit of a dance that I found counter-intuitive, so I've had a go at making that match what I would expect a little better - `i` implies 4 other extensions, so add them as extensions and set them for the craic. Sure why not like... * b4-shazam-merge: RISC-V: always report presence of extensions formerly part of the base ISA dt-bindings: riscv: explicitly mention assumption of Zicntr & Zihpm support RISC-V: remove decrement/increment dance in ISA string parser RISC-V: rework comments in ISA string parser RISC-V: validate riscv,isa at boot, not during ISA string parsing RISC-V: split early & late of_node to hartid mapping RISC-V: simplify register width check in ISA string parsing Link: https://lore.kernel.org/r/20230607-audacity-overhaul-82bb867a825f@spud Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-06-21RISC-V: split early & late of_node to hartid mappingConor Dooley
Some back and forth with Drew [1] about riscv_fill_hwcap() resulted in the realisation that it is not very useful to parse the DT & perform validation of riscv,isa every time we would like to get the id for a hart. Although it is no longer called in riscv_fill_hwcap(), riscv_of_processor_hartid() is called in several other places. Notably in setup_smp() it forms part of the logic for filling the mask of possible CPUs. Since a possible CPU must have passed this basic validation of riscv,isa, a repeat validation is not required. Rename riscv_of_processor_id() to riscv_early_of_processor_id(), which will be called from setup_smp() & introduce a new riscv_of_processor_id() which makes use of the pre-populated mask of possible cpus. Link: https://lore.kernel.org/linux-riscv/xvdswl3iyikwvamny7ikrxo2ncuixshtg3f6uucjahpe3xpc5c@ud4cz4fkg5dj/ [1] Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Signed-off-by: Conor Dooley <conor.dooley@microchip.com> Reviewed-by: Sunil V L <sunilvl@ventanamicro.com> Link: https://lore.kernel.org/r/20230607-glade-pastel-d8cbd9d9f3c6@spud Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-06-08riscv: Add prctl controls for userspace vector managementAndy Chiu
This patch add two riscv-specific prctls, to allow usespace control the use of vector unit: * PR_RISCV_V_SET_CONTROL: control the permission to use Vector at next, or all following execve for a thread. Turning off a thread's Vector live is not possible since libraries may have registered ifunc that may execute Vector instructions. * PR_RISCV_V_GET_CONTROL: get the same permission setting for the current thread, and the setting for following execve(s). Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Reviewed-by: Greentime Hu <greentime.hu@sifive.com> Reviewed-by: Vincent Chen <vincent.chen@sifive.com> Link: https://lore.kernel.org/r/20230605110724.21391-22-andy.chiu@sifive.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-06-08riscv: signal: Report signal frame size to userspace via auxvVincent Chen
The vector register belongs to the signal context. They need to be stored and restored as entering and leaving the signal handler. According to the V-extension specification, the maximum length of the vector registers can be 2^16. Hence, if userspace refers to the MINSIGSTKSZ to create a sigframe, it may not be enough. To resolve this problem, this patch refers to the commit 94b07c1f8c39c ("arm64: signal: Report signal frame size to userspace via auxv") to enable userspace to know the minimum required sigframe size through the auxiliary vector and use it to allocate enough memory for signal context. Note that auxv always reports size of the sigframe as if V exists for all starting processes, whenever the kernel has CONFIG_RISCV_ISA_V. The reason is that users usually reference this value to allocate an alternative signal stack, and the user may use V anytime. So the user must reserve a space for V-context in sigframe in case that the signal handler invokes after the kernel allocating V. Signed-off-by: Greentime Hu <greentime.hu@sifive.com> Signed-off-by: Vincent Chen <vincent.chen@sifive.com> Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Acked-by: Conor Dooley <conor.dooley@microchip.com> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Reviewed-by: Guo Ren <guoren@kernel.org> Reviewed-by: Heiko Stuebner <heiko.stuebner@vrull.eu> Tested-by: Heiko Stuebner <heiko.stuebner@vrull.eu> Link: https://lore.kernel.org/r/20230605110724.21391-16-andy.chiu@sifive.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-06-08riscv: Add task switch support for vectorGreentime Hu
This patch adds task switch support for vector. It also supports all lengths of vlen. Suggested-by: Andrew Waterman <andrew@sifive.com> Co-developed-by: Nick Knight <nick.knight@sifive.com> Signed-off-by: Nick Knight <nick.knight@sifive.com> Co-developed-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Co-developed-by: Vincent Chen <vincent.chen@sifive.com> Signed-off-by: Vincent Chen <vincent.chen@sifive.com> Co-developed-by: Ruinland Tsai <ruinland.tsai@sifive.com> Signed-off-by: Ruinland Tsai <ruinland.tsai@sifive.com> Signed-off-by: Greentime Hu <greentime.hu@sifive.com> Signed-off-by: Vineet Gupta <vineetg@rivosinc.com> Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Reviewed-by: Heiko Stuebner <heiko.stuebner@vrull.eu> Tested-by: Heiko Stuebner <heiko.stuebner@vrull.eu> Link: https://lore.kernel.org/r/20230605110724.21391-11-andy.chiu@sifive.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-09-11kernel: exit: cleanup release_thread()Kefeng Wang
Only x86 has own release_thread(), introduce a new weak release_thread() function to clean empty definitions in other ARCHs. Link: https://lkml.kernel.org/r/20220819014406.32266-1-wangkefeng.wang@huawei.com Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com> Acked-by: Guo Ren <guoren@kernel.org> [csky] Acked-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk> Acked-by: Geert Uytterhoeven <geert@linux-m68k.org> Acked-by: Brian Cain <bcain@quicinc.com> Acked-by: Michael Ellerman <mpe@ellerman.id.au> [powerpc] Acked-by: Stafford Horne <shorne@gmail.com> [openrisc] Acked-by: Catalin Marinas <catalin.marinas@arm.com> [arm64] Acked-by: Huacai Chen <chenhuacai@kernel.org> [LoongArch] Cc: Alexander Gordeev <agordeev@linux.ibm.com> Cc: Anton Ivanov <anton.ivanov@cambridgegreys.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Christian Borntraeger <borntraeger@linux.ibm.com> Cc: Christophe Leroy <christophe.leroy@csgroup.eu> Cc: Chris Zankel <chris@zankel.net> Cc: Dave Hansen <dave.hansen@linux.intel.com> Cc: "David S. Miller" <davem@davemloft.net> Cc: Dinh Nguyen <dinguyen@kernel.org> Cc: Guo Ren <guoren@kernel.org> [csky] Cc: Heiko Carstens <hca@linux.ibm.com> Cc: Helge Deller <deller@gmx.de> Cc: Ingo Molnar <mingo@redhat.com> Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru> Cc: James Bottomley <James.Bottomley@HansenPartnership.com> Cc: Johannes Berg <johannes@sipsolutions.net> Cc: Jonas Bonn <jonas@southpole.se> Cc: Matt Turner <mattst88@gmail.com> Cc: Max Filippov <jcmvbkbc@gmail.com> Cc: Michal Simek <monstr@monstr.eu> Cc: Nicholas Piggin <npiggin@gmail.com> Cc: Palmer Dabbelt <palmer@dabbelt.com> Cc: Paul Walmsley <paul.walmsley@sifive.com> Cc: Richard Henderson <richard.henderson@linaro.org> Cc: Richard Weinberger <richard@nod.at> Cc: Rich Felker <dalias@libc.org> Cc: Stefan Kristiansson <stefan.kristiansson@saunalahti.fi> Cc: Sven Schnelle <svens@linux.ibm.com> Cc: Thomas Bogendoerfer <tsbogend@alpha.franken.de> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Vasily Gorbik <gor@linux.ibm.com> Cc: Vineet Gupta <vgupta@kernel.org> Cc: Will Deacon <will@kernel.org> Cc: Xuerui Wang <kernel@xen0n.name> Cc: Yoshinori Sato <ysato@users.osdn.me> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2022-07-19riscv: cpu: Add 64bit hartid support on RV64Sunil V L
The hartid can be a 64bit value on RV64 platforms. Add support for 64bit hartid in riscv_of_processor_hartid() and update its callers. Signed-off-by: Sunil V L <sunilvl@ventanamicro.com> Reviewed-by: Atish Patra <atishp@rivosinc.com> Link: https://lore.kernel.org/r/20220527051743.2829940-5-sunilvl@ventanamicro.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-04-26riscv: compat: Support TASK_SIZE for compat modeGuo Ren
Make TASK_SIZE from const to dynamic detect TIF_32BIT flag function. Refer to arm64 to implement DEFAULT_MAP_WINDOW_64 for efi-stub. Limit 32-bit compatible process in 0-2GB virtual address range (which is enough for real scenarios), because it could avoid address sign extend problem when 32-bit enter 64-bit and ease software design. The standard 32-bit TASK_SIZE is 0x9dc00000:FIXADDR_START, and compared to a compatible 32-bit, it increases 476MB for the application's virtual address. Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Reviewed-by: Arnd Bergmann <arnd@arndb.de> Tested-by: Heiko Stuebner <heiko@sntech.de> Link: https://lore.kernel.org/r/20220405071314.3225832-11-guoren@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2021-10-15sched: Add wrapper for get_wchan() to keep task blockedKees Cook
Having a stable wchan means the process must be blocked and for it to stay that way while performing stack unwinding. Suggested-by: Peter Zijlstra <peterz@infradead.org> Signed-off-by: Kees Cook <keescook@chromium.org> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Acked-by: Geert Uytterhoeven <geert@linux-m68k.org> Acked-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk> [arm] Tested-by: Mark Rutland <mark.rutland@arm.com> [arm64] Link: https://lkml.kernel.org/r/20211008111626.332092234@infradead.org
2021-08-03riscv: Implement thread_struct whitelist for hardened usercopyTong Tiangen
This whitelists the FPU register state portion of the thread_struct for copying to userspace, instead of the default entire struct. Signed-off-by: Tong Tiangen <tongtiangen@huawei.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-03-09riscv: process: Fix no prototype for arch_dup_task_structNanyong Sun
Fix the following W=1 compilation warning: arch/riscv/kernel/process.c:114:5: warning: no previous prototype for ‘arch_dup_task_struct’ [-Wmissing-prototypes] 114 | int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src) | ^~~~~~~~~~~~~~~~~~~~ Reported-by: Hulk Robot <hulkci@huawei.com> Signed-off-by: Nanyong Sun <sunnanyong@huawei.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-01-14riscv: Add uprobes supportedGuo Ren
This patch adds support for uprobes on riscv architecture. Just like kprobe, it support single-step and simulate instructions. Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Reviewed-by: Pekka Enberg <penberg@kernel.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-06-10riscv: use vDSO common flow to reduce the latency of the time-related functionsVincent Chen
Even if RISC-V has supported the vDSO feature, the latency of the functions for obtaining the system time is still expensive. It is because these functions still trigger a corresponding system call in the process, which slows down the response time. If we want to remove the system call to reduce the latency, the kernel should have the ability to output the system clock information to userspace. This patch introduces the vDSO common flow to enable the kernel to achieve the above feature and uses "rdtime" instruction to obtain the current time in the user space. Under this condition, the latency cost by the ecall from U-mode to S-mode can be eliminated. After applying this patch, the latency of gettimeofday() measured on the HiFive unleashed board can be reduced by %61. Signed-off-by: Vincent Chen <vincent.chen@sifive.com> Reviewed-by: Atish Patra <atish.patra@wdc.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-06-09RISC-V: Rename and move plic_find_hart_id() to arch directoryAnup Patel
The plic_find_hart_id() can be useful to other interrupt controller drivers (such as RISC-V local interrupt driver) so we rename this function to riscv_of_parent_hartid() and place it in arch directory along with riscv_of_processor_hartid(). Signed-off-by: Anup Patel <anup.patel@wdc.com> Reviewed-by: Atish Patra <atish.patra@wdc.com> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Acked-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com> Reviewed-by: Marc Zyngier <maz@kernel.org> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2019-11-05riscv: abstract out CSR names for supervisor vs machine modeChristoph Hellwig
Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>