| Age | Commit message (Collapse) | Author |
|---|
|
Pull more KVM updates from Paolo Bonzini:
"ARM64:
- Fix a couple of embarrassing merge mistakes in kvmarm-7.4
- Cleanup the HCR_EL2 bit fields to exclusively use the generated
definitions, fixings a nasty regression introduced in 6.16, where
barriers were not upgraded from NSH to ISH anymore
- Fix publication of state when a vcpu is forced out of the guest
- Let vcpu pausing nest correctly
- Sanitise ID_AA64MMFR3_EL1.S1P[IO]E to follow the rules of the
architecture
- Don't warn on SMCCC filter reserved range allocation failure
- Prevent GICv5 and TLBI hypercalls to be used from the host when
pKVM is active
- Enforce that only GICv3 guests are supported for pKVM
- Don't try to access GICv5 EL2 sysregs from EL1
- Compute architectrally correct values for ID_AA64MMFR3_EL1.S1P[IO]E
by checking the feature dependencies
- Require matching VGIC and ITS model
- Cleanup TRACE_REMOTE dependencies
- Fix random comments
ARM64 selftests:
- selftests: Skip unwritable ID fields in the failed-run test
- Move pkvm_selftests() after all CPUs have initialized EL2
RISC-V:
- Allow Zicclsm, Ziccamoa, Ziccif, Za64rs for KVM guest
- Implement KVM_PRE_FAULT_MEMORY ioctl()
- Replace imsic switch-case macros with range checks
RISC-V fixes:
- Avoid redundant interrupt check for pending timers
- Reject vCPU creation after AIA initialization
- Fix build with vector support disabled
- Fix IPI delivery for out-of-order vcpu_id
- Use a raw lock for IMSIC wakeup state
- Reject vCPU creation after AIA initialization
RISC-V selftests:
- Lazy V-extension enablement for KVM selftests
- Show sepc/scause/stval CSRs in KVM selftests vcpu dump"
* tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (42 commits)
KVM: selftests: Reject GICv3 ITS with a GICv5 VGIC
KVM: arm64: vgic: Require matching VGIC and ITS models
KVM: arm64: gic-v5: Drop unused addr variable
KVM: arm64: Sanitise ID_AA64MMFR3_EL1.S1P[IO]E
KVM: arm64: selftests: Skip unwritable ID fields in the failed-run test
tracing/remotes: Fix TRACE_REMOTE build when TRACING=n
KVM: arm64: Move pkvm_selftests() after all CPUs have initialized EL2
KVM: arm64: Get rid of compatibility HCR_EL2 aliases
KVM: arm64: Convert HCR_xyz to HCR_EL2_xyz
KVM: arm64: Set HCR_EL2.BSU to Inner Shareable for guests
KVM: arm64: Fix stale VGICv3 comments in the nVHE world switch
KVM: arm64: vgic: Do not access the GICv5 CPU interface from EL1
KVM: arm64: Validate the host-provided vgic model in pKVM
KVM: arm64: Reject the stage-2 flush hypercalls under pKVM
KVM: arm64: Return on kvm_gmem_get_pfn() failure in gmem_abort()
KVM: riscv: selftest: add Zicclsm, Ziccamoa, Ziccif, Za64rs
RISC-V: KVM: Allow Zicclsm, Ziccamoa, Ziccif, Za64rs in guest
MAINTAINERS: name the kvm-riscv/linux riscv_kvm_next branch
RISC-V: KVM: Don't write MMIO load result to x0
RISC-V: KVM: Use a raw lock for IMSIC wakeup state
...
|
|
* kvm-arm64/hcr-cleanup:
: \
: Cleanup the HCR_EL2 bit fields to exclusively use the generated
: definitions. This also fixes a nasty regression introduced in
: 6.16, where barriers were not upgraded from NSH to ISH anymore.
: /
KVM: arm64: Get rid of compatibility HCR_EL2 aliases
KVM: arm64: Convert HCR_xyz to HCR_EL2_xyz
KVM: arm64: Set HCR_EL2.BSU to Inner Shareable for guests
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
Bulk convert all usages of HCR_xyz to HCR_EL2_xyz (the autogenerated
version) using the following proof that I can't write shell scripts:
awk 'match($3, /__HCR.*/) { print $2}' arch/arm64/include/asm/kvm_arm.h |
while read i; do
j=$(echo $i | sed -e "s/HCR_\(.*\)/\1/");
git grep -lw $i arch/arm64/ | xargs sed -i -e "s/\b$i\b/HCR_EL2_$j/g";
done
... and fixup the result.
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
Tested-by: Fuad Tabba <fuad.tabba@linux.dev>
Tested-by: Itaru Kitayama <itaru.kitayama@fujitsu.com>
Link: https://patch.msgid.link/20261015173605.1072963-2-maz@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
Pull KVM updates from Paolo Bonzini:
"ARM:
- pKVM state isolation: years after pKVM started being enabled
upstream, we finally get the basics in the form of state isolation.
The vcpu state is now off-limit for the host for protected VMs
- pKVM also grows a hypervisor-private memory allocator, allowing it
to deal with sub-page allocation, much like everywhere else
- pKVM also lands a trace_printk() equivalent, which hopefully will
make debugging a bit easier
- First shot at annotating portions of the KVM/arm64 code to be
extracted and built on s390. The real fun is yet to come
- Allow pre-faulting of stage-2 canonical mappings from userspace,
joining x86 and s390 in implementing this feature
- Add a reverse mapping infrastructure for shadow stage-2,
significantly improving performance when MMU notifiers kick in, and
avoiding full unmap of all shadow tables for this VM
- Next step of GICv5 enablement, adding support for the IRS block,
and providing both SPI and LPI support. This is the first point
where KVM/arm64 can support an SMP GICv5 guest
- As a preliminary to eventually enabling CCA, add a bunch of
infrastructure to KVM/arm64, splitting some of the basic operations
into individual backends (normal VM, normal pKVM VM, protected pKVM
VM). This allows CCA to later slot in without polluting the rest of
KVM with its firmware-driven nonsense
- Make ID registers immutable from the point where a vcpu is able to
run, even if it fails to run. This prevents KVM from sampling
untrusted values
- Restore some type checking to the EL2 object. KVM/arm64 originally
had some strong type checking between the kernel and the EL2
object, but lost it along the way when the private hypercall
interface was (somewhat pointlessly) converted to SMCCC. Type
checking is back, syscall style, and with added sparse goodies
- Expand the FFA support to FFA_NOTIFICATION_*, and add some
sanitisation of the protocol that pKVM relays to the "trusted"
runtime
- Make GICv3 RD assignment more robust to userspace nonsense
- Convert the remaining MMU fault handler backends to
kvm_s2_fault_vma_info and co., fixing a number of bugs in the
process
- Fix the SError/SEA selftests to be both more robust and less
braindead
- Fix a smattering of documentation bugs, some of them as a result of
our AI friends spitting out credible assertions based on
non-existing symbols
- The usual collection of low impact bug fixes (SMCCC, timers, PMU)
Loongarch:
- Add separate vmid support
- Simplify MSI interrupt injection
- Add crash hypercall support
- Allow to set pv_feature until vCPU run
- Enable HW page table walk automatically in selftests
s390:
- rework page allocation for secure guest variable storage
- Selftests and documentation improvements
- Replace get_zeroed_page with kzalloc
- Fix corner cases of floating interrupt injection
- Initial arm64 on s390 support
x86:
- Clean up the handling of vcpu->arch.xen.long_mode to be consistent
and correctly handle 32-bit/64-bit alignment, and various other bug
fixes have accumulated over the months since v1
- Grab KVM's SRCU "lock" in kvm_zap_gfn_range() to fix a recurring
bug where callers fail to do so
- Fix a variety of issues in the pre-fault code that are mostly
benign in the current code base, but will show up in big ways
if/when the pre-faulting logic is used to implementing
prefetch-on-fault for the TDP MMU
- For PREEMPT_RT=y kernels, drop the rmaps' individual bit-spinlock
and just walk the rmaps' with mmu_lock taken. On the write side
(which always uses mmu_lock), the bit-spinlock was assuming that
preemption was disabled, but that is *not* true when rwlocks become
sleepable
- Prepare a memory_fault exit (paired with -EFAULT) when KVM hits an
error PFN, e.g. so that userspace can resolve transient issues
instead of killing the guest. Convert a variety of WARN()+EFAULT
paths into KVM_BUG_ON()+EIO so that KVM can hard commit to filling
memory_fault on -EFAULTs that occur in the page fault path
- Fix a NX hugepage un-accounting bug where KVM would always uncharge
the shadow MMU, even when handling TDP MMU pages
- Mask off the address space ID when checking for illegal usage of
kvm_set_internal_memslot() to prevent false negatives on the sanity
check
- Explicitly zero reserved flags when delivering a SynIC message to
avoid leaking a byte of stack information to the guest
- Rework TDX's handling of the canonical MMU root level to not rely
on userspace to fill CPUID correctly, as doing so lets userspace
trivially trigger a KVM_BUG_ON()
- Reject KVM_SET_NESTED_STATE if userspace "restores" L1 with
EFER.LMA=1 && EFER.LME=0, which confuses KVM into trying to
configure an 64-bit MMU without PAE
- Ignore EFER.LMA (clear it from KVM's cached save state) if
EFER.LME=0 when preparing L2 state, to fix another way to confuse
KVM's MMU (and because real hardware also ignores EFER.LMA)
- Improve KVM's sanity checks for a misconfigured MMU to (hopefully)
detect all of the ways KVM can botch EFER.LMA vs. CR4.PAE
- Remove the "full" out-param from kvm_x86_ops.flush_tlb_gva() as the
param is only used to avoid what effectively amounts to a single
memory write
- Fix a mostly-benign bug where KVM can write through a guest's
read-only mapping due to not checking the error code when
expediting emulated MMIO page faults
- Honor the guest's view of the "EFER.LMSLE must be zero" defeature
so that userspace can provide migration compatibility between Rome
and Milan+
- Pend a KVM_REQ_EVENT request when an exception is morphed into a
nested VM-Exit, otherwise KVM can fail to actually deliver the
nested VM-Exit, e.g. when injecting a page fault
- Add support for PML on SVM
- Add support for virtualizing Bus Lock Detect on SVM, and fix a
number of DR6 bugs and warts along the way
- Fix a variety of warts in KVM's handling of guest TSC, kvmclock,
and timekeeping in general, as a (big) first step towards
addressing several longstanding problems with the kvmclock
implementation. Most notably, fix a precision issue where
get_kvmclock_ns() computed the clock from the *host* TSC without
applying guest TSC scaling, causing systemic drift from the values
the guest computes from its own TSC
- Rip out KVM's shadow timekeeping structures, now that ktime
snapshots are powerful enough to replace it
- Misc fixes and cleanups
x86 (AMD):
- Fix a benign-in-practice bug where KVM unintentionally sets a
non-zero error code when delivering exceptions to SEV-ES+ guests
- Force a new ASID in both VMCBs on pCPU migration to fix a bug where
KVM can run a VMCB with a stale ASID due to the tracking being
per-CPU, not per-VMCB
- Return INVALID_INPUT to the guest instead of exiting to userspace
with -EIO if KVM can't access "guest request" buffer, i.e. try to
keep the guest alive if the guest or userspace goofs
x86 (Intel):
- Enable Notify VM-Exits (now called Instruction Timeout in the SDM)
and Bus Lock VM-Exits for TDX guests when they're enabled by
userspace (KVM forgot to wire up the TDX Module knobs)
- Move all common (between VMX and TDX) APIs/helpers to
common.h/main.c, and change all namespace prefixes from "vmx" to
"vt"
- Fix a thankfully benign bug (due to other areas of KVM being
hardened) where userspace can bypass INIT_MEM_REGION's address wrap
check by providing a nr_pages that will overflow a u64 when
converted to bytes, and fix an also- benign off-by-one bug in the
check (inclusive vs. exclusive)
- Set the vmcs.GUEST_ACTIVITY_STATE back to ACTIVE when userspace
sets KVM's mp_state to anything other than HALTED, e.g. so that a
vCPU doesn't get stuck in HLT if userspace emulates RESET
guest_memfd:
- Take the invalidate lock when unbinding a dying file to fix a race
with kvm_gmem_invalidate_end() that results in KVM leaving the MMU
invalidation count elevated. Not taking the lock in the first place
was pure misguided paranoid
- When binding a memslot, purge the bindings if storing the new
memslot fails, as Xarray's "store range" API can subtly fail
partway through
- Establish memslot<=>guest_memfd bindings *after* the memslot is
ready, as the memslot becomes reachable the instant the binding is
installed
- Don't grab a page refcount when reloading an SNP VMSA and stop
returning a "struct page" from guest_memfd's PFN lookup altogether,
to ensure that KVM doesn't hold any unnecessary refcounts (the
lifetime of the page is ensured by MMU notifiers and similar
hooks). Preventing unnecessary refcounts greatly simplifies
supporting in-place SHARED=>PRIVATE conversions
- Add support for in-place PRIVATE<=>SHARED conversion, i.e. allow a
single guest_memfd instance to be used for both PRIVATE and SHARED
variations of CoCo guests' memory (though not at the same time).
This eliminates the need to double allocate guest memory for TDX
and SNP, and is basically a hard requirement for pKVM, which relies
on being able to preserve memory contents on conversions
- Hack-a-fix a deadlock bug in ->error_remove_folio() by returning
-EBUSY if the filemap invalidate lock is contended. Punt on a
proper fix for the moment, as a proper fix likely involves
reworking truncate_error_folio()
- Add a pile of selftests for in-place conversions
- Misc guest_memfd cleanups
Generic:
- Move some configuration items to preprocessor symbols, to help with
arm-on-s390's need to compile the same code twice with different
values
- Allow KVM implementations to choose alternative device names, also
for s390's support of ARM virtualmachines
- Switch the KVM-VFIO interface and external consumers over to
standard file-based reference counting, eliminating all external
KVM symbol exports
Selftests:
- Rework KVM selftest's handling of memory regions all so that PPC
can force its page table allocations to be naturally aligned
- PPC support
- Fix emulation of VMPTRST and VMPTRLD when eVMCS is in-use to match
real Hyper-V's behavior
- Saturate L2's TSC frequency if stacking L1 and L2 scaling
multipliers exceeds what hardware supports, as attempting to run L2
with an illegal multiplier will either cause VM-Entry to fail (VMX)
or trigger a #GP when writing MSR_AMD64_TSC_RATIO (SVM). The #GP is
thankfully benign in the grand scheme as even "unsafe" WRMSRs have
exception fixup
- Rework KVM selftest's handling of memory regions so that PPC can
force its page table allocations to be naturally aligned
- Clean up selftest's VMX instruction helpers, and drop an absurd
number of completely useless return values, e.g. functions that
unconditionally return %true and then have every caller assert
success
- Add selftests support for Zhaoxin CPUs
- Overhaul the TSC scaling test to add a regression test for the
saturation behavior, and to increase its coverage in general
- Fix selftests usage various NUMA syscalls to account for an
undocumented, but known since 2007, off-by-one kernel bug. Add
helpers to try and avoid re-introducing the same flaws in the
future
- Automatically run xAPIC IPI migration test (between NUMA nodes)
when possible, as the entire point of the test is verify KVM's
handling of page migration of the APIC backing page
- Misc cleanups"
* tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (506 commits)
KVM: always make in-place conversion a compile-time constant when possible
KVM: x86: leave the legacy private attribute in for TDX and SEV
KVM: split CONFIG_KVM_PRIVATE_MEMATTR_VM out of KVM_VM_MEMORY_ATTRIBUTES
selftests: kvm: remove useless comma operator
arm64: Use proper include variant
MAINTAINERS: Add entry for ARM on s390 KVM
KVM: s390: arm64: Integrate arm on s390
KVM: s390: arm64: Implement basic page fault handler
KVM: s390: arm64: Implement vCPU IOCTLs
KVM: s390: arm64: Implement vm/vcpu create destroy
KVM: s390: arm64: Implement required functions
KVM: s390: Add basic arm64 kvm module
s390/hwcaps: Report SAE support as hwcap
KVM: s390: arm64: Introduce host definitions
s390: Introduce Start Arm Execution instruction
KVM: s390: Use arm64 code
s390/tools: Use arm64 headers
KVM: s390: Use dedicated function for migration mode
s390/Kconfig: remove PCI dependency from HAS_IOMEM
KVM: s390: Kick PV cpus at the right time for service irqs
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux
Pull arm64 updates from Catalin Marinas:
"Here are the arm64 updates for 7.4.
The more interesting bits are the removal of big-endian kernel support
(already marked BROKEN for some time), preemptible this_cpu_*()
operations and MPAM support for the firmware-backed (Fb) interface.
CCA (Arm's confidential compute) is slowly trickling in with host RMI
(Realm Management Interface) support and some refactoring, including a
dedicated SMCCC bus to advertise firmware services. There are also the
usual PMU updates for new hardware and a few fixes/cleanups.
Core features:
- Remove big-endian kernel support: CONFIG_CPU_BIG_ENDIAN has
depended on BROKEN for a while, so drop the remaining code (BE
hypervisor object, BPF JIT, assembly/crypto/library routines,
endianness helper macros, selftests) and force a little-endian
build
- Add preemptible this_cpu_*() operations (read/write, RMW, xchg,
cmpxchg and cmpxchg128) that no longer need to disable preemption
around the per-CPU address generation and access
- Preparation patches for implementing CONFIG_HOTPLUG_PARALLEL on
arm64
Memory management:
- Treat faults taken by EFI runtime firmware as kernel faults
- Clear PTE_WRITE for EFI_MEMORY_RO mappings (with hardware dirty
state management enabled, this is not truly read-only)
- Raise the default ARCH_MMAP_RND_BITS for 48-bit and 52-bit VA
configurations (28/26/24 bits for 4K/16K/64K pages)
- Implement clear_page[s]() using memset()
- Enable BBML2 no-abort (a.k.a. BBML3) for the NVIDIA Rigel CPU
Perf and PMU:
- Support for HiSilicon's ITS PMU
- Add support for some new events: INST_SIMD_ALU_VEC for Apple PMUs
and a cycle counter for the latest version of HiSilicon's MN PMU
hardware
- A bunch of fixes to the CXL PMU driver to address pre-existing
issues from Sashiko
- Fix generation of PERF_RECORD_ITRACE_START records for SPE and ETM
drivers
- Reduce excessive stack consumption in the Marvell DDR PMU driver
- Minor driver updates and cleanups all over
Arm Confidential Compute Architecture (CCA):
- Add an Arm SMCCC bus for firmware services discovered through SMCCC
and move the TRNG and CCA guest (TSM provider) drivers onto it,
removing the dummy CCA platform device and adding a sysfs ABI for
Realm guests
- Move the Realm Service Interface (RSI) code out of arch/arm64/ into
drivers/firmware/arm_rmm/
- Add the host side of the Realm Management Interface (RMI): detect,
configure and activate the RMM, add SRO support and wrappers for
the Realm RMI commands
- Block hibernation (new arch hook in the PM core) and kexec while
the RMM is active. To be revisited in the future with proper
support for deactivation
- Handle Granule Protection Faults (GPFs) taken by the host, caused
by accesses to memory delegated to the RMM. In a future patch, the
exception table lookup will be limited to is_kdump_kernel() cases
MPAM (Memory Partitioning And Monitoring):
- Add support for the MPAM firmware-backed (Fb) interface, allowing
MSC accesses to be mediated by firmware
- Fix a list of theoretical issues with the driver (MSC access error
propagation, MBWU monitor save/restore and locking, resctrl class
handling, error interrupt restore, maximum number of PARTIDs)
- Add the mailing list and a missing file to the MAINTAINERS entry
- Document when to set arm64.nompam
Errata and CPU features:
- Add the NXP i.MX8QM/8QP workaround for the Cortex-A53 cache
coherency issue (ERR050104), using a new ARM64_CPUCAP_SOC_ERRATUM
capability type for SoC-level errata
Miscellaneous:
- Mask debug exceptions (DAIF) around the hardware breakpoint
register manipulations when called from generic kernel code
- Don't zero the SVE state buffer when the SVE state is live
- Return -EPERM instead of -EBUSY from the GCS prctl() locking
failures (matching x86), with the selftest updated. Slight ABI
change but no software deployment yet
- kselftest: fix unsound MTE fault skipping by introducing MTE-safe
memory accessors, avoid copying each signal test binary more than
once
- Update sysreg definitions (fix three enum values and two field
names, mark architecturally dropped fields as RES0) and remove the
TME transaction state check from the BRBE driver
- Fix the description of ARM64_HAS_LPA2 and remove a redundant
ID_AA64MMFR0 define"
* tag 'arm64-upstream' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux: (144 commits)
arm64: cpufeature: Enable BBML2 for NVIDIA Rigel
arm64: cputype: Add NVIDIA Rigel definitions
arm64: raise default ARCH_MMAP_RND_BITS for 48-bit and 52-bit VA
arm64/debug: mask debug exceptions when called from common code
kselftest/arm64: signal: copy each test binary only once
kselftest/arm64: Check for -EPERM not -EBUSY in the locking test
arm64: gcs: Return -EPERM not -EBUSY for prctl() locking failures
perf: arm_brbe: Remove the check for TME transaction states
arm64: sysreg: Mark fields the architecture dropped as RES0
arm64: sysreg: Fix the names of two misspelt fields
arm64: sysreg: Fix three incorrect enum values
arm64: smp: Defer RCU registration during secondary CPU bringup
arm64: smp: Don't bother printing the I-cache policy for each CPU
arm64: smp: Defer update of secondary CPU capabilities
arm64: smp: Tidy up cpuinfo init and cpufeature updates
arm64: smp: Tidy up smp_prepare_cpus()
arm64: cpufeature: Store every ID register with its overrides applied
arm64: cpufeature: Read MPAMIDR_EL1 in __cpuinfo_store_cpu()
arm64: cpufeature: Add read_cpuid_with_overrides()
arm64: cpufeature: Check arm64_ftr_regs[] before the first store
...
|
|
Commit 06499de8f998 ("KVM: arm64: Skip fixed-feature state flush for
protected vCPUs") moved the copy of the host's TWI and TWE in
flush_hyp_vcpu() into the non-protected branch: a protected vCPU takes
them in __pkvm_vcpu_load() instead, and only VSE comes from the host on
each entry.
Commit 04447b95c62b ("KVM: arm64: Use the host's HCR_EL2 for
non-protected VMs in pKVM") was written against the old code, where
both VM types took TWI, TWE and VSE on each entry, and kept that for a
protected VM as PKVM_HCR_EL2_HOST_PVM. The merge of the two applies
PKVM_HCR_EL2_HOST_PVM in the common path, so a protected vCPU takes TWI
and TWE from the host's vCPU on every entry again.
That doesn't change what a protected guest runs with today: once the
vCPU is initialised, the host only changes TWI and TWE in
vcpu_set_wfx_traps(), which pkvm_vcpu_load() calls right before
__pkvm_vcpu_load(). But it undoes part of 06499de8f998, and has EL2
read a protected vCPU's trap configuration from host memory on every
entry, which nothing needs.
Make PKVM_HCR_EL2_HOST_PVM VSE alone and add TWI and TWE to
PKVM_HCR_EL2_HOST_NPVM, so that each mask again lists what EL2 takes
from the host on each entry for its VM type. A non-protected VM takes
the same bits as before. Say in the comments that a protected vCPU's
TWI and TWE come at vCPU load.
llminus, an LLM-assisted merge conflict resolution tool, found the
difference while resolving the same conflict in an integration tree.
Fixes: 439d8f9cb01c ("Merge tag 'kvmarm-7.4' of git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD")
Assisted-by: LLM
Signed-off-by: Sasha Levin <sashal@kernel.org>
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
Tested-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20261017165204.3343275-1-sashal@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD
KVM/arm64 updates for 7.4
* New features
- pKVM state isolation: years after pKVM started being enabled
upstream, we finally get the basics in the form of state
isolation. The vcpu state is now off-limit for the host for
protected VMs.
- pKVM also grows a hypervisor-private memory allocator, allowing it
to deal with sub-page allocation, much like everywhere else.
- pKVM also lands a trace_printk() equivalent, which hopefully will
make debugging a bit easier.
- First shot at annotating portions of the KVM/arm64 code to be
extracted and built on s390. The real fun is yet to come.
- Allow pre-faulting of stage-2 canonical mappings from userspace,
joining x86 and s390 in implementing this feature.
- Add a reverse mapping infrastructure for shadow stage-2,
significantly improving performance when MMU notifiers kick in,
and avoiding full unmap of all shadow tables for this VM.
- Next step of GICv5 enablement, adding support for the IRS block,
and providing both SPI and LPI support. This is the first point
where KVM/arm64 can support an SMP GICv5 guest.
* Improvements
- As a preliminary to eventually enabling CCA, add a bunch of
infrastructure to KVM/arm64, splitting some of the basic
operations into individual backends (normal VM, normal pKVM VM,
protected pKVM VM). This allows CCA to later slot in without
polluting the rest of KVM with its firmware-driven nonsense.
- Make ID registers immutable from the point where a vcpu is able
to run, even if it fails to run. This prevents KVM from sampling
untrusted values.
- Restore some type checking to the EL2 object. KVM/arm64 originally
had some strong type checking between the kernel and the EL2
object, but lost it along the way when the private hypercall
interface was (somewhat pointlessly) converted to SMCCC. Type
checking is back, syscall style, and with added sparse goodies.
- Expand the FFA support to FFA_NOTIFICATION_*, and add some
sanitisation of the protocol that pKVM relays to the "trusted"
runtime.
* Fixes
- Make GICv3 RD assignment more robust to userspace nonsense
- Convert the remaining MMU fault handler backends to
kvm_s2_fault_vma_info and co, fixing a number of bugs in the
process.
- Fix the SError/SEA selftests to be both more robust and less
braindead.
- Fix a smattering of documentation bugs, some of them as a result
of our AI friends spitting out credible assertions based on
non-existing symbols.
- The usual collection of low impact bug fixes (SMCCC, timers, PMU).
|
|
* kvm-arm64/pkvm-tracing-7.4:
: \
: trace_printk() support for pKVM tracing infrastructure. Patches courtesy
: of Vincent Donnefort. From the cover letter:
:
: "This series adds a hypervisor event "hyp_printk" which enables
: developers to log pretty much anything into the hypervisor tracing
: buffer, just like the kernel function trace_printk()."
: /
KVM: arm64: Add hyp_printk event to nVHE/pKVM hyp
tracing/remotes: Add REMOTE_EVENT_CUSTOM_PRINTK() helper
KVM: arm64: Move kvm_define_hypevents.h to arch/arm64/kvm/
KVM: arm64: Allow early calls to pKVM host_share/unshare_hyp
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
* kvm-arm64/pkvm-allocator-7.4: (21 commits)
: \
: Dynamic allocator for pKVM, courtesy of Vincent Donnefort.
: From the cover letter:
:
: "pKVM historically lacked a dynamic memory allocator: all hypervisor-side
: VM and VCPU structures had to be sized on the host, allocated as
: contiguous pages and donated to the hypervisor.
:
: This design tightly coupled the hypervisor's memory footprint to
: host-side constraints, complicated memory reclaim, and severely
: restricted VM scalability.
:
: This patch series introduces a dynamically-mapped custom heap allocator
: (hyp_allocator) to the pKVM hypervisor. The initial users are the
: pkvm_hyp_vm and pkvm_hyp_vcpu structs, and the hypervisor tracing
: metadata."
: /
KVM: arm64: Fix bpages backing allocation size in hyp_trace
KVM: arm64: Prevent 64-bit overflow when computing hyp clock rate
KVM: arm64: pkvm: Drop keneldoc markers for the allocator structs
KVM: arm64: Alloc simple_buffer_page using pKVM hyp allocator
KVM: arm64: Reject hyp trace descriptors with fewer than 3 pages
KVM: arm64: Reject hyp trace descriptors with fewer CPUs than hyp_nr_cpus
KVM: arm64: Rename vCPU pkvm_memcache to stage2_mc
KVM: arm64: Alloc pkvm_hyp_vcpu using pKVM heap allocator
KVM: arm64: Alloc pkvm_hyp_vm using pKVM heap allocator
KVM: arm64: Move hyp_vm refcount into the structure
KVM: arm64: Filter out non-kernel addresses in kern_hyp_va
KVM: arm64: Add a shrinker for pKVM
KVM: arm64: Add selftests for the pKVM heap allocator
KVM: arm64: Add reclaim interface for the pKVM heap alloc
KVM: arm64: Add PKVM_HYP_REQ_HYP_ALLOC request
KVM: arm64: Add pkvm_hyp_req infrastructure
KVM: arm64: Allow kvm_hyp_memcache usage outside of stage-2
KVM: arm64: Add a heap allocator for the pKVM hyp
KVM: arm64: Add pkvm_map_private_va_range
KVM: arm64: Add pkvm_remove_mappings
...
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
* kvm-arm64/pkvm-state-7.4:
: \
: State protection for protected pKVM guests, finally starting to deliver
: on its promises... Patches courtesy of Fuad Tabba.
:
: "Following the vCPU state-sync series, this series completes the
: job for protected VMs: a protected guest's register state stays at
: EL2, and the host sees only what handling each exit requires."
: /
KVM: arm64: Only emulate an SError's entry for vCPUs with NV
KVM: arm64: Document the protected VM userspace API
KVM: arm64: Advertise the capabilities that protected VMs support
KVM: arm64: Reject host power-on of a vCPU that EL2 holds powered off
KVM: arm64: Reject host access to protected VM private state
KVM: arm64: Add per-EC entry/exit state marshalling for protected guests
KVM: arm64: Inject an UNDEF at EL2 for unhandled protected guest exits
KVM: arm64: Prevent host PC adjustments for protected vCPUs
KVM: arm64: Restrict KVM_ARM_VCPU_INIT and PSCI version for protected VMs
KVM: arm64: Handle PSCI calls for protected VMs at EL2
KVM: arm64: Implement HVC handling for protected guests at EL2
KVM: arm64: Add system register reset framework for protected VMs
KVM: arm64: Add {flush,sync}_hyp_timer_state() primitives
KVM: arm64: Skip fixed-feature state flush for protected vCPUs
KVM: arm64: Introduce per-EC entry handlers for pKVM
KVM: arm64: Disable steal time for protected VMs
KVM: arm64: Pin the host vCPU before adjusting its PC under pKVM
KVM: arm64: Validate the host vCPU's VM before reading it under pKVM
KVM: arm64: Sync HCR_EL2.VSE back to the host vCPU under pKVM
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
* kvm-arm64/s390-7.4:
: \
: Add the first attempt at sharing code with s390 in an effort to support
: the arm64-on-s390 effort. This relies on ARM64_S390_COMMON markers in
: relevant places. Patches courtesy of Steffen Eiden.
: /
KVM: arm64: Share arm64 code with s390
arm64: Share arm64 headers with s390
arm64: Prepare sharing arm64 headers with s390
KVM: arm64: Use accessor functions for core regs
KVM: arm64: Access elements of vcpu_gp_regs individually
arm64: sysreg: Convert SPSR_ELx to automatic register generation
arm64: ptrace: Use constants for compat register numbers
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
* kvm-arm64/s2-rmap: (31 commits)
: \
: Add a per-shadow stage-2 reverse map tracking infrastructure,
: considerably reducing the overhead of MMU notifiers when running L2
: guests. Also fixes issues with the shadow S2 MMU debugfs infrastrtcture.
:
: Patches courtesy of Wei-Lin Chang.
:
: This branch drags the kvmarm-fixes-7.3-1 tag, as this series depends
: on some of the fixes.
: /
KVM: arm64: ptdump: Fix shadow ptdump sleep-in-atomic-context problem
KVM: arm64: ptdump: Check the page tables aren't freed when accessing
KVM: arm64: Refactor kvm_unmap_gfn_range() with common variables
KVM: arm64: nv: Drop kvm_s2_mmu pointer from kvm_guest_s2_mapping
KVM: arm64: nv: Avoid full shadow stage-2 unmap
KVM: arm64: nv: Track guest stage-2 mapping removal
KVM: arm64: nv: Track guest stage-2 mapping creation
KVM: arm64: nv: Introduce guest stage-2 tracking structures
KVM: arm64: Use a variable for the canonical IPA in kvm_s2_fault_map()
KVM: arm64: Fix AArch32 DBGBXVR<n> handling
KVM: arm64: Fix FGT mapping for HFGITR_EL2.nGCSEPP
KVM: selftests: fix steal_time for arm64 with host page size > 4K
KVM: arm64: Don't WARN on an unknown VM ioctl in protected mode
KVM: arm64: nv: Delay freeing of shadow S2 structures until VM destruction
KVM: arm64: nv: Fix life cycle of the nested_mmus array
KVM: arm64: Check every private mapping is hyp-owned at pKVM init
KVM: arm64: Move the private VA allocation cursor to __io_map_next
KVM: arm64: Match hyp text by physical address in fix_host_ownership()
KVM: arm64: Transfer the hyp stack pages out of the host stage-2
KVM: arm64: selftests: Test empty SMCCC filter range at base 0
...
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
Create an event to allow developers to log pretty much anything into the
hypervisor tracing buffer with trace_hyp_printk(), just like the kernel
tracing has the function trace_printk().
#include <asm/kvm_hypevents.h>
trace_hyp_printk("foobar");
trace_hyp_printk("foo=%d", foo);
trace_hyp_printk("foo=%d bar=0x%016llx", foo, bar);
To ensure writing into the tracing buffer is fast, store the string
format into a kernel-accessible ELF section. The hypervisor only has to
write into the event the string ID, which is the delta between the
hyp_string_fmt and the start of the ELF section.
The string format is parsed by the kernel. It is therefore not possible
to dereference hypervisor pointers and the format specifier %s is not
supported. Also the %p specifiers will fail to find the hypervisor
function.
To not waste tracing buffer data, use a dynamic size. Each argument is 8
bytes.
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Reviewed-by: Fuad Tabba <tabba@google.com>
Tested-by: Fuad Tabba <tabba@google.com>
Link: https://patch.msgid.link/20261001143000.3371786-5-vdonnefort@google.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
Transition the allocation of the hypervisor vCPU state structure
(pkvm_hyp_vcpu) from the host to the hypervisor using the new pKVM heap
allocator (hyp_alloc()).
Previously, the host was responsible for calculating the size of,
allocating, and donating memory for pkvm_hyp_vcpu during vCPU creation.
With the heap allocator in place, the hypervisor now allocates this
structure dynamically at EL2.
Use the pkvm_call_hyp_req() wrapper in the host to invoke
__pkvm_create_hyp_vcpu, which automatically handles any top-up requests
if the hypervisor runs out of heap memory during allocation.
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Tested-by: Fuad Tabba <fuad.tabba@linux.dev>
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20261001142849.3367614-15-vdonnefort@google.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
Transition the allocation of the hypervisor VM state structure
(pkvm_hyp_vm) from the host to the hypervisor using
the new pKVM heap allocator (hyp_alloc()).
Previously, the host was responsible for calculating the size of,
allocating, and donating memory for pkvm_hyp_vm during VM creation. With
the heap allocator in place, the hypervisor now allocates this structure
dynamically at EL2.
Use the pkvm_call_hyp_req() wrapper in the host to invoke
__pkvm_init_vm, which automatically handles any top-up requests if the
hypervisor runs out of heap memory during allocation.
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Tested-by: Fuad Tabba <fuad.tabba@linux.dev>
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20261001142849.3367614-14-vdonnefort@google.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
In preparation for allocating hyp_vm using the pKVM heap allocator
(hyp_alloc()), move its reference count out of the page metadata
(vmemmap) and place it into the structure itself. This transition is
necessary because hyp_alloc() allows multiple small objects to share the
same physical page.
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
Tested-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20261001142849.3367614-13-vdonnefort@google.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
Introduce a comprehensive runtime selftest for the pKVM hypervisor heap
allocator, executed during init when CONFIG_NVHE_EL2_DEBUG is enabled.
The selftest runs entirely at EL2 and exercises allocator's core
mechanisms:
* over-sized allocations
* basic allocation and alignment
* chunk recycling, splitting, merging
* memory reclaiming
* memory topup
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
Tested-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20261001142849.3367614-10-vdonnefort@google.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
Currently, memory used by the hypervisor comes from donations that are
embedded within HVCs. e.g. __pkvm_init_vm()'s hyp_vm.
This is cumbersome: the host needs to know the size of those
struct, the memory must be page-aligned and physically contiguous which
may be difficult to satisfy when host memory is highly fragmented.
Create a heap allocator to manage VA-contiguous memory. This allocator
grows upward, recycles unused chunks of memory and provides a simple API
to allocate and free:
hyp_alloc(size), hyp_free(addr)
This heap allocator also manages the underlying physical memory,
allowing the host to top up the allocator's pool and reclaim memory.
hyp_alloc_topup(), hyp_alloc_reclaim().
Pages remain mapped in the allocator's VA-space as long as they are not
reclaimed.
When the allocator runs out of memory, hyp_alloc() fails and
hyp_alloc_errno() returns -ENOMEM to signal that a top-up is required.
Additionally, harden the allocator with a fast hash of the chunks
meta-data. This intends to detect accidental corruptions and makes the
allocator slightly more robust to potential attacks.
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Tested-by: Fuad Tabba <fuad.tabba@linux.dev>
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20261001142849.3367614-5-vdonnefort@google.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
In preparation for the pKVM heap allocator, introduce
pkvm_map_private_va_range() to map a physical range into the pKVM
private range.
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
Tested-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20261001142849.3367614-4-vdonnefort@google.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
Add the counterpart to pkvm_create_mappings(), allowing previously
mapped ranges to be removed. This will be useful for the upcoming pKVM
heap allocator to manage its private mappings.
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
Tested-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20261001142849.3367614-3-vdonnefort@google.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
Mappings in the pKVM private range are not identity mapped, making the
standard __hyp_pa() unsuitable for translating these addresses.
Introduce pkvm_private_va_range_pa() to resolve physical addresses for
this range by walking the hypervisor page-table. This will be useful for
the upcoming pKVM heap allocator.
Signed-off-by: Vincent Donnefort <vdonnefort@google.com>
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
Tested-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20261001142849.3367614-2-vdonnefort@google.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
__kvm_adjust_pc() lets the host advance a vCPU's PC or inject an
exception, which for a protected vCPU would let the host redirect
guest execution. Leave the request on the host copy there: the entry
handlers apply the host's PC_UPDATE_REQ at the next entry, where EL2
allows it.
__kvm_adjust_pc() adjusts the vCPU a get/put pair returns, the one it
was given outside pKVM. Both host callers hold the vCPU mutex, so a
hyp vCPU loaded for the vCPU is loaded on the calling CPU, and EL2's
own calls pass the hyp vCPU itself. For a loaded protected vCPU,
kvm_adjust_pc_get() returns NULL and the request stays on the host
copy. For a loaded non-protected vCPU, PKVM_HOST_STATE_DIRTY selects
the copy, since adjusting the hyp vCPU while the host copy is
authoritative loses the update at the next flush. Adjusting the hyp
vCPU copies PC_UPDATE_REQ in and back out again: without the copy
back, INCREMENT_PC outlives the adjustment and the next
KVM_SET_VCPU_EVENTS trips WARN_ON(INCREMENT_PC) in
kvm_pend_exception(). With no hyp vCPU loaded, as under
KVM_SET_VCPU_EVENTS, the host copy is adjusted as before.
Until the marshalling patch clears PC_UPDATE_REQ on the host copy at
exit, a KVM_RUN that returns to userspace with INCREMENT_PC set leaves
it on the host copy of a loaded protected vCPU, and a
KVM_SET_VCPU_EVENTS before the next run then trips that WARN_ON.
Suggested-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20261001135711.1640520-13-fuad.tabba@linux.dev
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
EL2 implements PSCI 1.1 for protected VMs: CPU_ON, CPU_OFF,
PSCI_VERSION and PSCI_FEATURES are resolved at EL2 (CPU_ON and CPU_OFF
still exit to the host, which only schedules the target or stops
scheduling it), AFFINITY_INFO, CPU_SUSPEND and the platform power
operations are forwarded to the host, and anything else returns
NOT_SUPPORTED, including the TRNG calls and the functions above 1.1,
SYSTEM_OFF2 among them, that the host handled for a protected guest
until now. TRNG for protected guests is a follow-up. AFFINITY_INFO
stays with the host, which returns OFF only once it has stopped
scheduling the target: the host is what a guest polls to see a CPU_OFF
complete before it issues the next CPU_ON, as Linux does on hotplug.
Three consequences follow:
- A protected VM has one primary vCPU, the first whose hyp vCPU is
created with mp_state RUNNABLE. A second one, or an mp_state other
than RUNNABLE or STOPPED, fails that vCPU's first KVM_RUN with
-EINVAL.
- CPU_ON finds its target among the hyp vCPUs, which exist from the
target's first KVM_RUN; before that the guest gets
INVALID_PARAMETERS.
- A vCPU EL2 holds powered off doesn't run: handle___kvm_vcpu_run()
returns ARM_EXCEPTION_IL, reported as KVM_EXIT_FAIL_ENTRY. Its
existing bail-outs return the same code, for every hyp vCPU, instead
of an -EINVAL whose bit 31 the host read as a pending SError.
Non-protected VMs keep power_state ON and accept any mp_state.
Each protected vCPU is OFF, ON_PENDING or ON. CPU_ON moves the target
to ON_PENDING, and the target's next run resets it and moves it to ON.
The racing transitions are cmpxchg, and the reset state is published
with a release/acquire pair, documented at each site. CPU_OFF clears
the reset flag and stores OFF with a release, and CPU_ON's writes are
control dependent on its cmpxchg reading OFF, so the target's accesses
to reset_state are ordered before the winner's writes. Rolling a CPU_ON
the host failed back to OFF requires the host's return value, which
the per-EC marshalling patch delivers along with the rollback. Until
then such a target stays ON_PENDING, and the reset has no observable
effect: flush_hyp_vcpu() copies the host's context in on every entry
until that patch removes the copy, so the target enters on the host's
values rather than the ones EL2 reset.
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20261001135711.1640520-11-fuad.tabba@linux.dev
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
Add kvm_reset_pvm_sys_regs() and the pvm_sys_reg_reset_vals[] table
that drives it, and call it from init_pkvm_hyp_vcpu() for protected
vCPUs. The table holds the registers KVM resets that EL2 loads for a
protected vCPU: the ones __sysreg_restore_state_nvhe() restores, and
CNTV_CTL_EL0, which flush_hyp_timer_state() loads from a copy the
host's kvm_timer_vcpu_reset() doesn't write. The ptrauth keys, which
entry.S loads, are not in the table: sys_regs.c resets them to
UNKNOWN, and a vCPU that CPU_ON brings back keeps the keys the guest
last set, one UNKNOWN value in place of another.
A register that resets to 0 still takes an entry: a later patch calls
kvm_reset_pvm_sys_regs() again for PSCI CPU_ON at EL2, on a context
that has run.
A poison value stands in where sys_regs.c resets to UNKNOWN, and for
VBAR_EL1 and CONTEXTIDR_EL1 in place of the 0 it uses.
MPIDR_EL1 is derived from vcpu_id, as for any KVM guest.
AMAIR_EL1 resets to 0 where sys_regs.c reads the hardware: the host
writes it freely before the hypercall, and it's loaded on every guest
entry.
Until the per-EC marshalling patch removes the host's full-context
copy from flush_hyp_vcpu(), that copy overwrites these values on every
entry. It preserves CNTV_CTL_EL0, whose entry writes the zero the page
already holds. So this patch has no observable effect on its own.
No functional change intended.
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20261001135711.1640520-9-fuad.tabba@linux.dev
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
Add an ESR_EL2.EC-indexed handler table, entry_hyp_vm_handlers[],
read from flush_hyp_vcpu() on re-entry when the previous exit was
a trap: sync_hyp_vcpu() records the exit reason in the hyp vCPU as
exit_code, and the trap's exception class comes from its ESR_EL2. Its
single generic handler stands in until the per-EC marshalling patch
replaces the table with protected-only ones.
Entry carries the host's requested PC updates to the hyp vCPU: add
vcpu_copy_flag() to copy a masked set of flags between the two vCPU
structures, with PC_UPDATE_REQ covering INCREMENT_PC and the
pending-exception flags. The wholesale iflags copy moves into the
non-protected branch; a protected vCPU takes only PC_UPDATE_REQ.
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Co-developed-by: Fuad Tabba <fuad.tabba@linux.dev>
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20261001135711.1640520-6-fuad.tabba@linux.dev
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
On an MTE-capable host under pKVM, enter_exception64() reads the VM's
MTE flag through vcpu->kvm, which for a host vCPU is a host-writable
pointer nothing validates. The host can point it at any address in the
hyp linear map and read back bit 1 of that word through PSR_TCO in the
vCPU's CPSR, or panic the hypervisor with an unmapped one.
Get the VM through a get/put pair around the read: a loaded vCPU's is
the hyp VM, an unloaded host vCPU's is read once and pinned, and a
pointer the host never shared leaves TCO clear.
Fixes: ea7fc1bb1cd1b ("KVM: arm64: Introduce MTE VM feature")
Reported-by: Sashiko <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/all/20260914070536.877D91F000FF@smtp.kernel.org/
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Cc: stable@vger.kernel.org
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20261001135711.1640520-3-fuad.tabba@linux.dev
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
While for arm64 the members of vcpu_gp_regs are allocated continuous
this is not necessarily true for other architectures implementing ARM.
Let vcpu_gp_regs() no longer return the address of the user_pt_regs in
the vcpu context but the address of the gp-register array field in the
user_pt_reg struct. Note, this change semantically excludes pc from the
GPRs.
No functional change.
Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Signed-off-by: Steffen Eiden <seiden@linux.ibm.com>
Link: https://patch.msgid.link/20260930173441.66762-8-seiden@linux.ibm.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
For each vCPU, pKVM's EL2 builds its own HCR_EL2 and takes only TWI, TWE
and VSE from the host's value on each entry. For a non-protected VM it
has fallen behind the host's: on a CPU with MTE the guest can read
GMID_EL1, on one with FEAT_EVT2 but no FGT it can execute a TLBI OS its
ID registers hide, an interrupt injected without a vGIC (VI, VF) never
arrives, and set/way emulation loses the TVM trap it relies on.
For a non-protected VM, also take the bits the host varies with the VM's
configuration or at runtime: VI, VF, TVM, TID2, TID4, TID5 and TTLBOS.
They only pend interrupts for, or add traps to, a VM the host controls.
The traps exit to the host, which handles them as it does without pKVM.
The bits that change what EL2 does on entry and exit (E2H, RW, API/APK)
or depend only on the CPU (TEA, TERR, FWB) stay EL2's. A protected VM
still takes only TWI, TWE and VSE.
EL2 now sets TID2 or TID4 only for a protected VM, and never sets ATA,
as the host rejects KVM_CAP_ARM_MTE in pKVM.
Fixes: b56680de9c648 ("KVM: arm64: Initialize trap register values in hyp in pKVM")
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20260929090031.3829185-4-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Big-endian support on arm64 depends on BROKEN. In preparation for
removing the dead code altogether, remove the support for generating
ELF relocations for a big-endian hypervisor object and drop the unused
big-endian spinlock definition from nVHE.
Acked-by: Catalin Marinas <catalin.marinas@arm.com>
Acked-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Marc Zyngier <maz@kernel.org>
Acked-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Will Deacon <will@kernel.org>
Acked-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
|
|
fix_host_ownership() transfers only what it walks, so a hyp mapping
outside the linear map is not manipulated by the walk.
Walk the quarter of the VA space holding the private range and the
vmemmap once the transfer is done, and fail init unless every valid
leaf is hyp-owned: in the vmemmap when the page is memory, and in the
host stage-2, where hyp text may instead be mapped without write
access. A leaf that is not memory has no vmemmap entry and is checked
against the host stage-2 alone. Hyp text is matched by physical
address, since the only executable mapping in the range is the
Spectre-v3a vectors, whose VA is a private allocation, and an
executable mapping of anything else must not be host-readable. The
vmemmap can be block-mapped, so the walker checks each page of a leaf.
Suggested-by: Will Deacon <will@kernel.org>
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Tested-by: Vincent Donnefort <vdonnefort@google.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260908110713.1540304-5-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
On a non-hVHE host, fix_host_ownership_walker()'s test for PAGE_HYP_EXEC
never matches: KVM_PGTABLE_PROT_UX is cleared at map time and only PX is
reported on read-back. Hyp text is therefore donated rather than left
read-only in the host stage-2, and the instruction dump in
nvhe_hyp_panic_handler() reads a page the host has no access to.
Match the text by physical address instead, in a helper a later patch
reuses. A test on the permissions would leave any other executable
mapping host-readable too.
Fixes: 80cbfd7174f31 ("KVM: arm64: Honor UX/PX attributes for EL2 S1 mappings")
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Tested-by: Vincent Donnefort <vdonnefort@google.com>
Reviewed-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260908110713.1540304-3-fuad.tabba@linux.dev
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
The nVHE hypervisor takes host virtual addresses as hypercall arguments
and translates each with kern_hyp_va() before use. Nothing marks them as
host-owned, so dereferencing one untranslated at EL2 - a recurring bug
class - is invisible to the compiler.
Add a __kern sparse address space, active only for EL2 code, and tag the
host-VA parameters in the hypercall declarations. kern_hyp_va_host() is
the only sanctioned unwrap: it translates the address, preserves the
pointee type (stripped of qualifiers, as with the percpu accessors) and
drops the tag with a __force cast, so an untranslated host VA fails
sparse. The tag flows from the shared declaration into the generated
handler and on into the donated-memory and tracing-descriptor helpers,
so a handler cannot extract a host VA without it. Host code sees plain
pointers, and the tag is checker-only: no code is generated.
container_of() casts through void * and drops the address space, so
__get_host_hyp_vcpus() now takes an already translated vCPU and its
callers unwrap. Translating a vgic_v3_cpu_if before taking its container
is equivalent, since va_mask spans every bit in which two linear-map
addresses differ.
Reviewed-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20260901140326.3812068-12-fuad.tabba@linux.dev
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
Four hypercalls take host VAs as unsigned long: the donated vm, pgd
and vcpu regions, and the tracing descriptor. Retype the arguments
and their EL2 consumers as void *, so that the typed hypercall
declarations introduced later in the series can attach a sparse
address space to them; an address space attaches only to pointers.
No functional change intended.
Reviewed-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20260901140326.3812068-8-fuad.tabba@linux.dev
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
* kvm-arm64/feat-nv3: (29 commits)
: Support for FEAT_NV2p1 and FEAT_NV3, courtesy of Marc Zyngier
:
: Couple of accelerations for nested virtualization:
:
: - Elide traps for EL1 alias registers when hardware guarantees that
: EL2-only bits are stateful (NV2p1)
:
: - Use TGE-aware hardware to avoid unnecesary ERET/TLBI traps that
: remain 'in-context' (i.e. L1 kernel/userspace)
arm64: Add override for ID_AA64MMFR4_EL1.NV_frac
KVM: arm64: selftest: Add NVHCR_EL2 to get-reg-list
KVM: arm64: Expose FEAT_NV3 to guests
KVM: arm64: Add FEAT_NV3 detection
KVM: arm64: Engage NV3 TLBI trap elision
KVM: arm64: Engage NV3 ERET trap elision
KVM: arm64: Add NVHCR_EL2 context switching
KVM: arm64: Add routing for NVHCR_EL2 trap
KVM: arm64: Add NVHCR_EL2 handling to the sysreg array
KVM: arm64: Add sanitisation for NVHCR_EL2
KVM: arm64: Make HCR_EL2 a non-VNCR register
KVM: arm64: Add kvm_has_nv{2,3}() predicates
KVM: arm64: Add NV3 control bits to HCRX_EL2 sanitisation
KVM: arm64: Split NV-specific exit fixups from the non-NV handling
arm64: Add ARM64_HAS_NV3 capability
arm64: sysreg: Add HCRX_EL2 bits related to FEAT_NV3
arm64: sysreg: Add NVHCR_EL2 description as a mirror of HCR_EL2
arm64: Add FEAT_NV2p1 detection
KVM: arm64: Expose FEAT_NV2p1 to NV guests
KVM: arm64: Relax CNTHCTL_EL2 handling when FEAT_NV2p1 is present
...
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
* kvm-arm64/pkvm-7.3: (26 commits)
: pKVM updates for 7.3
:
: - Avoid name collision on trace_clock() when CONFIG_NVHE_EL2_TRACING is
: disabled (Mostafa Saleh)
:
: - Clean up state tracking for whether the EL2 shadow VM has been
: created (Fuad Tabba)
:
: - Synchronize SCTLR_EL1 when injecting an exception to use current
: PAN/SSBS state (Fuad Tabba)
:
: - Avoid unnecessary cache maintenance when I/D-cache are known to be
: coherent in pKVM (Mostafa Saleh)
:
: - Lazy vCPU context save/restore for pKVM (Fuad Tabba)
:
: - Various fixes to the stage-2 MMU for pKVM (Fuad Tabba)
:
: - Allow counter offsetting of non-protected guests in protected mode
: (Mostafa Saleh)
:
: - Condition the 'broken CNTVOFF' mitigation on a VM actually having a
: nonzero offset, fixing boot failures of pVMs on affected hardware
: (Mostafa Saleh)
KVM: arm64: Fix hvhe and broken CNTVOFF_EL2
KVM: arm64: Fix timer offsets for non-protected VMs
KVM: arm64: Make timer_get_offset() work in all contexts
KVM: arm64: selftests: Add stage-2 block transition test
KVM: arm64: Don't advertise eager page splitting under pKVM
KVM: arm64: Don't WARN on pKVM stage-2 map failures
KVM: arm64: Skip pKVM stage-2 flush when FWB is enabled
KVM: arm64: Top up stage-2 memcache for dirty logging faults
KVM: arm64: Top up the memcache for pKVM permission faults
KVM: arm64: Skip cache maintenance for non-cacheable pKVM mappings
KVM: arm64: Implement lazy vCPU state sync for non-protected guests
KVM: arm64: Add primitives to flush/sync the VGIC state at EL2
KVM: arm64: Minimise EL2's exposure of host VGIC state during world switch
KVM: arm64: Add host and hypervisor vCPU lookup primitives
KVM: arm64: Move PSCI helper functions to a shared header
KVM: arm64: Factor out reusable vCPU reset helpers
KVM: arm64: Make vcpu_{read,write}_sys_reg available to HYP code
KVM: arm64: Extract MPIDR computation into a shared header
KVM: arm64: selftests: Add a userspace watchpoint test
KVM: arm64: Flush external_mdscr_el1 to the pKVM hyp vCPU
...
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
We currently have two implementations of get_timer offset(), one
in arm_arch_timer.h, and another one in switch.h.
These two only differ by a pair of kern_hyp_va(), which seems a
pretty weak reason to open-code it.
Turn this function into a macro to avoid the include dependency hell
on kern_hyp_va(), and make it work correctly in all contexts.
Signed-off-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Link: https://patch.msgid.link/20260808085824.732659-2-smostafa@google.com
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Since NVHCR_EL2 represents the HCR_EL2 state of the EL1 guest, it
must be dealt with in some particular way:
- for a guest in hyp context (an L1 by definition), NVHCR_EL2 directly
reflects HCR_EL2 as read and written by the guest itself. It must
therefore be eagerly synced back with the emulation code which only
knows about HCR_EL2. This is unconditional if NV3 is available on
the host.
- For an L2 guest, NVHCR_EL2 is controlled by the L1 guest, and we
just context switch it like any other EL1 register. Yes, EL1, as
that's where this thing runs from the PoV of L1. This is conditioned
on the guest using NV3.
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-24-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
With FEAT_NV2P1, it is no longer necessary to trap CPTR_EL2 accesses
via CPACR_EL1, as CPACR_EL1.TCPAC is guaranteed to be stateful.
Prevent such trapping and context switch CPACTR_EL1 in NV contexts
when NV2P1 is present.
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://patch.msgid.link/20260730071022.296811-10-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
The way we merge the guest-provided HCRX_EL2 value with the host's
is bonkers. We try to make it look like the FGT registers by using
positive and negative polarities for traps, but most of these bits
are not strictly about trapping, as they actively change the way
some architectural state is managed.
It would be far better to deal with these bits like we do for
HCR_EL2, by enumerating the list of bits we don't allow the guest
to override. This is simplified by the fact that HCRX_EL2 only
affects EL1, and not EL2.
Re-jig the HCRX_EL2 handling with a macro that list the bits excluded
from the merge (TMEA, PTTWI, EnIDCP128).
Reviewed-by: Yuan Yao <yaoyuan@linux.alibaba.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Reviewed-by: Joey Gouly <joey.gouly@arm.com>
Link: https://patch.msgid.link/20260730071022.296811-4-maz@kernel.org
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD
KVM/arm64 fixes for 7.2, take #1
- Fix an accounting buglet when reclaiming pages from a protected
guest
- Fix a bunch of architectural compliance issues when injecting a
synthesised exception, most of which were missing the PSTATE.IL bit
indicating a 32bit-wide instruction
- Another set of fixes addressing issues with translation of VNCR_EL2,
including corner cases where the guest point that register at a RO
page...
- Don't warn when trapping accesses to ZCR_EL2 from an L2 guest, as
that's not unexpected at all
- Address a bunch of races with LPI migration vs LPIs being disabled
- Fix a total howler of a bug combining FEAT_MOPS and NV, resulting in
exception returning in the wrong place...
- Coerce Fuad Tabba into a reviewer role, and may his Inbox catch
fire!
|
|
When the hypervisor tracing (CONFIG_NVHE_EL2_TRACING) is disabled, it
defines a static inline stub for trace_clock().
However, trace_clock() is already declared as an extern function in
linux/trace_clock.h which is pulled in EL2 compilation.
If the file <nvhe/clock.h> is included when CONFIG_NVHE_EL2_TRACING
is disabled (by including it manually in setup.c) it will cause:
In file included from arch/arm64/kvm/hyp/nvhe/setup.c:22:
./arch/arm64/kvm/hyp/include/nvhe/clock.h:14:19: error: static declaration of ‘trace_clock’ follows non-static declaration
14 | static inline u64 trace_clock(void) { return 0; }
| ^~~~~~~~~~~
on GCC and a linker error on LLVM (it seems to change the linkage to
global)
Although that is not a problem at the moment, as no other files
include <nvhe/clock.h>. That does not seem to be the intent of
this code and that will cause issues with more users as the SMMUv3
driver.
Signed-off-by: Mostafa Saleh <smostafa@google.com>
Reviewed-by: Fuad Tabba <fuad.tabba@linux.dev>
Tested-by: Fuad Tabba <fuad.tabba@linux.dev>
Reviewed-by: Vincent Donnefort <vdonnefort@google.com>
Link: https://patch.msgid.link/20260713141320.4065600-1-smostafa@google.com
Signed-off-by: Oliver Upton <oupton@kernel.org>
|
|
Pull kvm updates from Paolo Bonzini:
"arm64:
This is a bit of an odd merge window on the KVM/arm64 front. There
is absolutely no new feature in the pull request. It is purely
fixes, because it is simply becoming too hard to review new stuff
when so many AI-fuelled fixes hit the list.
- Significant cleanup of the vgic-v5 PPI support which was merged in
7.1. This makes the code more maintainable, and squashes a couple
of bugs in the meantime
- Set of fixes for the handling of the MMU in an NV context,
particularly VNCR-triggered faults. S1POE support is fixed as well
- Large set of pKVM fixes, mostly addressing recurring issues around
hypervisor tracking of donated pages in obscure cases where the
donation could fail and leave things in a bizarre state
- Fixes for the so-called "lazy vgic init", which resulted in
sleeping operations in non-preemptible sections. This turned out to
be far more invasive than initially expected..
- Reduce the overhead of L1/L2 context switch by not touching the FP
registers
- Fix the way non-implemented page sizes are dealt with when a guest
insist on using them for S2 translation
- The usual set of low-impact fixes and cleanups all over the map
Loongarch:
- On a request for lazy FPU load, load all FPU state that the VM
supports instead of enabling only the part (FPU, LSX or LASX) that
caused the FPU load request
- Some enhancements about interrupt injection
- Some bug fixes and other small changes
RISC-V:
- Batch G-stage TLB flushes for GPA range based page table updates
- Convert HGEI line management to fully per-HART
- Fix missing CSR dirty marking when FWFT state updated via ONE_REG
- Fix stale FWFT feature exposure to Guest/VM
- Speed up dirty logging write faults using MMU rwlock and atomic PTE
updates using cmpxchg() for permission-only changes
- Use flexible array for APLIC IRQ state
- Use kvm_slot_dirty_track_enabled() for logging enable check on a
memslot
- Avoid skipping valid pages in kvm_riscv_gstage_wp_range()
- Avoid skipping valid pages in kvm_riscv_gstage_unmap_range()
- Use endian-specific __lelong for NACL shared memory
S390:
- KVM_PRE_FAULT_MEMORY support
- Support for 2G hugepages
- Support for the ASTFLEIE 2 facility
- Support for fast inject using kvm_arch_set_irq_inatomic
- Fix potential leak of uninitialized bytes
- A few more misc gmap fixes
x86:
- Generic support for the more granular permissions allowed by EPT,
namely "read" (which was previously usurping the U bit) and
separate execution bits for kernel and userspace
- Do not assume that all page tables start with U=1/W=1/NX=0 at the
root, as AMD GMET needs to have U=0 at the root
- Introduce common assembly macros for use within Intel and AMD
vendor-specific vmentry code. This touches the SPEC_CTRL handling,
which is now entirely done in assembly for Intel (by reusing the
AMD code that already existed), and register save/restore which
uses some macro magic to compute the offsets in the struct. Both of
these are preparatory changes for upcoming APX support
- Clean up KVM's register tracking and storage, primarily to prepare
for APX support, which expands the maximum number of GPRs from 16
to 32
- Keep a single copy of the PDPTRs rather than two, since
architecturally there is just one
- Handle EXIT_FASTPATH_EXIT_USERSPACE in vendor code to ensure vendor
code gets a chance to handle things like reaping the PML buffer
- Update KVM's view of PV async enabling if and only if the MSR write
fully succeeds
- Fix a variety of issues where the emulator doesn't honor
guest-debug state, and clean up related code along the way
- Synthesize EPT Violation and #NPF "error code" bits when injecting
faults into L1 that didn't originate in hardware (in which case the
VMCS/VMCB doesn't hold relevant information)
- Add support for virtualizing (well, emulating) AMD's flavor of
CPL>0 CPUID faulting
- Clean up the GPR APIs so that KVM's use of "raw" is consistent, and
fix a variety of minor bugs along the way
- Fix an OOB memory access due to not checking the VP ID when
handling a Hyper-V PV TLB flush for L2
- Fix a bug in the mediated PMU's handling of fixed counters that
allowed the guest to bypass the PMU event filter
- Allow userspace to return EAGAIN when handling SNP and TDX
hypercalls, so the KVM can forward a "retry" status code to the
guest, and reserve all unused error codes for future usage
- Overhaul the TDP MMU => S-EPT code to move as much S-EPT specific
logic as possible into the TDX code, and to funnel (almost) all
S-EPT updates into a single chokepoint. The motivation is largely
to prepare for upcoming Dynamic PAMT support, but the cleanups are
nice to have on their own
- Plug a hole in shadow page table handling, where KVM fails to
recursively zap nested EPT/NPT shadow page tables when the nested
hypervisor tears down its own EPT/NPT page tables from the bottom
up
x86 (Intel):
- Support for nested MBEC (Mode-Based Execute Control), see above in
the generic section; also run with MBEC enabled even for non-nested
mode
- Use the kernel's "enum pg_level" in the TDX APIs instead of the
TDX-Module's level definitions (which are 0-based)
- Rework the TDX memory APIs to not require/assume that guest memory
is backed by "struct page" (in prepartion for guest_memfd hugepage
support)
- Fix a largely benign bug where KVM TDX would incorrectly state it
could emulate several x2APIC MSRs
- Use the "safe" WRMSR API when proxying LBR MSR writes as the
to-be-written value is guest controlled and completely unvalidated
x86 (AMD):
- Support for nested GMET (Guest Mode Execution Trap), see above in
the generic section; also run with GMET enabled even for non-nested
mode
- Fixes and minor cleanups to GHCB handling, on top of the earlier
work already merged into 7.1-rc
- Ensure KVM's copy of CR0 and CR3 are up-to-date prior to invoking
fastpath handlers
- Add support for virtualizing gPAT (KVM previously just used L1's
PAT when running L2)
- Fix goofs where KVM mishandles side effects (e.g. single-step and
PMC updates) when emulating VMRUN
- Fix a variety of bugs in AVIC's handling of x2APIC MSR
interception, most notably where KVM didn't disable interception of
IRR, ISR, and TMR regs
- Add support for virtualizing Host-Only/Guest-Only bits in the
mediated PMU
- Don't advertise support for unusable VM types, and account for VM
types that are disabled by firmware, e.g. to mitigate security
vulnerabilities
- Rewrite the SEV {en,de}crypt debug ioctls as they were riddle with
bugs and unnecessarily complicated, and add comprehensive tests
- Clean up and deduplicate the SEV page pinning code
- Fix minor goofs related to writing back CPUID information after
firmware rejects a CPUID page for an SNP vCPU
Generic:
- Rename invalidate_begin() to invalidate_start() throughout KVM to
follow the kernel's nomenclature, e.g. for mmu_notifiers
- Use guard() to cleanup up various KVM+VFIO flows
- Minor cleanups
guest_memfd:
- Return -EEXIST instead of -EINVAL if userspace attempts to bind a
gmem range to multiple memslots, and fix the test that was supposed
to ensure KVM returns -EEXIST
- Treat memslot binding offsets and sizes as unsigned values to fix a
bug where KVM interprets a large "offset + size" as a negative
value and allows a nonsensical offset
- Use the inode number instead of the page offset for the NUMA
interleaving index to fix a bug where the effective index would
jump by two for consecutive pages (the caller also adds in the page
offset)
Selftests:
- Randomize the dirty log test's delay when reaping the bitmap on the
first pass, as always waiting only 1ms hid a KVM RISC-V bug as the
test reaped the bitmap before KVM could build up enough state to
hit the bug
- A pile of one-off fixes and cleanups"
* tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (326 commits)
KVM: x86/mmu: Ensure hugepage is in by slot before checking max mapping level
KVM: x86: Fix shadow paging use-after-free due to unexpected role
KVM: s390: Introducing kvm_arch_set_irq_inatomic fast inject
KVM: s390: Enable adapter_indicators_set to use mapped pages
KVM: s390: Add map/unmap ioctl and clean mappings post-guest
riscv: kvm: Use endian-specific __lelong for NACL shared memory
KVM: selftests: access_tracking_perf_test: bump number of NUMA nodes to 32
KVM: s390: vsie: Implement ASTFLEIE facility 2
KVM: s390: vsie: Refactor handle_stfle
s390/sclp: Detect ASTFLEIE 2 facility
KVM: s390: Minor refactor of base/ext facility lists
KVM: x86/mmu: move pdptrs out of the MMU
KVM: x86: check that kvm_handle_invpcid is only invoked with shadow paging
KVM: nSVM: invalidate cached PDPTRs across nested NPT transitions
KVM: nVMX: remove unnecessary code in prepare_vmcs02_rare
KVM: x86: remove nested_mmu from mmu_is_nested()
KVM: arm64: vgic-its: Make ABI commit helpers return void
KVM: s390: Initialize KVM_S390_GET_CMMA_BITS memory
LoongArch: KVM: Add missing slots_lock for device register/unregister
LoongArch: KVM: Validate irqchip index in irqfd routing
...
|
|
It is entirely possible for a guest to write to the ZCR_EL2 sysreg alias
while in a nested context, as it is expected if FEAT_NV2 is advertised
to the L1 hypervisor.
Get rid of the bogus WARN which, since the hyp vectors were installed at
this point, has the effect of a hyp_panic...
Cc: stable@vger.kernel.org
Fixes: 0cfc85b8f5cf ("KVM: arm64: nv: Load guest FP state for ZCR_EL2 trap")
Signed-off-by: Oliver Upton <oupton@kernel.org>
Link: https://patch.msgid.link/20260615051324.830045-1-oupton@kernel.org
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
kvm_hyp_handle_mops() resets the single-step state machine as part of
rewinding state for a MOPS exception by modifying vcpu_cpsr() and
writing the result directly into hardware.
In the case of nested virtualization, vcpu_cpsr() is a synthetic value
such that the rest of KVM can deal with vEL2 cleanly. That means the
value requires translation before being written into hardware, which is
unfortunately missing from the MOPS handler.
Fix it by directly modifying SPSR_EL2 and avoiding the synthetic state
altogether, which will be resynchronized on the next 'full' exit back
to KVM.
Fixes: 2de451a329cf ("KVM: arm64: Add handler for MOPS exceptions")
Reported-by: Zhong Wang <wangzhong.c0ss4ck@bytedance.com>
Reported-by: Xuanqing Shi <shixuanqing.11@bytedance.com>
Link: https://lore.kernel.org/all/ajE4lHQevXNHpl1M@Air.local/
Cc: stable@vger.kernel.org
Signed-off-by: Weiming Shi <bestswngs@gmail.com>
Link: https://patch.msgid.link/20260617040820.2194831-2-bestswngs@gmail.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
gitolite.kernel.org:pub/scm/linux/kernel/git/arm64/linux
Pull arm64 updates from Will Deacon:
"It feels like the new world of AI tooling has slowed us down a little
on the feature side when compared to the fixes side. The extra rounds
of Sashiko review have also pushed a few things out until next time.
Still, there's some good foundational stuff here for the fpsimd code
and hardening work towards removing the predictable linear alias of
the kernel image.
CPU errata handling:
- Extend CnP disabling workaround to HiSilicon HIP09 hardware.
- Work around eternally broken broadcast TLB invalidation on more
CPUs.
- Documentation and code cleanups.
CPU features:
- Add new hwcaps for the 2025 dpISA extensions.
Floating point / SVE / SME:
- Significant cleanup to the low-level state management code in the
core architecture code and KVM.
- Use correct register widths during SVE/SME save/restore assembly.
- Expose SVE/SME save/restore memory accesses to sanitisers.
Memory management:
- Preparatory work for unmapping the kernel data and bss sections
from the linear map.
Miscellaneous:
- Inline DAIF manipulation helpers so they can be used safely from
non-instrumentable code.
- Fix handling of the 'nosmp' cmdline option to avoid marking
secondary cores as "possible".
MPAM:
- Add support for v0.1 of the MPAM architecture.
Perf:
- Update HiSilicon PMU MAINTAINERS entry.
- Fix event encodings for the DVM node in the CMN driver.
Selftests:
- Extend sigframe tests to cover POE context.
- Add coverage for the newly added 2025 dpISA hwcaps.
System registers:
- Add new registers and ESR encodings for the HDBSS feature.
Plus minor fixes and cleanups across the board"
* tag 'arm64-upstream' of gitolite.kernel.org:pub/scm/linux/kernel/git/arm64/linux: (73 commits)
arm64: errata: Mitigate TLBI errata on Microsoft Azure Cobalt 100 CPU
arm64: errata: Mitigate TLBI errata on NVIDIA Olympus CPU
arm64: errata: Mitigate TLBI errata on various Arm CPUs
arm64: cputype: Add C1-Premium definitions
arm64: cputype: Add C1-Ultra definitions
Revert "arm64: mm: Unmap kernel data/bss entirely from the linear map"
Revert "arm64: mm: Defer remap of linear alias of data/bss"
arm64: arch_timer: reuse arch_timer_read_cnt{p,v}ct_el0() helpers
arm64/mm: Rename ptdesc_t
arm64: mm: Defer remap of linear alias of data/bss
KVM: arm64: Omit tag sync on stage-2 mappings of the zero page
arm64: Avoid double evaluation of __ptep_get()
kasan: Move generic KASAN page tables out of BSS too
arm64: Rename page table BSS section to .bss..pgtbl
arm64: patching: replace min_t with min in __text_poke
perf/arm-cmn: Fix DVM node events
arm64: fpsimd: Remove <asm/fpsimdmacros.h>
arm64: fpsimd: Move SME save/restore inline
arm64: fpsimd: Move sve_flush_live() inline
arm64: fpsimd: Move SVE save/restore inline
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/kvmarm/kvmarm into HEAD
KVM/arm64 updates for 7.2
* New features:
- None. Zilch. Nada. Que dalle.
* Fixes and other improvements:
- Significant cleanup of the vgic-v5 PPI support which was merged in
7.1. This makes the code more maintainable, and squashes a couple
of bugs in the meantime.
- Set of fixes for the handling of the MMU in an NV context,
particularly VNCR-triggered faults. S1POE support is fixed
as well.
- Large set of pKVM fixes, mostly addressing recurring issues
around hypervisor tracking of donated pages in obscure cases
where the donation could fail and leave things in a bizarre
state.
- Fixes for the so-called "lazy vgic init", which resulted in
sleeping operations in non-preemptible sections. This turned
out to be far more invasive than initially expected...
- Reduce the overhead of L1/L2 context switch by not touching
the FP registers.
- Fix the way non-implemented page sizes are dealt with when
a guest insist on using them for S2 translation.
- The usual set of low-impact fixes and cleanups all over the map.
|
|
* kvm-arm64/nv-mmu-7.2:
: .
: Assorted collection of fixes for NV MMU bugs
:
: - Correctly plug AT S1E1A handling in the emulation backend
:
: - Make CPTR_EL2.E0POE depend on FEAT_S1POE
:
: - Drop the reference on the page if the VNCR translation
: races with an MMU notifier
:
: - Correctly synthesise an SEA if a page table walk fails due
: to a guest error
:
: - Fully invalidate the VNCR TLB and fixmap when translating
: for a new VNCR
:
: - Restart S1 walk when the S2 walk fails due to a race condition
:
: - Correctly return -EAGAIN when a S1 walk fails
:
: - Fix block mapping validity check in stage-1 walker for 64kB pages
:
: - Fix potential NULL dereference when performing an EL2 TLBI targeting
: the VNCR page
:
: - Hold kvm->mmu_lock while initialising the vncr_tlb pointer
: .
KVM: arm64: nv: Hold kvm->mmu_lock while initialising vcpu->arch.vncr_tlb
KVM: arm64: nv: Avoid dereferencing NULL VNCR pseudo-TLB
KVM: arm64: Fix block mapping validity check in stage-1 walker
KVM: arm64: nv: Restart stage-1 walk if stage-2 desc update fails
KVM: arm64: Restart instruction upon race in __kvm_at_s12()
KVM: arm64: nv: Inject SEA TTW when desc update can't write to GPA
KVM: arm64: nv: Fully update VNCR fixmap state in kvm_translate_vncr()
KVM: arm64: Don't leak PFN when kvm_translate_vncr() races MMU notifier
arm64: cpufeature: Expose ID_AA64ISAR2_EL1.ATS1A to KVM
KVM: arm64: Wire AT S1E1A in the system instruction handling table
KVM: arm64: Key CPTR_EL2.E0POE propagation on FEAT_S1POE
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
* kvm-arm64/misc-7.2:
: .
: - Check for a valid vcpu pointer upon deactivating traps when handling
: a HYP panic in VHE mode
:
: - Make the __deactivate_fgt() macro use its arguments instead of the
: surrounding context
:
: - Don't bother with initialising TPIDR_EL2 in the hyp stubs, as this
: is already taken care of in more obvious places
:
: - Drop the unused kvm_arch pointer passed to __load_stage2()
:
: - Return -EOPNOTSUPP when a hypercall fails for some reason, instead of
: returning whatever was in the result structure
:
: - Make the ITS ABI selection helpers return void, which avoids wondering
: about the nature of the return code (always 0)
: .
KVM: arm64: vgic-its: Make ABI commit helpers return void
KVM: arm64: Set a Linux errno on SMCCC error in kvm_call_hyp_nvhe()
KVM: arm64: Remove @arch from __load_stage2()
KVM: arm64: Don't populate TPIDR_EL2 in finalise_el2()
KVM: arm64: Fix __deactivate_fgt macro parameter typo
KVM: arm64: Guard against NULL vcpu on VHE hyp panic path
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
* kvm-arm64/pkvm-fixes-7.2:
: .
: Assorted pKVM fixes for 7.2:
:
: - Ensure that the vcpu memcache is filled in a number of cases (donate,
: share, selftest)
:
: - Fix vmemmap page order handling by resetting it when initialising the
: memory pool
:
: - Don't leak page references on failed memory donation
:
: - Add sanity-check for refcounted pages when donating/sharing pages
:
: - Clear __hyp_running_vcpu on state flush
:
: - Check LR upper bound against a trusted value
:
: - Assorted fixes for the host-side tracking of the pages shared with
: EL2 as a result of some Sashiko testing from Fuad
:
: - Correctly forward HCR_EL2.VSE from host to guest, so that protected
: guests can see SErrors
: .
KVM: arm64: Roll back partial shares on kvm_share_hyp() failure
KVM: arm64: Avoid host/hyp share desync on unshare hypercall failure
KVM: arm64: Free hyp-share tracking node when share hypercall fails
KVM: arm64: Flush HCR_EL2.VSE to deliver SErrors to pKVM guests
KVM: arm64: Bound used_lrs when flushing the pKVM hyp vCPU
KVM: arm64: Clear __hyp_running_vcpu when flushing the pKVM hyp vCPU
KVM: arm64: Pre-check vcpu memcache for host->guest donate
KVM: arm64: Pre-check vcpu memcache for host->guest share
KVM: arm64: Seed pkvm_ownership_selftest vcpu memcache
KVM: arm64: Add fail-safe for refcounted pages in __pkvm_hyp_donate_host
KVM: arm64: Fix __pkvm_init_vm error path
KVM: arm64: Reset page order in pKVM hyp_pool
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
CPTR_EL2.E0POE was being cleared in __deactivate_cptr_traps_vhe(), which meant
that any accesses to POR_EL0 from host EL0 would trap and be reported to
userspace as an Illegal instruction. This would happen after running any VM,
regardless if it used POE or not.
Signed-off-by: Joey Gouly <joey.gouly@arm.com>
Link: https://sashiko.dev/#/patchset/20260602155430.2088142-1-maz@kernel.org?part=1
Link: https://patch.msgid.link/20260604105434.2297268-1-joey.gouly@arm.com
Signed-off-by: Marc Zyngier <maz@kernel.org>
Cc: stable@vger,kernel.org
|
