| Age | Commit message (Collapse) | Author |
|---|
|
# Conflicts:
# drivers/gpu/drm/amd/amdkfd/kfd_migrate.c
# net/ceph/osd_client.c
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull x86 TDX updates from Dave Hansen:
"There are two high-level things going on here.
First, TDX host systems currently consume ~0.4% of physical memory for
their own memory metadata. This is all-or-nothing; you either pay the
0.4% or can not use TDX. A new TDX module feature (Dynamic PAMT)
allows this metadata to be allocated for 2MB of host memory at a time.
This, of course, saves precious DRAM when TDX utilization is low on a
host.
Second, the TDX ABI has fixed-size "quotes" that are passed around
during attestation operations. These 128k buffers are far too small
for post-quantum cryptography where the quotes grow substantially. The
TDX module now has the ability to enumerate the sizes of the quote
buffers. Plumb the new size enumeration into the kernel"
* tag 'x86_tdx_for_7.4-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
virt: tdx-guest: Make the Quote buffer size dynamic
x86/tdx: Add a helper to query maximum Quote size
virt: tdx-guest: Add a helper for the Quote buffer size
virt: tdx-guest: Calculate the Quote buffer size safely
virt: tdx-guest: Give the Quote buffer an explicit type
x86/tdx: Take the Quote buffer as a generic pointer
x86/tdx: Restrict attestation exports to the tdx-guest driver
virt: tdx-guest: Remove unused and confusing function argument
x86/virt/tdx: Formalize SEAMCALL leaf version encoding support
Documentation/x86: Add documentation for TDX's Dynamic PAMT
x86/virt/tdx: Enable Dynamic PAMT
KVM/TDX: Get/put PAMT pages when (un)mapping private memory
x86/virt/tdx: Add APIs to support Dynamic PAMT ops from KVM's fault path
KVM/TDX: Allocate PAMT memory for TD and vCPU control structures
x86/virt/tdx: Handle multiple callers in tdx_pamt_get/put()
x86/virt/tdx: Allocate refcounts for Dynamic PAMT memory
x86/virt/tdx: Add __tdx_pamt_get/put() helpers
x86/virt/tdx: Allocate page bitmap for Dynamic PAMT
x86/virt/tdx: Simplify PAMT layout calculation
|
|
The TDX architecture includes a syscall-like ABI for OSes to communicate
with SEAM mode software. This ABI has the concept of a "leaf". Each leaf
is roughly analogous to a Linux syscall: it has a set of register
arguments and does one logical thing like adding a page of memory to a
VM or running a VM. The TDX architecture refers to this interface
function as a "SEAMCALL leaf".
Just like syscalls, the TDX architecture wants to evolve the ABI to
extend functionality while keeping compatibility. But unlike syscalls,
which do this by picking a totally new syscall number, the ABI encodes
the "version" number directly into the bits of a register argument. So
instead of openatN being whatever the next free number is, the SEAMCALL
leaf number and version number are encoded into certain bits in how the
RAX register is defined in the ABI.
In Linux, several seamcall*() wrappers have been introduced to invoke
SEAMCALL leafs. They all take a u64 "fn" argument for the leaf number
which eventually gets set in the register. As above, the version number
lives in the same register as the leaf number. So callers that want to
select a specific version of the leaf, can jam it in the right place in
the register by passing it in the "fn" argument. Today only the caller
of TDH.VP.INIT does this hack, but future kernel changes will need to
select versions for more SEAMCALL leafs. So a less hacky solution is
needed.
Explicitly define the arguments for seamcall*() wrappers:
- Add a build-time assertion to ensure "fn" only contains valid
SEAMCALL leaf number bits. Note the P-SEAMLDR selector bit (bit 63)
is part of the leaf number for SEAM loader calls, so it is allowed.
This also means the "fn" can't be a narrower type, such as u16, to
exclude unrelated bits.
- Add a "version" field in struct tdx_module_args [1], so most
existing callers get a default "version == 0" behavior without code
churn. Update the TDH.VP.INIT caller to specify the version
descriptively.
Encode the leaf number and tdx_module_args.version into RAX in assembly,
because this is the place where the "fn" and struct tdx_module_args
fields are marshaled into registers.
AI was used under supervision to review code and workshop logs. In
particular, it helped evaluate the assembly changes.
Signed-off-by: Xu Yilun <yilun.xu@linux.intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Nikolay Borisov <nik.borisov@suse.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Reviewed-by: Kishen Maloor <kishen.maloor@intel.com>
Link: https://lore.kernel.org/kvm/4f4b0f29-424b-45ed-8cfd-c77da2ea390f@intel.com/ # [1]
Link: https://patch.msgid.link/20260921-seamcall-version-v7-1-cf05fe76b467@linux.intel.com
|
|
The RMPOPT table is a per-CPU table which indicates whether 1GB regions
of physical memory are entirely hypervisor-owned.
When performing host memory accesses in hypervisor mode as well as
non-SNP guest mode, the processor may consult the RMPOPT table to
potentially skip an RMP access and improve performance.
Normal guest events disable RMP optimizations: pages are converted from
shared to private as SNP guests are launched, and large pages are split
and collapsed during guest operation -- both disable the RMPOPT
optimizations for the affected 1GB regions.
When guests are torn down, their pages are converted back to shared, so those
regions may become eligible for RMP optimizations again. Without some
intervention, all those optimizations would eventually be lost, so optimize
all of physical memory on SNP guest teardown.
snp_enable_rmpopt() performs the re-optimization after a delay, which gets
reset on each call. This way multiple guest terminations are batched into
a single pass: the re-optimization runs 10 seconds after the *last*
termination rather than after the first. mod_delayed_work() also re-queues
work that is already in-flight, so a re-scan request during an active scan is
not silently dropped.
Guest teardown is currently the only event that returns guest memory to
hypervisor ownership: SNP guests do not support ballooning or memory hotplug,
so pages freed during a guest's lifetime remain guest-owned. It is therefore
the only point at which memory becomes eligible for RMP re-optimization, which
is why re-optimization is driven by guest teardown rather than by a periodic
scan.
[ bp: Massage commit message. ]
Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Ackerley Tng <ackerleytng@google.com>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://patch.msgid.link/7a15a880ea6b6d53b3702def3b3434307656afd7.1789594774.git.ashish.kalra@amd.com
|
|
When SNP is enabled, all writes to memory are checked to ensure memory
integrity. This imposes performance overhead on the whole system.
RMPOPT is a new instruction that minimizes the performance overhead of
RMP checks on the hypervisor and on non-SNP guests by allowing such checks to
be skipped for 1GB regions of memory that are known not to contain any SNP
guest memory.
Add support for performing RMP optimizations asynchronously using a dedicated
per-CPU workqueue. Shortly after SNP initialization, run an optimization pass
over all physical memory.
As SNP guests are launched, RMPUPDATE assigns their private pages to
guest-owned state; when such a page falls within an optimized 1GB region, the
hardware clears that region's RMPOPT optimization and RMP checks resume there
to protect the guest memory.
Since launching SNP guests clears these optimizations, perform them again
asynchronously using the dedicated workqueue.
[ bp:
- Massage commit message and comments
- simplify code
- redo some of the logic ]
Suggested-by: Thomas Lendacky <thomas.lendacky@amd.com>
Suggested-by: Dave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://patch.msgid.link/daa513fbe8f0672b00bc5e026e4399244be6dc72.1789594774.git.ashish.kalra@amd.com
|
|
The new RMPOPT instruction helps manage per-CPU RMP optimization
structures inside the CPU. It takes a 1GB-aligned physical address and
either returns the status of the optimizations or tries to enable the
optimizations.
Initialize the per-CPU RMPOPT table base to the starting physical
address. This enables RMP optimization for up to 2 TB of system RAM on
all CPUs because this is the maximum the RMPOPT tables support.
The RMPOPT_BASE MSR can only be written after SNP is enabled, so the
enabling runs from the ccp SNP init path (and again on guest teardown)
rather than from an initcall. This is also in line with the intention to
not enable SNP by default but enable it on demand, when the ccp module
is loaded.
RMPOPT_BASE is programmed on all primary threads. RMPOPT_EN cannot be
cleared while SNP is enabled, and CPU hotplug is disabled while SNP is
active, so once programmed the MSRs stay set on all CPUs until SNP is
disabled. A set RMPOPT_EN on the local CPU therefore means the
programming has already been done and thus it can be skipped.
[ bp:
- Massage commit message
- move enabling logic into the commit message
- zap unnecessary comments
- cleanup ]
Suggested-by: Thomas Lendacky <thomas.lendacky@amd.com>
Suggested-by: Dave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Link: https://patch.msgid.link/7fdf0f5b1be4b561c884c5200d6606a947cbc09d.1789594774.git.ashish.kalra@amd.com
|
|
While SNP is active, every memory write is checked against the RMP to
protect SNP guest memory. A core performs these RMP checks only once
SNP has been initialized via SNP_INIT and the SNP-enable bit in SYSCFG is
set on that core; the firmware requires the SNP-enable bit to be set on
every present CPU before SNP initialization.
A core that is not SNP-enabled and not SNP-initialized performs no RMP
checks at all, so there is no valid configuration with SNP active and any
CPU exempt from RMP checks.
The firmware determines which CPUs are present from the processor and the
BIOS/UEFI configuration (e.g. SMT disabled in the BIOS) and enumerates
them at SNP init; it is not aware of the OS bringing CPUs online or
offline afterwards.
SNP_INIT fails unless SnpEn is set on all CPUs, so a CPU that is offline
when SNP_INIT is issued, does not have SnpEn set, SNP_INIT fails, and
there can be no SNP guest memory. OS CPU hotplug can thus diverge from
the firmware's expectations and break SNP.
Tie CPU hotplug to the SNP-enable bit: disable it in snp_prepare() before
SNP is enabled, and re-enable it in snp_shutdown() once the firmware has
disabled SNP.
If snp_prepare() fails before enabling SNP it re-enables hotplug itself;
once SNP is enabled hotplug stays disabled, including across a failed
SNP_INIT and across the legacy SNP_SHUTDOWN_EX path, both of which leave
SNP enabled.
A kexec target that boots with SNP already enabled, disables hotplug once
in snp_rmptable_init(), since snp_prepare() bails when SNP is already
enabled.
With CPU hotplug now disabled while SNP is active, the online CPU mask is
stable, so the cpus_read_lock() previously taken in snp_prepare() to
iterate it is redundant. Drop cpus_read_lock()/cpus_read_unlock() here.
Suggested-by: Thomas Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Link: https://patch.msgid.link/cd6962a88d691a020c2bf40fe77a6a3f3852fdfc.1789594774.git.ashish.kalra@amd.com
|
|
The Physical Address Metadata Table (PAMT) holds TDX metadata for
physical memory and must be allocated by the kernel during TDX module
initialization. Dynamic PAMT (DPAMT) is a TDX module feature that can
reduce this memory use by allocating part of the PAMT dynamically.
DPAMT's memory savings are significant enough to make it a good default
configuration. So turn it on whenever the TDX module reports support.
There is a TDX module bug where on unusual BIOS keyid configurations, the
TDX module reports DPAMT as supported even when it can't handle it. Expect
this not to be an issue in practice and don't attempt to work around it on
the kernel side.
The feature increases the kernel size by 2KB (when TDX is configured in the
build).
Based on a patch originally by Kiryl Shutsemau.
Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Vishal Annapurve <vannapurve@google.com>
Acked-by: Sohil Mehta <sohil.mehta@intel.com>
Tested-by: Hongyu Ning <hongyu.ning@linux.intel.com>
Link: https://patch.msgid.link/20260904215841.303070-10-rick.p.edgecombe@intel.com
|
|
Add Dynamic PAMT support to KVM's S-EPT MMU by "getting" a PAMT page when
adding guest memory (PAGE.ADD or PAGE.AUG), and "putting" the page when
removing guest memory (PAGE.REMOVE).
To access the per-vCPU PAMT caches without plumbing @vcpu throughout the
TDP MMU, begrudgingly use kvm_get_running_vcpu() to get the vCPU, and bug
the VM if KVM attempts to set an S-EPT leaf without an active vCPU. KVM
only supports creating _new_ mappings in page (pre)fault paths, all of
which require an active vCPU.
The PAMT memory holds metadata for TDX protected memory. With Dynamic
PAMT, PAMT_4K is allocated on demand. The kernel supplies the TDX module
with a few pages that cover 2MB of host physical memory.
Releases are balanced via tdx_pamt_put(): every control-page free goes
through tdx_free_control_page(), and guest data pages are put directly on
the successful tdh_mem_page_remove() path and in the
tdx_mem_page_add/aug() error path.
Co-developed-by: Sean Christopherson <seanjc@google.com>
[rick: enhance log, reviewing, rebase, with help from AI tooling]
Co-developed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Vishal Annapurve <vannapurve@google.com>
Acked-by: Sohil Mehta <sohil.mehta@intel.com>
Acked-by: Sean Christopherson <seanjc@google.com>
Tested-by: Hongyu Ning <hongyu.ning@linux.intel.com>
Link: https://patch.msgid.link/20260904215841.303070-9-rick.p.edgecombe@intel.com
|
|
When handling an EPT violation, KVM holds a spinlock while manipulating
the EPT. Before entering the spinlock it doesn't know how many EPT page
tables will need to be installed or whether a huge page will be used. For
this reason it allocates a worst case number of page tables that it might
need as part of servicing the EPT violation.
Under Dynamic PAMT (DPAMT) these pre-allocated pages will potentially need
to have DPAMT backing pages installed for them. KVM already has helpers to
manage topping up page caches before taking the MMU lock, but they cannot
be passed from KVM to arch/x86 code.
The problem of how and when to install the DPAMT backing pages for the
pages given to the TDX module during the fault path has had a lot of
design attempts.
- Extracting KVM's MMU caches requires too much inlined code added to
headers.
- A few varieties of installing DPAMT backing when allocating the
S-EPT page tables. (see links)
- Using mempool_t to transfer the pages between KVM and arch/x86 doesn't
work because the component is designed more around maintaining a pool
of pages, rather than topping up a continually drained cache.
So don't do these as they all had various problems. Instead just create a
small simple data structure to use for handing a pre-allocated list of
pages between KVM and arch/x86 code. Model this on KVM's existing MMU
memory caches.
Add a tdx_pamt_cache arg to tdx_pamt_get() so it can draw pages from a
cache when needed. Not all DPAMT page installations will happen under
spinlock, for example TD and vCPU scoped control pages. So have
tdx_pamt_get() maintain the existing behavior of allocating from the page
allocator when NULL is passed for the struct tdx_pamt_cache arg. This
prevents excess allocations for cases where it can be avoided.
Export the new helpers for KVM.
AI was used under supervision to review code and workshop logs.
Co-developed-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Reviewed-by: Chao Gao <chao.gao@intel.com>
Reviewed-by: Yan Zhao <yan.y.zhao@intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Nikolay Borisov <nik.borisov@suse.com>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Acked-by: Sohil Mehta <sohil.mehta@intel.com>
Tested-by: Hongyu Ning <hongyu.ning@linux.intel.com>
Link: https://lore.kernel.org/kvm/aXENNKjAKTM9UJNH@google.com/
Link: https://lore.kernel.org/kvm/20260129011517.3545883-20-seanjc@google.com/
Link: https://lore.kernel.org/kvm/aYW5CbUvZrLogsWF@yzhao56-desk.sh.intel.com/
Link: https://patch.msgid.link/20260904215841.303070-8-rick.p.edgecombe@intel.com
|
|
__tdx_pamt_get()/__tdx_pamt_put() unconditionally add or remove Dynamic
PAMT (DPAMT) backing for the 2MB region covering the passed page. However,
multiple callers can add or remove 4KB pages that fall within the same 2MB
region and in that scenario only a single PAMT entry is required.
Make the helpers handle only adding/removing DPAMT backing when required,
by refcounting each 2MB range. Gate the actual DPAMT add and remove on
refcount transitions (0->1 and 1->0). Serialize the refcount check and
SEAMCALL with a global spinlock so the read-decide-act sequence is atomic.
This also avoids TDX module BUSY errors, as the DPAMT add and remove
SEAMCALLs take internal TDX module locks for the 2MB ranges of the
specified PFN and the PAMT page pair PFNs. So simultaneous attempts on
the same 2MB ranges of the PFNs would otherwise encounter an error, which
would not be handleable in the put case.
The lock is global and heavyweight. Use simple conditional logic to keep
correctness obvious. This will be optimized in a later change.
The dpamt_refcounts[] are atomic_t's. They do not strictly need to be
because all access is protected by pamt_lock. The overhead of an atomic_t
in this situation is minuscule compared to the global lock. Leave the
atomic_t in place to enable future optimization with minimal churn.
Since the DPAMT helpers are broadly functional now, drop the "__"
to rename them tdx_pamt_get/put() and tdx_alloc/free_control_page().
Export them for use in KVM in subsequent changes.
AI was used under supervision to collect/apply feedback, split patches,
review code and workshop logs.
Based on a patch originally by Kiryl Shutsemau.
Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Reviewed-by: Chao Gao <chao.gao@intel.com>
Reviewed-by: Yan Zhao <yan.y.zhao@intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Nikolay Borisov <nik.borisov@suse.com>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Vishal Annapurve <vannapurve@google.com>
Acked-by: Sohil Mehta <sohil.mehta@intel.com>
Tested-by: Hongyu Ning <hongyu.ning@linux.intel.com>
Link: https://patch.msgid.link/20260904215841.303070-6-rick.p.edgecombe@intel.com
|
|
The PAMT memory holds metadata for all possible TDX protected memory. Each
physical address range is covered by PAMT entries at three levels (1GB,
2MB, 4KB). With Dynamic PAMT (DPAMT), the 4KB level of PAMT is allocated
on demand. The kernel supplies the TDX module with page pairs to store the
4KB level entries, which cover 2MB of host physical memory. The kernel must
provide this page pair before using pages from the range for TDX. If this
is not done, SEAMCALLs that give the pages to be protected by the TDX
module will fail.
Allocate reference counters for every 2MB range to track TDX memory usage.
This can be used to handle concurrent get/put callers, in order to
accurately determine when the dynamic 4KB level of DPAMT needs to be
allocated and when it can be freed.
This allocation will currently consume 2MB for every 1TB of address
space from 0 to max_pfn. The allocation size will depend on how the RAM is
physically laid out. In a worst case scenario where the entire 52 bit
address space is covered this would be 8GB. Then the DPAMT refcount
allocations could hypothetically cause the savings from DPAMT to go
negative on exotic platforms with sparse, small amounts of memory.
Future changes could reduce this refcount overhead to be only allocating
refcounts for physical ranges that contain memory that TDX can use.
However, this is left for future work.
AI was used under supervision to collect/apply feedback, review code and
workshop logs.
Based on a patch originally by Kiryl Shutsemau.
Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Reviewed-by: Chao Gao <chao.gao@intel.com>
Reviewed-by: Yan Zhao <yan.y.zhao@intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Vishal Annapurve <vannapurve@google.com>
Acked-by: Sohil Mehta <sohil.mehta@intel.com>
Tested-by: Hongyu Ning <hongyu.ning@linux.intel.com>
Link: https://patch.msgid.link/20260904215841.303070-5-rick.p.edgecombe@intel.com
|
|
Add helpers to use when allocating or preparing pages that are handed to
the TDX module for use as control/S-EPT pages, and thus need Dynamic PAMT
(DPAMT) adjustments.
The TDX module tracks some state for each page of physical memory that it
might use. It calls this state the PAMT. It includes separate state for
each page size a physical page could be utilized at within the TDX module
(1GB, 2MB, 4KB). In DPAMT, only the 4KB page size state is
allocated dynamically.
KVM will need to hand pages to the TDX module that it will use at 4KB
granularity. So these pages will need DPAMT backing added before they are
used by the TDX module, and removed afterwards.
Create __tdx_pamt_get/put() to handle installing DPAMT 4KB backing for
pages that are already allocated (such as KVM's use of S-EPT page tables
or guest private memory). Have them take a pfn instead of a struct page,
as future changes will want to use these helpers for guest pages which are
tracked by PFN.
Also add __tdx_alloc_control_page() and __tdx_free_control_page() to handle
both page allocation and DPAMT installation. Make them behave like
normal alloc/free functions where allocation can fail in the case of no
memory, but free (with any necessary DPAMT release) always
succeeds. Do this so they can eventually support the existing TDX flows
that require teardowns to succeed.
Don't CLFLUSH the DPAMT pages handed to the TDX module, as is done
for some other SEAMCALLs, as the TDX docs specify that this is only
needed on "TD private memory or TD control structure page".
Since these allocations will be easily user triggerable, account the
memory.
Only one pair of DPAMT pages is required for each 2MB-aligned physical
region, so multiple callers could trip over each other if they try to
manage the shared backing for two separate 4KB pages contained in one.
To build the logic up iteratively, don't do anything to handle pages
from the same 2MB region yet. Functionality to handle this will be added
before DPAMT can be enabled.
AI was used under supervision to collect/apply feedback, split patches,
review code and workshop logs.
Based on a patch originally by Kiryl Shutsemau.
Co-developed-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Chao Gao <chao.gao@intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Vishal Annapurve <vannapurve@google.com>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Acked-by: Sohil Mehta <sohil.mehta@intel.com>
Tested-by: Hongyu Ning <hongyu.ning@linux.intel.com>
Link: https://patch.msgid.link/20260904215841.303070-4-rick.p.edgecombe@intel.com
|
|
The TDX Physical Address Metadata Table (PAMT) holds data about the
physical memory used by TDX, and must be allocated by the kernel during
TDX module initialization.
The exact size of the required PAMT memory is determined by the TDX module
and may vary between TDX module versions. Currently it is approximately
0.4% of the system memory. This is a significant commitment, especially if
it is not known upfront whether the machine will run any TDX guests.
Each memory region that the TDX module might use needs three separate PAMT
allocations. One for each supported page size (1GB, 2MB, 4KB). The
TDX module supports a new feature designed to reduce PAMT overhead called
Dynamic PAMT (DPAMT). Under DPAMT the 4KB level is allocated dynamically
during runtime, while the 1GB and 2MB levels remain allocated on TDX
module initialization.
However, in the details, DPAMT still needs some smaller per 4KB page
scoped data (currently it is 1 bit per page). The TDX module exposes the
number of bits as a separate piece of metadata than the 4KB static
allocation for normal PAMT. Although the size is enumerated differently,
it is handed to the TDX module in the same way the 4KB page size PAMT
allocation is for normal PAMT.
Begin to implement DPAMT in the kernel by reading the bits-per-page needed
for DPAMT. Calculate the size needed for the bitmap, and use it instead of
the 4KB size determined for normal PAMT, in the case of DPAMT.
The existing metadata reading code was generated by a script, but the
current plan is to stop generating this code, as the script has continued
to need adjustments. So add manually written code and adjust the comment
about it being autogenerated to be more generic. Start to adopt a more
normal kernel code style without the ternary statements and if
conditionals assignments that the auto generated code has.
AI was used under supervision to collect/apply feedback, review code and
workshop logs.
Based on a patch originally by Kiryl Shutsemau.
Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Yan Zhao <yan.y.zhao@intel.com>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Vishal Annapurve <vannapurve@google.com>
Acked-by: Sohil Mehta <sohil.mehta@intel.com>
Tested-by: Hongyu Ning <hongyu.ning@linux.intel.com>
Link: https://patch.msgid.link/20260904215841.303070-3-rick.p.edgecombe@intel.com
|
|
For each memory region that the TDX module might use (called TDMR), three
separate traditional PAMT allocations are needed. There is one for each
supported page size (1GB, 2MB, 4KB). These store information on each page
in the TDMR. In Linux, they are allocated out of one physically contiguous
block, in order to more efficiently use some internal TDX module
bookkeeping resources. So some simple math is needed to break the single
large allocation into three smaller allocations for each page size.
There are some commonalities in the math needed to calculate the base and
size for each smaller allocation, and so an effort was made to share logic
across the three. Unfortunately doing this turned out unnaturally tortured,
with a loop iterating over the three page sizes, only to call into a
function with case statements for each page size. In the future Dynamic
PAMT will add more logic that is special to the 4KB page size, making the
benefit of the math sharing even more questionable.
Three is not a very high number, so get rid of the loop and just duplicate
the small calculation three times. In doing so, setup for future Dynamic
PAMT changes.
Since the loop that iterates over it is gone, further simplify the code by
dropping the array of intermediate size and base storage. Just store the
values to their final locations.
AI was used under supervision to collect/apply feedback, review code and
workshop logs.
Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Chao Gao <chao.gao@intel.com>
Reviewed-by: Yan Zhao <yan.y.zhao@intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Vishal Annapurve <vannapurve@google.com>
Acked-by: Sohil Mehta <sohil.mehta@intel.com>
Tested-by: Hongyu Ning <hongyu.ning@linux.intel.com>
Link: https://patch.msgid.link/20260904215841.303070-2-rick.p.edgecombe@intel.com
|
|
Nothing in suspend.h needs swap.h. However, many files indirectly
depend on some of swap.h's dependencies, so this is a large
cross-subsystem patch. Stats:
42 are missing includes of interrupt.h (the question of why swap.h
brings in interrupt.h remains unanswered).
10 missing includes of seq_file.h
5 missing includes of swap.h (obviously all files could have just added
swap.h, but I preferred to bring in a more minimal inclusion set)
3 missing includes of highmem.h
2 missing includes of device.h
2 missing includes of string_choices.h
1 missing include of cacheflush.h
1 missing include of dma-direction.h
1 missing include of kthread.h
1 missing include of pagemap.h
1 missing include of string_helpers.h
1 missing include of writeback.h
I tried to follow whatever conventions appeared to be in use for the
various subsystems I touched; for example I added string_choices.h to
drm_print.h instead of individually to each driver which used the
functions declared there.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
Pull MM updates from Andrew Morton:
- "mm: drop "sub" prefix from various places" (Dev Jain)
page->folio conversion and a naming cleanup
- "mm/kasan: remove redundant initialization for kasan_flag_write_only"
(Igor Putko)
KASAN cleanup work
- "mm/filemap: reduce unnecessary xarray lookups" (Chi Zhiling)
Small speedup in the pagecaache read code
- "mm/percpu: Fix possible NOFS/NOIO reclaim recursion" (Kaitao Cheng)
Improve the vmalloc code - mainly the avoidance of GFP_KERNEL
allocations when the caller asked for GFP_NOFS or GFP_NOIO
- "mm/kmemleak: avoid soft lockup when scanning task stacks" (Breno
Leitao)
Avoid a soft lockup watchdog trigger from the kmemleak scanning code
in extreme situations
- "mm/page_owner: misc cleanups" (Ye Liu)
Cleanups to the page_owner code. For some reason lots of people have
been working on the page_owner code this cycle.
- "mm: convert to walk_page_range_vma() to eliminate find_vma()"
(Kefeng Wang)
Simplify and accelerate the page walking library function
- "mm/migrate: preparatory cleanups for batch copy and offload"
(Shivank Garg)
Cleanups in the migration code
- "mm/page_owner: add per-fd filter infrastructure for print_mode and
NUMA filtering" (Zhen Ni)
Per-fd filtering to page_owner in order to reduce the sometimes vast
amount of output it can produce
- "mm: Refactor bootmem gigantic hugepage allocation" (Muchun Song)
Fixes and preparatory cleanups around bootmem HugeTLB handling,
sparse initialization ordering, and related vmemmap setup
- "mm/zsmalloc: reduce lock contention in zs_free()" (Wenchao Hao)
Reduce lock contention in zs_free(), which dominates the unmap path
under memory pressure on Android (LMK kills) and on x86 servers
running zswap-heavy workloads.
Up to 1.83x improvement in microbenchmarking.
- "move alloc_tag.c file under mm/" (Suren Baghdasaryan)
- "samples/damon: handle damon_{start,stop}() failures" (SJ Park)
Fix improper handling of damon_start(), damon_stop(), and
damon_call() failures across DAMON sample modules to prevent
potential memory leaks, operation disruptions and use-after-free
bugs
- "mm/damon/sysfs: kobject_del() directories that users can
create/remove" (SJ Park)
Fix delayed sysfs directory removal under DEBUG_KOBJECT_RELEASE
causeing creation failures due to duplicate directory names by adding
missing kobject_del() calls before creating new directories
- "mm: cleanup clear_not_present_full_ptes()" (David Hildenbrand)
Clean up the core pte handling code
- "selftests/damon: misc fixes for test bugs" (Kunwu Chan)
Fix several bugs in the DAMON selftests
- "selftests/damon: fix memcg_path staging handling" (Cheng Nie)
Fix a bug in _damon_sysfs.py for damos_filter memcg_path setup, and
add a test case for it in sysfs.py.
- "selftests/damon: test kdamond refresh_ms" (Ruslan Valiyev)
Selftest coverage for DAMON's refresh_ms sysfs feature by updating
the test control module and verifying that scheme stats update
automatically without manual intervention
- "mm/damon: five misc fixups" (Akinobu Mita)
Miscellaneous DAMON fixups.
- "mm/damon/core: detect internal variation above max_nr_regions/2"
(Jiayuan Chen)
Fix DAMON's region splitting behavior when region counts exceed half
the maximum budget by dynamically scaling down the split fraction as
the limit approaches, preventing large regions from staying un-split,
and add corresponding KUnit test coverage
- "mm: preparatory patches for PMD level swap entries" (Usama Arif)
Refactor and clean up PMD softleaf helpers, call sites, and
architecture flags to lay the groundwork for a follow-up series that
introduces PMD page table swap entries
- "mm/damon: update, optimize, and clean up doc, tests, and code" (SJ
Park)
Update DAMON design and ABI documentation, expands unit and selftest
coverage, optimize damon_commit_target_regions(), and clean up
recently added sysfs interface code for better readability
- "mm/vmpressure: reduce CPU, memory and code overhead on cgroup v2"
(Usama Arif)
Optimize vmpressure() by skipping unnecessary work on cgroup v2 for
userspace event notifications and refactor v1-only eventfd handling
into mm/memcontrol-v1.c to reduce memory overhead and code complexity
- "selftests/mm: refactor pkey helpers and fix mmap error handling"
(Hongfu Li)
Refactor pkeys shared tracing and assertion helpers into a common
file, unify protection key selftests to use consistent diagnostic
logging and assertions, and enforce standardized MAP_FAILED return
checks for mmap() calls across the tests
- "mm/damon: optimize out nr_accesses_bp" (SJ Park)
Replace the error-prone, continuously updated nr_accesses_bp field in
damon_region with an on-demand moving sum function, reducing
structure memory overhead and avoiding state corruption bugs
- "Open HugeTLB allocation routine for more generic use" (Ackerley Tng)
Decouple HugeTLB folio allocation from VMA dependencies by
introducing hugetlb_alloc_folio(), enabling subsystems like
guest_memfd to allocate HugeTLB folios without standard VMA
reservations or pseudo-VMAs
- "mm/damon: provide pseudo moving sum probe_hits" (SJ Park)
Integrate DAMON's probe_hits attribute counter into the pseudo moving
sum infrastructure, enabling real-time, online monitoring without
waiting for full aggregation intervals
- "mm: Some cleanups for page allocator APIs" (Brendan Jackman)
Simplify and refactor the page allocator entry points and flags by
unifying allocation paths, adding internal alloc_flags arguments, and
eliminating redundant __ prefixed alloc_pages variants.
- "Fix incorrect access of hugetlb pte entries" (Dev Jain)
Enforce the consistent use of huge_ptep_get() instead of ptep_get()
for HugeTLB entries and fixes an unaligned address issue in arm64's
huge_ptep_get() implementation
- "mm/damon: validate all parameters in the core" (SJ Park)
Consolidate parameter validation into the DAMON core specifically
within damon_start() and damon_commit_ctx() to centralize error
checking, eliminate caller-side redundant checks and to improve
maintenance efficiency
- "tools/mm/page_owner_sort: fix filtering and cleanup issues" (Yichong
Chen)
Rename is_need() to filter_record() for clearer return semantics, fix
per-record allocation memory leaks and bound output copies in
search_pattern() to address an existing buffer issue
- "memcg: bail out reclaim when memcg is dying" (Jiayuan Chen)
Mitigate a system-wide stall which occurs when a cgroup is removed
while one of its memory control files is doing synchronous reclaim
- "mm/memory-failure: add panic option for unrecoverable pages" (Breno
Leitao)
Introduce an opt-in vm.panic_on_unrecoverable_memory_failure sysctl
that immediately panics the kernel on unrecoverable memory errors in
kernel-owned pages to preserve error context and prevent delayed,
silent data corruption
- "mm/damon: refactor damon_{start,stop,commit}() for simple error
handling" (SJ Park)
Refactor the DAMON core API functions to guarantee that all contexts
are fully stopped when damon_start(), damon_stop(), or damon_commit()
fail, eliminating the need for complex and error-prone caller-side
cleanup code
- "Keep tail page private zero at free and folio split" (Zi Yan)
Add checks to ensure tail_page->private is zero when freeing compound
or high-order pages and when promoting tail pages during large folio
splits. By validating these fields at free and split time, it allows
the removal of redundant private field clearing inside
prep_compound_tail()
- "mm: drop redundant lru_add_drain in anon folio reuse paths" (Barry
Song)
Eliminate redundant lru_add_drain() calls in
wp_can_reuse_anon_folio() and do_swap_page() to reduce LRU lock
contention and system overhead
By validating folio refcounts against the LRU cache before draining
and removing unnecessary drains in the swap path, it achieves up to a
30.5% reduction in drain calls during heavy swap workloads
- "mm: clean up folio LRU and swap declarations" (Jianyue Wu)
Reorganize folio LRU and swap code by relocating page-cluster state
to mm/swap_state.c, renaming mm/swap.c to mm/folio.c, and moving
MM-internal reclaim declarations into mm/internal.h.
- "userfaultfd: working set tracking for VM guest memory" (Kiryl
Shutsemau)
Add userfaultfd support for tracking the working set of VM guest
memory, so a VMM can identify hot pages and reclaim cold ones to
tiered or remote storage
- "mm: remove CONFIG_HAVE_BOOTMEM_INFO_NODE (Part 2)" (David
Hildenbrand)
Remove the remaining pieces of CONFIG_HAVE_BOOTMEM_INFO_NODE,
performing some smaller cleanups around freeing of reserved vmemmap
pages on the way.
- "mm/damon: update probe hits for runtime parameter commits" (SJ Park)
Ensure that DAMON's probe_hits attribute counter is properly updated
when monitoring intervals are changed at runtime, matching the
behavior of nr_accesses. To achieve this, it refactors and renames
existing helper functions for shared use, applies the updates to
probe_hits, and handles edge cases in damon_probe_hits_mvsum() to
maintain measurement accuracy.
- "KSM: performance optimizations for rmap_walk_ksm" (xu xin)
Resolve a severe KSM reverse-mapping performance bottleneck where
thousands of split VMAs sharing a single anon_vma cause extended lock
contention.
By adding an interval-filtering check during the rmap walk, it
reduces worst-case anon_vma lock hold times from over 500ms down to
under 2ms, preventing application freezes and latency spikes under
memory pressure.
- "mm: split a couple of headers from internal.h" (Mike Rapoport)
Split declarations related to mm_init, memblock, vmalloc and sparse
into new headers
- "KSM: use linear_page_index in collect_procs_ksm()" (xu xin)
Apply the interval tree optimization from rmap_walk_ksm() to
collect_procs_ksm() to avoid iterating over non-matching VMAs during
KSM memory error handling.
It hoists loop-invariant address initialization and restricts the
anon_vma_interval_tree_foreach walk to a targeted page offset range,
reducing redundant checks and improving lookup efficiency.
- "selftests/mm: avoid false failures in hugetlb and KSM tests" (Sayali
Patil)
Fix issues in the hugetlb and KSM MM selftest categories that can
report failures when the prerequisites for the tests are not
satisfied
- "mm/damon: introduce data attributes only monitoring" (SJ Park)
Introduce attribute-weighted region management in DAMON, allowing
users to prioritize specific data attributes (such as page sizes or
cgroups) over or instead of access monitoring.
By assigning weights to attribute probes, DAMON can completely
disable access tracking and adjust monitoring regions based on
weighted probe-hit counters to optimize monitoring quality for
attribute-focused workloads.
- "mm/hmm: Add mmap lock-drop support for userfaultfd-backed mappings"
(Stanislav Kinsburskii)
Extend hmm_range_fault() to support userfaultfd-backed regions by
allowing the mmap lock to be dropped during fault handling via a new
hmm_range_fault_locked() helper.
By accepting a locked pointer and signaling retry status when lock
release occurs, it enables page fault resolution in userfaultfd
regions while preserving backward compatibility for existing callers.
- "mm: make VMA page offset handling more consistent" (Lorenzo Stoakes)
Clean up and standardize how vma->vm_pgoff is accessed and
manipulated across file-backed and anonymous mappings in the kernel
It introduces dedicated helper functions such as vma_start_pgoff(),
vma_end_pgoff(), vma_set_pgoff() and linear_page_delta() while
renaming rmap interval tree helpers to better reflect their
functionality.
These changes establish a cleaner foundation for future work that
will unify virtual page offset indexing for all anonymous and CoW'd
folios.
- "mm: handle device-private PMDs in walk callbacks" (Usama Arif)
Address kernel panics and state corruption caused by MM walk
callbacks reaching non-present device-private PMD swap entries
created during HMM migrations
It ensures that functions which acquire pmd_trans_huge_lock()
properly recognize device-private PMDs instead of assuming a present
THP or a standard migration entry.
- "mm/rmap: Refactor try_to_unmap_one" (Dev Jain)
Refactor try_to_unmap_one by modularizing Hugetlb,
anonymous-lazyfree, and anonymous-swapbacked logic into dedicated
functions, laying the structural groundwork for batched anonymous
large folio unmapping.
- "Docs/ABI/damon: sysfs ABI document fixes and additions" (Song Hu)
Fix typos and fills in missing entries in the DAMON sysfs ABI
document
- "dax/kmem: atomic whole-device hotplug via sysfs" (Gregory Price)
Introduce an atomic sysfs state attribute and supporting DAX/MM
infrastructure to prevent userland races when offlining and removing
entire memory regions
By adding an unplugged state alongside standard online modes, it
enables whole-device atomic hotplug control while preserving backward
compatibility.
- "mm: convert more vm_flags_t users to vma_flags_t" (Lorenzo Stoakes)
Continue transitioning the kernel from the deprecated vm_flags_t type
to vma_flags_t across core memory management infrastructure.
It replaces legacy type usage in core functions such as do_mmap(),
unmapped area allocation, mm->def_vma_flags, and VMA operations like
mlock, mprotect, and mremap.
- "Two small patches to clean up mm/mm_slot.h" (xu xin)
Refactor mm_slot.h by introducing mm_slot_remove() to unify duplicate
slot deletion sequences in khugepaged and KSM. It also adds code
documentation explaining why mm_slot_lookup and mm_slot_insert must
remain as preprocessor macros rather than static inline functions.
- "mm/damon/core: hide core-private struct fields" (SJ Park)
Clean up DAMON core structures by consistently marking internal-only
fields with private: comment tags to prevent improper direct access
from outer layers.
It enforces encapsulation across core structures including
damon_region, damon_target, and damon_ctx and updates DAMON_SYSFS to
interact through approved access APIs instead of exposing raw struct
members.
- "mm/damon: unurgent fixes for infinite loop, NULL de-ref and races"
(SJ Park)
Address potential infinite loops, NULL dereferences, and race
conditions identified in DAMON
It fixes an infinite loop triggered by extreme user configurations, a
NULL pointer dereference within unit tests and minor monitoring
accuracy degradation caused by subtle runtime races.
- "mm/page_alloc: fixes for free_pages_nolock() on RT/UP" (Brendan
Jackman)
Fix an NMI safety flaw in __free_frozen_pages() where freeing pages
on non-SMP or PREEMPT_RT kernels can bypass can_spin_trylock() checks
via non-PCP or isolated migration paths.
It also resolves potential kernel crashes and privilege escalation
risks triggered when BPF tracing runs in NMI context alongside memory
hotplug or large allocation frees.
- "mm/page_alloc: couple of followups for recent cleanups" (Brendan
Jackman)
Clean up and update page allocator nomenclature, documentation, and
debug assertions.
It aligns internal FPI_ flags with the public "nolock" naming
convention, removes outdated internal implementation details from
high-level page allocator comments, and eliminates obsolete
VM_BUG_ON() assertions in allocation paths.
- "mm/mseal: further cleanups" (Lorenzo Stoakes)
Refactor and simplify the mseal implementation by clarifying API
boundaries and removing unnecessary code complexity.
It replaces generic do_mseal() usage outside the syscall with a
dedicated mseal_mmap_page_zero() helper for MMAP_PAGE_ZERO,
eliminates mm_struct parameters to enforce that sealing applies only
to current->mm, and streamlines overall logic and comments with no
functional changes intended.
- "mm/vmscan: fix swappiness=max and clean up per-node proactive
reclaim" (Ridong Chen)
Resolve reclaim behavior bugs and clean up function parameters across
memory reclaim paths
It fixes swappiness=max in both standard reclaim and MGLRU so
unswappable anonymous memory no longer falls back to evicting page
cache, ensures reclaim_store() returns accurate error codes instead
of collapsing all failures into -EAGAIN, and removes the obsolete
gfp_mask parameter from __node_reclaim().
- "mm: mincore: misc cleanups" (Kefeng Wang)
Clean up and simplifies the mincore code. Most importantly, it
removes the historical special behavior that always reports VM_PFNMAP
pages as non-resident.
- "mm/huge_memory: drop dead split helper variants" (Kiryl Shutsemau)
Two trivial cleanups in the folio split API
- "mm/damon: fix uninitialized DAMOS field and kunit exec expectation
bugs" (SJ Park)
Resolve minor operational and testing bugs in DAMON identified by
Sashiko. It initializes the damos->last_applied field to prevent
occasional efficiency degradation and fixes invalid memory accesses
in DAMON KUnit tests during test failure handling.
- "cleanup for stable_page_flags()" (Jinjiang Tu)
Clean up and refactor stable_page_flags() used by /proc/kpageflags
without altering functionality.
It uses BIT_ULL() to prevent shift-overflow warnings on 64-bit flag
bits, converts folio-specific flag checks to standard folio_test_*()
helpers, and removes redundant CONFIG_PAGE_IDLE_FLAG handling.
- "Batch unmap of uffd-wp file folios" (Dev Jain)
Extend batched folio unmapping support to file folios within
userfaultfd write-protect (uffd-wp) VMAs by adding batching
capabilities to pte_install_uffd_wp_if_needed().
This removes special-case restrictions on uffd-wp VMAs in
try_to_unmap_one(), significantly simplifying the function's control
flow and complexity.
- "mm/early_ioremap: clarify and clean up early_ioremap_reset()"
(Sang-Heon Jeon)
Clarify and clean up the architecture-specific usage of
__late_set_fixmap() and __late_clear_fixmap() after
early_ioremap_reset()
It adds explicit documentation regarding when early_ioremap_reset()
must be called and removes redundant macro definitions and reset
calls in the RISC-V and ARM64 architectures.
- "mm: fix reclaim storms in defrag_mode" (Johannes Weiner)
Address severe performance regressions, swap storms, and spurious
OOMs caused by vm.defrag_mode=1 under high memory pressure in Meta
production
It updates the page allocator slowpath so non-movable allocation
requests actively trigger direct reclaim and direct compaction at
pageblock_order scale, allowing them to claim whole pageblocks rather
than spinning unproductively.
- "zram: lockmap tweaks" (Sebastian Siewior)
Optimize and fix lockdep tracking for zram devices by consolidating
per-entry lockmaps and isolate lock classes across multiple instances
This reduces memory overhead by replacing per-entry lockdep_map
instances with a single map per struct zram, and assigns a dynamic
lock_class_key to each instance to prevent false deadlock reports
when different zram devices are backed by distinct filesystems.
* tag 'mm-stable-2026-08-18-18-39' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (501 commits)
selftests/mm: thuge-gen: fix test_shmget() for PAGE_SIZE check
selftests/mm: unpoison pages in memory-failure teardown
mm/shmem: downgrade final i_blocks check in shmem_evict_inode() to pr_warn()
mm/khugepaged: replace mutex_lock/mutex_unlock usage with guard macro
mm/zsmalloc: fix release order of locks in zs_page_migrate()
Documentation: zram: remove sections numbering
ksm: stop iterating VMAs when ksm_test_exit returns true
mm: fold userfaultfd_rwp() to false without CONFIG_ARCH_HAS_PTE_PROTNONE
mm/migrate: report RCU-tasks quiescent states in migrate_pages_batch()
zram: use a custom key for each zram object
zram: move lockmap to be per-zram instead per table
selftests/mm: fix gup_longterm EINVAL error message
mm: page_alloc: fix non-movable reclaim storm in defrag_mode
mm: page_alloc: move capture_control to the page allocator
mm: compaction: support non-movable compaction for pageblock requests
mm: page_alloc: __GFP_FS lockdep annotation for direct compaction
hugetlb: evaluate subpool free state while locked
mm/damon: remove trailing semicolons after function definitions
mm/damon/ops-common: prevent migration fallback to non-target nodes
mm/damon: update outdated comment about DAMOS filter handling
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull x86 MSR updates from Ingo Molnar:
- Streamline the x86 MSR handling APIs along the 64-bit variants,
simplifying the interfaces.
Removal of the old APIs is planned for the next cycle, to reduce
churn & integration pain (Juergen Gross)
* tag 'x86-msr-2026-08-17' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (21 commits)
x86/mce: Work around build warning after MSR-interface switch
cpufreq: Stop using 32-bit MSR interfaces
x86/featctl: Stop using 32-bit MSR interfaces
KVM/x86: Stop using 32-bit MSR interfaces
x86/mtrr: Stop using 32-bit MSR interfaces
acpi: Stop using 32-bit MSR interfaces
powercap: Stop using 32-bit MSR interfaces
thermal/intel: Stop using 32-bit MSR interfaces
x86/olpc: Stop using 32-bit MSR interfaces
x86/hyperv: Stop using 32-bit MSR interfaces
hwmon: Stop using 32-bit MSR interfaces
EDAC: Stop using 32-bit MSR interfaces
x86/cpu: Stop using 32-bit MSR interfaces
x86/apic: Stop using 32-bit MSR interfaces
x86/resctrl: Stop using 32-bit MSR interfaces
x86/tsc: Stop using 32-bit MSR interfaces
x86/amd: Stop using 32-bit MSR interfaces
x86/pci: Stop using 32-bit MSR interfaces
x86/hygon: Stop using 32-bit MSR interfaces
x86/mce: Stop using 32-bit MSR interfaces
...
|
|
The difference between __alloc_pages_node() and alloc_pages_node() is
that the latter allows you to pass NUMA_NO_NODE.
The former is going away and the latter works fine here so switch over.
No functional change intended.
Link: https://lore.kernel.org/20260703-alloc-trylock-v5-10-c87b714e19d3@google.com
Signed-off-by: Brendan Jackman <jackmanb@google.com>
Reviewed-by: Suren Baghdasaryan <surenb@google.com>
Reviewed-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Cc: Thomas Gleixner <tglx@kernel.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Assisted-by: Gemini:unknown-version
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
|
|
The 32-bit MSR interfaces rdmsr(), wrmsr() and rdmsr_safe() are
planned to be removed. Use the related 64-bit variants instead.
No change in functionality intended.
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Kiryl Shutsemau <kas@kernel.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Rick Edgecombe <rick.p.edgecombe@intel.com>
Cc: Sean Christopherson <seanjc@google.com>
Cc: kvm@vger.kernel.org
Link: https://patch.msgid.link/20260629060526.3638272-10-jgross@suse.com
|
|
Revert
99cf1fb58e68 ("x86/virt/sev: Drop WBINVD before setting MSR_AMD64_SYSCFG_SNP_EN").
Section 8.8 of the SNP spec says:
Before invoking SNP_INIT_EX with INIT_RMP set to 1, software must ensure
that no CPUs contain dirty cache lines for the memory containing the RMP.
Cachelines can be moved from cache to cache in a dirty state. The
wbinvd_on_all_cpus() before SNP_INIT_EX flushes the caches for each CPU, but
if the IPIs for WBINVD race with this dirty cacheline movement, it is possible
that they may not get flushed, violating the firmware requirement.
Doing wbinvd_on_all_cpus() before setting SNPEn is safer since the RMP
table is not yet in use.
[ Heroically bisected by Srikanth. ]
[ bp: Massage commit message. ]
Fixes: 99cf1fb58e68 ("x86/virt/sev: Drop WBINVD before setting MSR_AMD64_SYSCFG_SNP_EN")
Reported-by: Srikanth Aithal <Srikanth.Aithal@amd.com>
Signed-off-by: Tycho Andersen (AMD) <tycho@kernel.org>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Tested-by: Srikanth Aithal <Srikanth.Aithal@amd.com>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
Cc: <stable@kernel.org>
Link: https://patch.msgid.link/20260707150033.2364758-1-tycho@kernel.org
|
|
Pull kvm updates from Paolo Bonzini:
"arm64:
This is a bit of an odd merge window on the KVM/arm64 front. There
is absolutely no new feature in the pull request. It is purely
fixes, because it is simply becoming too hard to review new stuff
when so many AI-fuelled fixes hit the list.
- Significant cleanup of the vgic-v5 PPI support which was merged in
7.1. This makes the code more maintainable, and squashes a couple
of bugs in the meantime
- Set of fixes for the handling of the MMU in an NV context,
particularly VNCR-triggered faults. S1POE support is fixed as well
- Large set of pKVM fixes, mostly addressing recurring issues around
hypervisor tracking of donated pages in obscure cases where the
donation could fail and leave things in a bizarre state
- Fixes for the so-called "lazy vgic init", which resulted in
sleeping operations in non-preemptible sections. This turned out to
be far more invasive than initially expected..
- Reduce the overhead of L1/L2 context switch by not touching the FP
registers
- Fix the way non-implemented page sizes are dealt with when a guest
insist on using them for S2 translation
- The usual set of low-impact fixes and cleanups all over the map
Loongarch:
- On a request for lazy FPU load, load all FPU state that the VM
supports instead of enabling only the part (FPU, LSX or LASX) that
caused the FPU load request
- Some enhancements about interrupt injection
- Some bug fixes and other small changes
RISC-V:
- Batch G-stage TLB flushes for GPA range based page table updates
- Convert HGEI line management to fully per-HART
- Fix missing CSR dirty marking when FWFT state updated via ONE_REG
- Fix stale FWFT feature exposure to Guest/VM
- Speed up dirty logging write faults using MMU rwlock and atomic PTE
updates using cmpxchg() for permission-only changes
- Use flexible array for APLIC IRQ state
- Use kvm_slot_dirty_track_enabled() for logging enable check on a
memslot
- Avoid skipping valid pages in kvm_riscv_gstage_wp_range()
- Avoid skipping valid pages in kvm_riscv_gstage_unmap_range()
- Use endian-specific __lelong for NACL shared memory
S390:
- KVM_PRE_FAULT_MEMORY support
- Support for 2G hugepages
- Support for the ASTFLEIE 2 facility
- Support for fast inject using kvm_arch_set_irq_inatomic
- Fix potential leak of uninitialized bytes
- A few more misc gmap fixes
x86:
- Generic support for the more granular permissions allowed by EPT,
namely "read" (which was previously usurping the U bit) and
separate execution bits for kernel and userspace
- Do not assume that all page tables start with U=1/W=1/NX=0 at the
root, as AMD GMET needs to have U=0 at the root
- Introduce common assembly macros for use within Intel and AMD
vendor-specific vmentry code. This touches the SPEC_CTRL handling,
which is now entirely done in assembly for Intel (by reusing the
AMD code that already existed), and register save/restore which
uses some macro magic to compute the offsets in the struct. Both of
these are preparatory changes for upcoming APX support
- Clean up KVM's register tracking and storage, primarily to prepare
for APX support, which expands the maximum number of GPRs from 16
to 32
- Keep a single copy of the PDPTRs rather than two, since
architecturally there is just one
- Handle EXIT_FASTPATH_EXIT_USERSPACE in vendor code to ensure vendor
code gets a chance to handle things like reaping the PML buffer
- Update KVM's view of PV async enabling if and only if the MSR write
fully succeeds
- Fix a variety of issues where the emulator doesn't honor
guest-debug state, and clean up related code along the way
- Synthesize EPT Violation and #NPF "error code" bits when injecting
faults into L1 that didn't originate in hardware (in which case the
VMCS/VMCB doesn't hold relevant information)
- Add support for virtualizing (well, emulating) AMD's flavor of
CPL>0 CPUID faulting
- Clean up the GPR APIs so that KVM's use of "raw" is consistent, and
fix a variety of minor bugs along the way
- Fix an OOB memory access due to not checking the VP ID when
handling a Hyper-V PV TLB flush for L2
- Fix a bug in the mediated PMU's handling of fixed counters that
allowed the guest to bypass the PMU event filter
- Allow userspace to return EAGAIN when handling SNP and TDX
hypercalls, so the KVM can forward a "retry" status code to the
guest, and reserve all unused error codes for future usage
- Overhaul the TDP MMU => S-EPT code to move as much S-EPT specific
logic as possible into the TDX code, and to funnel (almost) all
S-EPT updates into a single chokepoint. The motivation is largely
to prepare for upcoming Dynamic PAMT support, but the cleanups are
nice to have on their own
- Plug a hole in shadow page table handling, where KVM fails to
recursively zap nested EPT/NPT shadow page tables when the nested
hypervisor tears down its own EPT/NPT page tables from the bottom
up
x86 (Intel):
- Support for nested MBEC (Mode-Based Execute Control), see above in
the generic section; also run with MBEC enabled even for non-nested
mode
- Use the kernel's "enum pg_level" in the TDX APIs instead of the
TDX-Module's level definitions (which are 0-based)
- Rework the TDX memory APIs to not require/assume that guest memory
is backed by "struct page" (in prepartion for guest_memfd hugepage
support)
- Fix a largely benign bug where KVM TDX would incorrectly state it
could emulate several x2APIC MSRs
- Use the "safe" WRMSR API when proxying LBR MSR writes as the
to-be-written value is guest controlled and completely unvalidated
x86 (AMD):
- Support for nested GMET (Guest Mode Execution Trap), see above in
the generic section; also run with GMET enabled even for non-nested
mode
- Fixes and minor cleanups to GHCB handling, on top of the earlier
work already merged into 7.1-rc
- Ensure KVM's copy of CR0 and CR3 are up-to-date prior to invoking
fastpath handlers
- Add support for virtualizing gPAT (KVM previously just used L1's
PAT when running L2)
- Fix goofs where KVM mishandles side effects (e.g. single-step and
PMC updates) when emulating VMRUN
- Fix a variety of bugs in AVIC's handling of x2APIC MSR
interception, most notably where KVM didn't disable interception of
IRR, ISR, and TMR regs
- Add support for virtualizing Host-Only/Guest-Only bits in the
mediated PMU
- Don't advertise support for unusable VM types, and account for VM
types that are disabled by firmware, e.g. to mitigate security
vulnerabilities
- Rewrite the SEV {en,de}crypt debug ioctls as they were riddle with
bugs and unnecessarily complicated, and add comprehensive tests
- Clean up and deduplicate the SEV page pinning code
- Fix minor goofs related to writing back CPUID information after
firmware rejects a CPUID page for an SNP vCPU
Generic:
- Rename invalidate_begin() to invalidate_start() throughout KVM to
follow the kernel's nomenclature, e.g. for mmu_notifiers
- Use guard() to cleanup up various KVM+VFIO flows
- Minor cleanups
guest_memfd:
- Return -EEXIST instead of -EINVAL if userspace attempts to bind a
gmem range to multiple memslots, and fix the test that was supposed
to ensure KVM returns -EEXIST
- Treat memslot binding offsets and sizes as unsigned values to fix a
bug where KVM interprets a large "offset + size" as a negative
value and allows a nonsensical offset
- Use the inode number instead of the page offset for the NUMA
interleaving index to fix a bug where the effective index would
jump by two for consecutive pages (the caller also adds in the page
offset)
Selftests:
- Randomize the dirty log test's delay when reaping the bitmap on the
first pass, as always waiting only 1ms hid a KVM RISC-V bug as the
test reaped the bitmap before KVM could build up enough state to
hit the bug
- A pile of one-off fixes and cleanups"
* tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (326 commits)
KVM: x86/mmu: Ensure hugepage is in by slot before checking max mapping level
KVM: x86: Fix shadow paging use-after-free due to unexpected role
KVM: s390: Introducing kvm_arch_set_irq_inatomic fast inject
KVM: s390: Enable adapter_indicators_set to use mapped pages
KVM: s390: Add map/unmap ioctl and clean mappings post-guest
riscv: kvm: Use endian-specific __lelong for NACL shared memory
KVM: selftests: access_tracking_perf_test: bump number of NUMA nodes to 32
KVM: s390: vsie: Implement ASTFLEIE facility 2
KVM: s390: vsie: Refactor handle_stfle
s390/sclp: Detect ASTFLEIE 2 facility
KVM: s390: Minor refactor of base/ext facility lists
KVM: x86/mmu: move pdptrs out of the MMU
KVM: x86: check that kvm_handle_invpcid is only invoked with shadow paging
KVM: nSVM: invalidate cached PDPTRs across nested NPT transitions
KVM: nVMX: remove unnecessary code in prepare_vmcs02_rare
KVM: x86: remove nested_mmu from mmu_is_nested()
KVM: arm64: vgic-its: Make ABI commit helpers return void
KVM: s390: Initialize KVM_S390_GET_CMMA_BITS memory
LoongArch: KVM: Add missing slots_lock for device register/unregister
LoongArch: KVM: Validate irqchip index in irqfd routing
...
|
|
gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip
Pull x86 TDX updates from Dave Hansen:
"There are a few cleanups, and some changes that should allow TDX and
kexec to coexist nicely.
The biggest change, however, is support for updating the TDX module
after boot, just like CPU microcode. TDX users really want this
because it lets them do security updates without tearing things down
and rebooting.
- Add TDX module update support
- Make kexec and TDX finally place nice together
- Put TDX error codes into a single header"
* tag 'x86_tdx_for_7.2-rc1' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip: (30 commits)
x86/virt/tdx: Document TDX module update
x86/virt/tdx: Enable TDX module runtime updates
x86/virt/tdx: Refresh TDX module version after update
coco/tdx-host: Lock out module updates when reading version
x86/virt/seamldr: Add module update locking
x86/virt/tdx: Restore TDX module state
x86/virt/seamldr: Initialize the newly-installed TDX module
x86/virt/seamldr: Install a new TDX module
x86/virt/tdx: Reset software states during TDX module shutdown
x86/virt/seamldr: Shut down the current TDX module
x86/virt/seamldr: Abort updates after a failed step
x86/virt/seamldr: Introduce skeleton for TDX module updates
x86/virt/seamldr: Allocate and populate a module update request
coco/tdx-host: Implement firmware upload sysfs ABI for TDX module updates
coco/tdx-host: Don't expose P-SEAMLDR information on CPUs with erratum
coco/tdx-host: Expose P-SEAMLDR information via sysfs
x86/virt/seamldr: Add a helper to retrieve P-SEAMLDR information
x86/virt/seamldr: Introduce a wrapper for P-SEAMLDR SEAMCALLs
coco/tdx-host: Expose TDX module version
coco/tdx-host: Introduce a "tdx_host" device
...
|
|
gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip
Pull x86 SEV updates from Borislav Petkov:
- Remove redundant GHCB initialization guards in the SEV page state and
SVSM call paths now that the GHCB helpers handle early-boot fallback
internally
- Skip SNP initialization in the CCP driver immediately when the
preparation step fails rather than proceeding to an operation that
will certainly fail
- Abort SNP preparation and return an error when not all CPUs are
online, since the firmware enforces that every CPU enables SNP and
will fail init if not
- Simplify the VMM communication exception entry path by replacing
separate kernel and user mode macros with a single handler that
dispatches based on the current privilege level
* tag 'x86_sev_for_v7.2_rc1' of gitolite.kernel.org:pub/scm/linux/kernel/git/tip/tip:
x86/sev: Remove redundant ghcbs_initialized checks around __sev_{get,put}_ghcb()
crypto/ccp: Skip SNP_INIT if preparation fails
x86/sev: Do not initialize SNP if missing CPUs
x86/entry: Zap the #VC entry user and kernel macros
|
|
KVM x86 MMU changes for 7.2
- Use the kernel's "enum pg_level" in the TDX APIs instead of the TDX-Module's
level definitions (which are 0-based).
- Rework the TDX memory APIs to not require/assume that guest memory is
backed by "struct page" (in prepartion for guest_memfd hugepage support).
- Overhaul the TDP MMU => S-EPT code to move as much S-EPT specific logic as
possible into the TDX code, and to funnel (almost) all S-EPT updates into
a single chokepoint. The motivation is largely to prepare for upcoming
Dynamic PAMT support, but the cleanups are nice to have on their own.
- Plug a hole in the shadow MMU where KVM fails to recursively zap nested TDP
shadow when L1 is tearing its TDP page tables from the bottom up, as KVM's
TDP MMU now does.
|
|
The kernel exposes the TDX module version through sysfs so userspace
can check update compatibility. That information needs to remain
accurate across runtime updates.
A runtime update may change the module's update_version, so refresh
the cached version right after a successful update.
Drop __ro_after_init from tdx_sysinfo because it is now updated at
runtime.
Do not refresh the rest of tdx_sysinfo, even if some values change
across updates. TDX module updates are backward compatible, so
existing tdx_sysinfo consumers, such as KVM, can continue to operate
without seeing the new values.
[ dhansen: trim changelog ]
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Link: https://patch.msgid.link/20260520133909.409394-22-chao.gao@intel.com
|
|
TDX metadata like the version number changes during a module update.
Add functions to lock out module updates.
The current stop_machine() implementation uses worker threads. The
scheduler actually does a full, normal context switch over to that
thread. preempt_disable() obviously inhibits that context switch and
thus, locks out stop_machine() users like the module update.
Thanks to Chao for the idea of using preempt_disable().
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
|
|
After per-CPU initialization, the module is nearly functional. It is
in a similar state to TDX initialization before TDH.SYS.CONFIG.
At this point, the kernel _could_ just repeat the boot-time sequence,
but that would land the new module in a slightly different state than
the old module. This would leave old TDs unrunnable, which is not a
good outcome.
Thankfully, the "handoff" data saved during module shutdown should
contain all the information needed to restore the TDX module state to
exactly what it was before the update.
Restore TDX module state. The TDX module only needs a single copy so
only do this on the lead CPU.
Restoration errors can theoretically be handled in a few ways. For
instance, userspace could try to load a different TDX module version.
Or, the kernel could give up on the handoff process and just
reinitialize the new module from scratch, which would lose all
existing TDs.
Simply propagate errors to userspace. Ignore the idea of a
TD-destroying reinitialization. It would destroy data like a reboot
and if things have gone that wrong a reboot is probably the best
option anyway.
Note: the location and the format of handoff data is defined by the
TDX module. The new module knows where to get handoff data and how to
parse it. The kernel does not touch it at all.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Link: https://patch.msgid.link/20260520133909.409394-21-chao.gao@intel.com
|
|
Continue fleshing out the update process. At this point the new module
is sitting in memory but has never been called and is not usable. It
is in a similar state to the when the system first boots.
Leave the P-SEAMLDR behind. Stop making calls to it. Transition to
calling the new TDX module itself to set up both global and per-cpu
state.
Share tdx_cpu_enable() with the fresh-boot module initialization code.
Export it and invoke it on all CPUs.
Note: "TDX global initialization" needs to be done once before "TDX
per-CPU initialization". It would be a great fit for the new runtime
update "is_lead_cpu" logic. But tdx_cpu_enable() already has some
logic to do the global initialization properly. Just use it directly
to maximize fresh-boot and runtime update code sharing.
== Background ==
The boot-time and post-update initialization flows share the same first
steps:
- TDX global initialization
- TDX per-CPU initialization
After that, they diverge:
- Fresh boot:
Prepare TDMRs/PAMTs
Configure the TDX module
Configure the global KeyID
Initialize TDMRs
- Runtime update:
Restore TDX module state from handoff data
Future changes will consume the handoff data.
[ dhansen: major changelog munging ]
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Link: https://patch.msgid.link/20260520133909.409394-20-chao.gao@intel.com
|
|
Continue fleshing out the update proces. The old module is shut down
and the system is ready for the new module image. Run the
SEAMLDR.INSTALL SEAMCALL on all CPUs.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Link: https://patch.msgid.link/20260520133909.409394-19-chao.gao@intel.com
|
|
The TDX module requires a one-time global initialization (TDH.SYS.INIT) and
per-CPU initialization (TDH.SYS.LP.INIT) before use. These initializations
are guarded by software flags to prevent repetition.
Reset all software flags guarding the initialization flows to allow the
global and per-CPU initializations to be triggered again after updates.
[ dhansen: trim down changelog ]
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Link: https://patch.msgid.link/20260520133909.409394-18-chao.gao@intel.com
|
|
The first step of TDX module updates is shutting down the current TDX
module. This step also packs state information that needs to be
preserved across updates, called "handoff data". This handoff data is
consumed by the updated module and stored internally in the SEAM range and
hidden from the kernel.
Since the handoff data layout may change between modules, the handoff
data is versioned. Each module has a native handoff version and
provides backward support for several older versions.
The complete handoff versioning protocol is complex as it supports both
module upgrades and downgrades. See details in "Intel Trust Domain
Extensions (Intel TDX) Module Base Architecture Specification", Chapter
"Handoff Versioning".
Ideally, the kernel needs to retrieve the handoff versions supported by
the current module and the new module and select a version supported by
both. But since this implementation only supports module upgrades, simply
request handoff data from the current module using its highest supported
version. That is sufficient for this upgrade-only implementation.
Retrieve the module's handoff version from TDX global metadata and add an
update step to shut down the module. Module shutdown only needs to run on
one CPU.
Don't cache the handoff information in tdx_sysinfo. It is used only for
module shutdown, and is present only when the TDX module supports updates.
Caching it in get_tdx_sys_info() would require extra update-support guards
and refreshing the cached value across module updates.
[ dhansen: fix up function variables, remove 'cpu'.
Return from tdx_module_shutdown() early if handoff call fails. ]
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Link: https://patch.msgid.link/20260520133909.409394-17-chao.gao@intel.com
|
|
A TDX module update is a multi-step process, and any step can fail.
The current update flow continues to later steps after an error.
Continuing after a failure can cause the TDX module to enter an
unrecoverable state.
But certain failures during the initial module shutdown step should
simply return an error to userspace, so the update can be retried
cleanly.
To preserve that recoverability, one option would be to abort the
update only for those failures, since they occur before any TDX module
state is changed. But special-casing specific failures in specific
steps would complicate the do-while() update loop for no benefit.
Simply abort update on any failure, at any step.
Track failures for each step, stop the update loop once a failure is
observed, and do not advance the state machine to the next step.
[ dhansen: style nits ]
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Link: https://lore.kernel.org/linux-coco/aQFmOZCdw64z14cJ@google.com/ # [1]
Link: https://patch.msgid.link/20260520133909.409394-16-chao.gao@intel.com
|
|
tl;dr: Use stop_machine() and a state machine based on the
"MULTI_STOP" pattern to implement core TDX module update logic.
Long version:
TDX module updates require careful synchronization with other TDX
operations. The requirements are (#1/#2 reflect current behavior that
must be preserved):
1. SEAMCALLs need to be callable from both process and IRQ contexts.
2. SEAMCALLs need to be able to run concurrently across CPUs
3. During updates, only update-related SEAMCALLs are permitted; all
other SEAMCALLs shouldn't be called.
4. During updates, all online CPUs must participate in the update work.
No single lock primitive satisfies all requirements. For instance,
rwlock_t handles #1/#2 but fails #4: CPUs spinning with IRQs disabled
cannot be directed to perform update work.
Use stop_machine() as it is the only well-understood mechanism that can
meet all requirements.
And TDX module updates consist of several steps (See Intel Trust Domain
Extensions (Intel TDX) Module Base Architecture Specification, Chapter
"TD-Preserving TDX module Update"). Ordering requirements between steps
mandate lockstep synchronization across all CPUs.
multi_cpu_stop() provides a good example of executing a multi-step task
in lockstep across CPUs, but it does not synchronize the individual
steps inside the callback itself.
Implement a similar state machine as the skeleton for TDX module
updates. Each state represents one step in the update flow, and the
state advances only after all CPUs acknowledge completion of the current
step. This acknowledgment mechanism provides the required lockstep
execution.
The update flow is intentionally simpler than multi_cpu_stop() in two ways:
a) use a spinlock to protect the control data instead of atomic_t and
explicit memory barriers.
b) omit touch_nmi_watchdog() and rcu_momentary_eqs(), which exist
there for debugging and are not strictly needed for this update flow
Potential alternative to stop_machine()
=======================================
An alternative approach is to lock all KVM entry points and kick all
vCPUs. Here, KVM entry points refer to KVM VM/vCPU ioctl entry points,
implemented in KVM common code (virt/kvm). Adding a locking mechanism
there would affect all architectures KVM supports. And to lock only TDX
vCPUs, new logic would be needed to identify TDX vCPUs, which the KVM
common code currently lacks. This would add significant complexity and
maintenance overhead to KVM for this TDX-specific use case, so don't take
this approach.
[ dhansen: normal changelog/style munging ]
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Link: https://patch.msgid.link/20260520133909.409394-15-chao.gao@intel.com
|
|
There are two important ABIs here:
'struct tdx_image' - The on-disk and in-memory format for a TDX
module update image.
'struct seamldr_params' - The in-memory ABI passed to the TDX module
loader. Points to a single 'struct tdx_image'
broken up into 4k pages.
Userspace supplies the update image in 'struct tdx_image' format. The
image consists of a header followed by a sigstruct and the module
binary. P-SEAMLDR, however, consumes 'struct seamldr_params' rather
than the image directly.
Parse the 'struct tdx_image' provided by userspace and populate a
matching 'struct seamldr_params'.
The 'tdx_image' ABI is versioned. Two public versions exist today:
0x100 and 0x200. This kernel only accepts 0x200. The older 0x100
format is being deprecated and is intentionally not supported here.
Future versions of the module might be able to use the same ABIs
(user/kernel and kernel/SEAMLDR) but they will not be able to use this
kernel code.
Reject module images without that specific version. This ensures that
the kernel is able to understand the passed-in format.
Validate the 'struct tdx_image' header before using it, because the
header is consumed solely by the kernel to locate the sigstruct and
module within the image. Do not validate the payload itself. The
sigstruct and module pages are passed through to P-SEAMLDR, which
validates them as part of the update.
sigstruct_pages_pa_list currently has only one entry, but it will grow
to four pages in the future. Keep it as an array for symmetry with
module_pages_pa_list and for extensibility.
[ dhansen: normal changelog clarification/munging ]
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://patch.msgid.link/20260520133909.409394-14-chao.gao@intel.com
|
|
tl;dr: Select fw_upload for doing TDX module updates. The process of
selecting among available update images is complicated and nuanced. Punt
the selection process out to userspace. One existing userspace
implementation today is the script in the Intel TDX Module Binaries
repository[1].
Long Version:
The kernel supports two primary firmware update mechanisms:
1. request_firmware() - used by microcode, SEV firmware, hundreds of
other drivers
2. 'struct fw_upload' - used by CXL, FPGA updates, dozens of others
The key difference between is that request_firmware() loads a named file
from the filesystem where the filename is kernel-controlled, while
fw_upload accepts firmware data directly from userspace.
TDX module firmware update selection policy is too complex for the kernel.
Leave it to userspace and use fw_upload.
Add a skeleton fw_upload implementation to be fleshed out in subsequent
patches.
Refactor the sysfs visiblity attribute function so it can be used as a
more generic flag for the presence of viable runtime update support.
Why fw_upload instead of request_firmware()?
============================================
Selecting a TDX module update image is not a simple "load the latest"
decision. Userspace needs to choose an image that is compatible with both
the platform and the currently running module.
Some constraints are hard requirements:
a. Module version series are platform-specific. For example, the 1.5.x
series runs on Sapphire Rapids but not Granite Rapids, which needs
2.0.x.
b. Updates are also constrained by version distance. A 1.5.6 module
might permit updates to 1.5.7 but not to 1.5.50.
There may also be userspace policy choices:
c. Decide the update direction: upgrade or downgrade
d. Choose whether to optimize for fewer updates or smaller version
steps, for example, 1.2.3=>1.2.5 versus 1.2.3=>1.2.4=>1.2.5.
Given that complexity, leave module selection to userspace and use
fw_upload.
1. https://github.com/intel/confidential-computing.tdx.tdx-module.binaries/blob/main/version_select_and_load.py
[ dhansen: add version script link, add more explanation of code moves,
fix some minor whitespace issues ]
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Link: https://lore.kernel.org/kvm/01fc8946-eb84-46fa-9458-f345dd3f6033@intel.com/
Link: https://patch.msgid.link/20260520133909.409394-13-chao.gao@intel.com
|
|
TDX-capable CPUs clobber the current VMCS on P-SEAMLDR calls. Clearing
the current VMCS behind KVM's back breaks KVM.
Future CPUs will fix this by preserving the current VMCS across
P-SEAMLDR calls. A future specification update will describe the
VMCS-clearing behavior as an erratum and to state that it does not
occur when IA32_VMX_BASIC[60] is set.
Add a CPU bug bit and refuse to expose P-SEAMLDR information on
affected CPUs.
Use a CPU bug bit to stay consistent with X86_BUG_TDX_PW_MCE. As a
bonus, the bug bit is visible to userspace, which allows userspace to
determine why these sysfs files are not exposed, and it can also be
checked by other kernel components in the future if needed.
== Alternatives ==
Two workarounds were considered but both were rejected:
1. Save/restore the current VMCS around P-SEAMLDR calls. This produces ugly
assembly code [1] and doesn't play well with #MCE or #NMI if they
need to use the current VMCS.
2. Move KVM's VMCS tracking logic to the TDX core code, which would break
the boundary between KVM and the TDX core code [2].
[ dhansen: comment and changelog munging. Add seamldr_call() bug check. ]
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://lore.kernel.org/kvm/fedb3192-e68c-423c-93b2-a4dc2f964148@intel.com/ # [1]
Link: https://lore.kernel.org/kvm/aYIXFmT-676oN6j0@google.com/ # [2]
Link: https://patch.msgid.link/20260520133909.409394-12-chao.gao@intel.com
|
|
P-SEAMLDR reports its state via SEAMLDR.INFO, including its version and
the number of remaining runtime updates.
This information is useful for userspace. For example, userspace can
use the P-SEAMLDR version to determine whether a candidate TDX module
is compatible with the running loader, and can use the remaining
update count to determine whether another runtime update is still
possible.
Add a helper to retrieve P-SEAMLDR information in preparation for
exposing P-SEAMLDR version and other necessary information to userspace.
Export the new kAPI for use by the "tdx_host" device.
Note that there are two distinct P-SEAMLDR APIs with similar names:
"SEAMLDR.INFO" is metadata about the loader. It's metadata for the
update process.
"SEAMLDR.SEAMINFO" is metadata about SEAM mode. It is for the module
init process, not for the update process.
Use SEAMLDR.INFO here.
For details, see "Intel Trust Domain Extensions - SEAM Loader (SEAMLDR)
Interface Specification".
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://patch.msgid.link/20260520133909.409394-10-chao.gao@intel.com
|
|
The TDX architecture uses the "SEAMCALL" instruction to communicate with
SEAM mode software. Right now, the only SEAM mode software that the kernel
communicates with is the TDX module. But, there is actually another
component that runs in SEAM mode but it is separate from the TDX module:
the persistent SEAM loader or "P-SEAMLDR". Right now, the only component
that communicates with it is the BIOS which loads the TDX module itself at
boot. But, to support updating the TDX module, the kernel now needs to be
able to talk to it.
P-SEAMLDR SEAMCALLs differ from TDX module SEAMCALLs in areas such as
concurrency requirements.
Add a P-SEAMLDR wrapper to handle these differences and prepare for
implementing concrete functions.
Use seamcall_prerr() (not '_ret') because current P-SEAMLDR calls do not
use any output registers other than RAX.
Note: Despite the similar name, the NP-SEAMLDR ("Non-Persistent")
(ACM) invoked exclusively by the BIOS at boot rather than a component
running in SEAM mode. The kernel cannot call it at runtime. It exposes
no SEAMCALL interface.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://cdrdv2.intel.com/v1/dl/getContent/733582 # [1]
Link: https://patch.msgid.link/20260520133909.409394-9-chao.gao@intel.com
|
|
For TDX module updates, userspace needs to select compatible update
versions based on the current module version.
For example, the 1.5.x series runs on Sapphire Rapids but not Granite
Rapids, which needs 2.0.x. Updates are also constrained by version
distance, so a 1.5.6 module might permit updates to 1.5.7 but not to
1.5.20.
Start the process of punting the version selection logic to userspace.
Expose the TDX module version in the new faux device.
Define TDX_VERSION_FMT macro for the TDX version format since it will be
used multiple times. Also convert an existing print statement to use it.
== Background ==
For posterity, here's what other firmware mechanisms do:
1. AMD SEV leverages an existing PCI device for the PSP to expose
metadata. TDX uses a faux device as it doesn't have PCI device
in its architecture.
2. Microcode uses per-CPU virtual devices to report microcode revisions
because CPUs can have different revisions. But, there is only a
single TDX module, so exposing the TDX module version through a global
TDX faux device is appropriate
3. ARM's CCA implementation isn't in-tree yet, but will likely follow a
similar faux device approach, though it's unclear whether they need
to expose firmware version information
[ dhansen: trim changelog ]
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://lore.kernel.org/all/2025073035-bulginess-rematch-b92e@gregkh/ # [1]
Link: https://patch.msgid.link/20260520133909.409394-8-chao.gao@intel.com
|
|
TDX depends on a platform firmware module that runs on the CPU.
Unlike other CoCo architectures, TDX has no hardware "device"
running the show, just a blob on the CPU.
Create a virtual device to anchor interactions with this platform
firmware. This lets later code:
- expose metadata: TDX module version, seamldr version, to userspace
as device attributes
- implement firmware uploader APIs (which are tied to a device) to
support TDX module runtime updates
Use a faux device because the TDX module is singular within the system
and has no platform resources. Using a faux device eliminates the need
to create a stub bus.
The call to tdx_get_sysinfo() ensures that the TDX module is ready to
provide services.
Note that AMD has a PCI device for the PSP for SEV and ARM CCA will
likely have a faux device [1].
Thanks to Dan and Yilun for all the help on this one.
[ dhansen: trim changelog ]
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Xu Yilun <yilun.xu@linux.intel.com>
Reviewed-by: Kai Huang <kai.huang@intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Link: https://lore.kernel.org/all/2025073035-bulginess-rematch-b92e@gregkh/ # [1]
Link: https://patch.msgid.link/20260520133909.409394-7-chao.gao@intel.com
|
|
TDX host core code implements three seamcall*() helpers to make SEAMCALLs
to the TDX module. Currently, they are implemented in <asm/tdx.h> and
are exposed to other kernel code which includes <asm/tdx.h>.
However, other than the TDX host core, seamcall*() are not expected to
be used by other kernel code directly. For instance, for all SEAMCALLs
that are used by KVM, the TDX host core exports a wrapper function for
each of them.
Move seamcall*() and related code out of <asm/tdx.h> and make them only
visible to TDX host core.
Since TDX host core tdx.c is already very heavy, don't put low level
seamcall*() code there but to a new dedicated "seamcall_internal.h". Also,
currently tdx.c has seamcall_prerr*() helpers which additionally print
error message when calling seamcall*() fails. Move them to
"seamcall_internal.h" as well. In such way all low level SEAMCALL helpers
are in a dedicated place, which is much more readable.
Copy the copyright notice from the original files and consolidate the
date ranges to:
Copyright (C) 2021-2023 Intel Corporation
Signed-off-by: Kai Huang <kai.huang@intel.com>
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Reviewed-by: Zhenzhong Duan <zhenzhong.duan@intel.com>
Reviewed-by: Binbin Wu <binbin.wu@linux.intel.com>
Reviewed-by: Tony Lindgren <tony.lindgren@linux.intel.com>
Reviewed-by: Kiryl Shutsemau (Meta) <kas@kernel.org>
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: Vishal Annapurve <vannapurve@google.com>
Acked-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://patch.msgid.link/20260520133909.409394-6-chao.gao@intel.com
|
|
Future changes will add support for new TDX features exposed as
TDX_FEATURES0 bits. The presence of these features will need to be
checked outside of arch/x86/virt. The feature query helpers and
the TDX_FEATURES0 defines they reference will need to live in the
widely accessible asm/tdx.h header. Move the existing TDX_FEATURES0 to
asm/tdx.h so that they can all be kept together.
Opportunistically switch to BIT_ULL() since TDX_FEATURES0 is 64-bit.
No functional change intended.
[ dhansen: grammar fixups ]
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://lore.kernel.org/kvm/20260427152854.101171-17-chao.gao@intel.com/ # [1]
Link: https://lore.kernel.org/kvm/20251121005125.417831-16-rick.p.edgecombe@intel.com/ # [2]
Link: https://patch.msgid.link/20260520133909.409394-5-chao.gao@intel.com
|
|
The kernel uses several global flags to guard one-time TDX initialization
flows and prevent them from being repeated.
When the TDX module is updated, all of those states must be reset so that
the module can be initialized again. Today those states are kept as
separate global variables, which makes the reset path awkward and easy to
miss when a new state is added.
Group the states into a single structure so they can be reset together, for
example with memset(), and so a newly added state won't be missed.
Drop the __ro_after_init annotation from tdx_module_initialized because
the other two states do not have it. And with TDX module update support,
all the states need to be writable at runtime.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://patch.msgid.link/20260520133909.409394-4-chao.gao@intel.com
|
|
TDX module global initialization is executed only once. The first call
caches both the result and the "done" state, and later callers reuse the
saved result. A lock protects that cached states.
Those states and the lock are currently kept as function-local statics
because they are used only by try_init_module_global().
TDX module updates need to reset the cached states so TDX global
initialization can be run again after an update. That will add another
access site in the same file.
Move the cached states to file scope so it is accessible outside
try_init_module_global(), and move the lock along with the states it
protects.
No functional change intended.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://patch.msgid.link/20260520133909.409394-3-chao.gao@intel.com
|
|
TDX module global initialization is executed only once. The first call
caches both the return code and the "done" state in static function
variables. Later callers read the variables. A lock protects the
saved state and serializes callers.
These variables will soon be moved to a global structure. Prepare for
that by treating the variables as a unit. Assign them together and
limit accesses to while the lock is held.
[ dhansen: mostly rewrite changelog ]
Signed-off-by: Chao Gao <chao.gao@intel.com>
Signed-off-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://patch.msgid.link/20260520133909.409394-2-chao.gao@intel.com
|
|
Move mk_keyed_paddr() from tdx.h to tdx.c to avoid unnecessary header
inclusion and improve encapsulation since there are no users outside of
tdx.c.
No functional change intended.
Signed-off-by: Yan Zhao <yan.y.zhao@intel.com>
Acked-by: Kiryl Shutsemau <kas@kernel.org>
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Acked-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://patch.msgid.link/20260430015014.24261-1-yan.y.zhao@intel.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
KVM invokes tdx_quirk_reset_page() to reset TDX control pages (including
S-EPT pages, TDR page, etc.), as all those pages are allocated by KVM TDX
and thus always have struct page.
However, it's also reasonable for KVM to reset those TDX control pages via
tdx_quirk_reset_paddr() directly, eliminating the need to export two
parallel APIs. Keeping tdx_quirk_reset_page() as a one-line helper in the
header file is also unnecessary.
No functional change intended.
Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Suggested-by: Xiaoyao Li <xiaoyao.li@intel.com>
Signed-off-by: Yan Zhao <yan.y.zhao@intel.com>
Acked-by: Kiryl Shutsemau <kas@kernel.org>
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: Ackerley Tng <ackerleytng@google.com>
Acked-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://patch.msgid.link/20260430015001.24242-1-yan.y.zhao@intel.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Remove struct page assumptions/constraints in APIs for unmapping guest
private memory and have them take physical address directly.
Having core TDX make assumptions that guest private memory must be backed
by struct page (and/or folio) will create subtle dependencies on how
KVM/guest_memfd allocates/manages memory (e.g., whether it uses memory
allocated from core MM, if the memory is refcounted, or if the folio is
split) that are easily avoided. [1].
KVM's MMUs work with PFNs. This is very much an intentional design choice.
It ensures that the KVM MMUs remain flexible and are not too tightly tied
to the regular CPU MMUs and the kernel code around them. Using
"struct page" for TDX guest memory is not a good fit anywhere near the KVM
MMU code [2].
Therefore, for unmapping guest private memory: export
tdx_quirk_reset_paddr() for direct KVM invocation, and convert the SEAMCALL
wrapper API tdh_phymem_page_wbinvd_hkid() to take PFN as input (thus
updating mk_keyed_paddr() and tdh_phymem_page_wbinvd_tdr()).
Intentionally have KVM pass PAGE_SIZE (rather than KVM_HPAGE_SIZE(level))
to tdx_quirk_reset_paddr() in tdx_sept_remove_private_spte() to avoid
mixing in huge page changes. The KVM_BUG_ON() check for !PG_LEVEL_4K in
tdx_sept_remove_private_spte() justifies using PAGE_SIZE.
Do not convert tdx_reclaim_page() to use PFN as input since it currently
does not remove guest private memory.
Use "kvm_pfn_t pfn" for type safety. Using this KVM type is appropriate
since APIs tdh_phymem_page_wbinvd_hkid() and tdx_quirk_reset_paddr() are
exported to KVM only.
[Yan: Use kvm_pfn_t,exclude tdx_reclaim_page(),use tdx_quirk_reset_paddr()]
Signed-off-by: Yan Zhao <yan.y.zhao@intel.com>
Link: https://lore.kernel.org/all/aWgyhmTJphGQqO0Y@google.com [1]
Link: https://lore.kernel.org/all/ac7V0g2q2hN3dU5u@google.com [2]
Acked-by: Kiryl Shutsemau <kas@kernel.org>
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: Ackerley Tng <ackerleytng@google.com>
Acked-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://patch.msgid.link/20260430014948.24226-1-yan.y.zhao@intel.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
|
Remove struct page assumptions/constraints in the SEAMCALL wrapper APIs for
mapping guest private memory and have them take PFN directly.
Having core TDX make assumptions that guest private memory must be backed
by struct page (and/or folio) will create subtle dependencies on how
KVM/guest_memfd allocates/manages memory (e.g., whether it uses memory
allocated from core MM, if the memory is refcounted, or if the folio is
split) that are easily avoided. [1].
KVM's MMUs work with PFNs. This is very much an intentional design choice.
It ensures that the KVM MMUs remain flexible and are not too tied to the
regular CPU MMUs and the kernel code around them. Using 'struct page' for
TDX guest memory is not a good fit anywhere near the KVM MMU code [2].
Use "kvm_pfn_t pfn" for type safety. Using this KVM type is appropriate
since APIs tdh_mem_page_add() and tdh_mem_page_aug() are exported to KVM
only.
[ Yan: Replace "u64 pfn" with "kvm_pfn_t pfn" ]
Signed-off-by: Yan Zhao <yan.y.zhao@intel.com>
Link: https://lore.kernel.org/all/aWgyhmTJphGQqO0Y@google.com [1]
Link: https://lore.kernel.org/all/ac7V0g2q2hN3dU5u@google.com [2]
Acked-by: Kiryl Shutsemau <kas@kernel.org>
Reviewed-by: Xiaoyao Li <xiaoyao.li@intel.com>
Reviewed-by: Ackerley Tng <ackerleytng@google.com>
Acked-by: Dave Hansen <dave.hansen@linux.intel.com>
Link: https://patch.msgid.link/20260430014929.24210-1-yan.y.zhao@intel.com
Signed-off-by: Sean Christopherson <seanjc@google.com>
|
