| Age | Commit message (Collapse) | Author |
|---|
|
Add function 0x23 to the set of CPUID functions that are indexed, as per
the SDM and the many sub-leafs defined in leaf_types.h.
Closes: https://lore.kernel.org/all/213d614fe73e183a230c8f4e0c8fa1cc3d45df39.camel@intel.com
Reported-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Signed-off-by: Sean Christopherson <seanjc@google.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Tested-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
Cc: stable@vger.kernel.org
Link: https://patch.msgid.link/20261022210134.612848-2-seanjc@google.com
|
|
Even with __builtin the compiler may decide to use the out of line function
instead of the inline implementation.
The existing code is broken with gcc-14/15 but not gcc-12/13 (Ubuntu 25.10)
and vmlinux no longer boots with CONFIG_PVH if CONFIG_KASAN_GENERIC is set.
The instrumented out of line function performs a memory access to a region not
yet initialized by KASAN, as it is still in the PVH kernel entry point.
For testing purposes, if the size argument in cpuid_base_hypervisor() is
reduced from 12 to 8 the compiler decides to use the inline implementation.
In xen_prepare_pvh(), it (still) decides to use the inline implementation
(at least in these compiler versions), but it is not guaranteed to remain.
Switch the builtin to the inline implementation to address this.
[ bp: Massage commit message. ]
Fixes: 416a33c9afce ("x86/cpu: fix unbootable VMs by inlining memcmp() in hypervisor_cpuid_base()")
Fixes: fbe5a6dfe492 ("xen, pvh: fix unbootable VMs by inlining memset() in xen_prepare_pvh()")
Signed-off-by: Mauricio Faria de Oliveira <mfo@igalia.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Juergen Gross <jgross@suse.com>
Link: https://patch.msgid.link/20260921-pvh-kasan-inline-v10-4-08da47943d8e@igalia.com
|
|
Introduce a CPUID parser for populating the system's CPUID tables.
Since accessing a leaf within the CPUID table requires compile time
tokenization, split the parser into two stages:
(a) Compile-time macros for tokenizing the leaf/subleaf offsets within
the CPUID table.
(b) Generic runtime code to fill the CPUID data, using a parsing table
which collects these compile-time offsets.
For actual CPUID output parsing, support both generic and leaf-specific
read functions.
To ensure CPUID data early availability, invoke the parser during early
boot, early Xen boot, and at early secondary CPUs bring up.
Provide call site APIs to refresh a single leaf, or a leaf range, within
the CPUID tables. This is for sites issuing MSR writes that partially
change the CPU's CPUID layout. Doing full CPUID table rescans in such
cases will be destructive since the CPUID tables will host all of the
kernel's X86_FEATURE flags at a later stage.
Suggested-by: Thomas Gleixner <tglx@kernel.org>
Signed-off-by: Ahmed S. Darwish <darwi@linutronix.de>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://lore.kernel.org/all/20260327021645.555257-1-darwi@linutronix.de
|
|
** Context
The x86-cpuid-db project generates a C header file with full C99 bitfield
listings for all known CPUID leaf/subleaf query outputs.
That header is now merged by parent commits at <asm/cpuid/leaf_types.h>,
and is of the form:
struct leaf_0x0_0 { /* CPUID(0x0).0 C99 bitfields */ };
...
struct leaf_0x4_n { /* CPUID(0x4).n C99 bitfields */ };
...
struct leaf_0xd_0 { /* CPUID(0xd).0 C99 bitfields */ };
struct leaf_0xd_1 { /* CPUID(0xd).1 C99 bitfields */ };
struct leaf_0xd_n { /* CPUID(0xd).n C99 bitfields */ };
...
** Goal
Introduce a structured, size-efficient, per-CPU, CPUID data repository.
Use the x86-cpuid-db auto-generated data types, and custom CPUID leaf
parsers, to build that repository. Given a leaf, subleaf, and index,
provide direct memory access to the parsed and cached per-CPU CPUID output.
** Long-term goal
Remove the need for drivers and other areas in the kernel to invoke direct
CPUID queries. Only one place in the kernel should be allowed to use the
CPUID instruction: the CPUID parser code.
** Implementation
Introduce CPUID_LEAF()/CPUID_LEAF_N() to build a compact CPUID storage
layout in the form:
struct leaf_0x0_0 leaf_0x0_0[1];
struct leaf_parse_info leaf_0x0_0_info;
struct leaf_0x1_0 leaf_0x1_0[1];
struct leaf_parse_info leaf_0x0_0_info;
struct leaf_0x4_n leaf_0x4_n[8];
struct leaf_parse_info leaf_0x4_n_info;
...
where each CPUID query stores its output at the designated leaf/subleaf
array and has an associated "CPUID query info" structure.
Embed the CPUID tables inside "struct cpuinfo_x86" to ensure early-boot and
per-CPU access through the CPUs capability structures.
Use an array of CPUID output storage entries for each leaf/subleaf
combination to accommodate leaves which produce the same output format for
a large subleaf range. This is typical for CPUID leaves enumerating
hierarchical objects; e.g. CPUID(0x4) cache topology enumeration,
CPUID(0xd) XSAVE enumeration, and CPUID(0x12) SGX Enclave Page Cache
enumeration.
** New CPUID APIs
Assuming a CPU capability structure 'c', provide macros to access the
parsed and cached CPUID leaf/subleaf output. These macros resolve to a
compile-time tokenization that ensures type-safety:
const struct leaf_0x7_0 *l7_0;
l7_0 = cpuid_subleaf(c, 0x7, 0);
| | └────────┐
| └─────────┐ |
* * *
&c.cpuid.leaf_0x7_0[0]
For CPUID leaves with multiple subleaves having the same output format,
provide the APIs:
const struct leaf_0x4_n *l4_0, *l4_1;
l4_0 = cpuid_subleaf_n(c, 0x4, 0);
| | └──────────┐
| └─────────┐ |
* * v
&c.cpuid.leaf_0x4_n[0]
l4_1 = cpuid_subleaf_n(c, 0x4, 1);
| | └──────────┐
| └─────────┐ |
* * v
&c.cpuid.leaf_0x4_n[1]
where the indices 0, 1, n above can be passed dynamically; e.g., in an
enumeration for loop.
Add a clear rationale on why call sites should use the these new APIs
instead of directly invoking CPUID.
** Next steps
For now, define cached parse entries for CPUID(0x0) and CPUID(0x1).
Generic parser logic to fill the CPUID tables, along with more CPUID leaves
support, will be added next.
Suggested-by: Thomas Gleixner <tglx@kernel.org> # CPUID data model
Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com> # x86-cpuid-db schema
Suggested-by: Borislav Petkov <bp@alien8.de> # Early CPUID centralization drafts
Suggested-by: Ingo Molnar <mingo@kernel.org> # CPUID headers restructuring
Suggested-by: Sean Christopherson <seanjc@google.com> # cpuid_subleaf_n() APIs
Signed-off-by: Ahmed S. Darwish <darwi@linutronix.de>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://lore.kernel.org/all/20260327021645.555257-1-darwi@linutronix.de
Link: https://lore.kernel.org/all/874ixernra.ffs@tglx
Link: https://gitlab.com/x86-cpuid.org/x86-cpuid-db
Link: https://lore.kernel.org/all/aBnSgu_JyEi8fvog@gmail.com
Link: https://lore.kernel.org/all/aJ9TbaNMgaplKSbH@google.com
|
|
A new CPUID model will be added where its APIs will be designated as the
official CPUID API. Free the cpuid_leaf() and cpuid_subleaf() function
names for that API. Rename them accordingly to cpuid_read() and
cpuid_read_subleaf().
For kernel/cpuid.c, rename its local file operations read function from
cpuid_read() to cpuid_read_f() so that it does not conflict with the new
API.
No functional change.
Signed-off-by: Ahmed S. Darwish <darwi@linutronix.de>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://lore.kernel.org/20260327021645.555257-1-darwi@linutronix.de
|
|
hypervisor_cpuid_base()/for_each_possible_hypervisor_cpuid_base() to cpuid_base_hypervisor()/for_each_possible_cpuid_base_hypervisor()
In order to let all the APIs under <cpuid/api.h> have a shared "cpuid_"
namespace, rename hypervisor_cpuid_base() to cpuid_base_hypervisor().
To align with the new style, also rename:
for_each_possible_hypervisor_cpuid_base(function)
to:
for_each_possible_cpuid_base_hypervisor(function)
Adjust call-sites accordingly.
Suggested-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Ahmed S. Darwish <darwi@linutronix.de>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: David Woodhouse <dwmw2@infradead.org>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: John Ogness <john.ogness@linutronix.de>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Sean Christopherson <seanjc@google.com>
Cc: Vitaly Kuznetsov <vkuznets@redhat.com>
Cc: x86-cpuid@lists.linux.dev
Link: https://lore.kernel.org/r/aCZOi0Oohc7DpgTo@lx-t490
|
|
Rename the CPUID(0x2) register accessor function:
cpuid_get_leaf_0x2_regs(regs)
to:
cpuid_leaf_0x2(regs)
for consistency with other <cpuid/api.h> accessors that return full CPUID
registers outputs like:
cpuid_leaf(regs)
cpuid_subleaf(regs)
In the same vein, rename the CPUID(0x2) iteration macro:
for_each_leaf_0x2_entry()
to:
for_each_cpuid_0x2_desc()
to include "cpuid" in the macro name, and since what is iterated upon is
CPUID(0x2) cache and TLB "descriptos", not "entries". Prefix an
underscore to that iterator macro parameters, so that the newly renamed
'desc' parameter do not get mixed with "union leaf_0x2_regs :: desc[]" in
the macro's implementation.
Adjust all the affected call-sites accordingly.
While at it, use "CPUID(0x2)" instead of "CPUID leaf 0x2" as this is the
recommended style.
No change in functionality intended.
Signed-off-by: Ahmed S. Darwish <darwi@linutronix.de>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: John Ogness <john.ogness@linutronix.de>
Cc: x86-cpuid@lists.linux.dev
Link: https://lore.kernel.org/r/20250508150240.172915-6-darwi@linutronix.de
|
|
In order to let all the APIs under <cpuid/api.h> have a shared "cpuid_"
namespace, rename have_cpuid_p() to cpuid_feature().
Adjust all call-sites accordingly.
Suggested-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Ahmed S. Darwish <darwi@linutronix.de>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: John Ogness <john.ogness@linutronix.de>
Cc: x86-cpuid@lists.linux.dev
Link: https://lore.kernel.org/r/20250508150240.172915-4-darwi@linutronix.de
|
|
Move all of the CPUID(0x2) APIs at <cpuid/leaf_0x2_api.h> into
<cpuid/api.h>, in order centralize all CPUID APIs into the latter.
While at it, separate the different CPUID leaf parsing APIs using
header comments like "CPUID(0xN) parsing: ".
Suggested-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Ahmed S. Darwish <darwi@linutronix.de>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: John Ogness <john.ogness@linutronix.de>
Cc: x86-cpuid@lists.linux.dev
Link: https://lore.kernel.org/r/20250508150240.172915-2-darwi@linutronix.de
|
|
Align the backspaces vertically again, after recent cleanups.
No functional changes.
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: Ahmed S. Darwish <darwi@linutronix.de>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Link: https://lore.kernel.org/r/20250414094130.6768-1-bp@kernel.org
|
|
Multiple code paths at cacheinfo.c and amd_nb.c check for AMD/Hygon CPUs
L3 cache presensce by directly checking leaf 0x80000006 EDX output.
Extract that logic into its own function. While at it, rework the
AMD/Hygon LLC topology ID caclculation comments for clarity.
Signed-off-by: Ahmed S. Darwish <darwi@linutronix.de>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Link: https://lore.kernel.org/r/20250324133324.23458-29-darwi@linutronix.de
|
|
Use u32 instead of uint32_t in hypervisor_cpuid_base().
Yes, uint32_t is used in Xen code et al, but this is a core x86
architecture header and we should standardize on the type that
is being used overwhelmingly in related x86 architecture code.
The two types are the same so there should be no build warnings.
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: John Ogness <john.ogness@linutronix.de>
Cc: "Ahmed S. Darwish" <darwi@linutronix.de>
Cc: x86-cpuid@lists.linux.dev
Link: https://lore.kernel.org/r/20250317221824.3738853-6-mingo@kernel.org
|
|
Convert all uses of 'unsigned int' to 'u32' in <asm/cpuid/api.h>.
This is how a lot of the call sites are doing it, and the two
types are equivalent in the C sense - but 'u32' better expresses
that these are expressions of an immutable hardware ABI.
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Reviewed-by: Xin Li (Intel) <xin@zytor.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: John Ogness <john.ogness@linutronix.de>
Cc: "Ahmed S. Darwish" <darwi@linutronix.de>
Cc: x86-cpuid@lists.linux.dev
Link: https://lore.kernel.org/r/20250317221824.3738853-5-mingo@kernel.org
|
|
- Include <asm/cpuid/types.h> first, as is customary. This also has
the side effect of build-testing the header dependency assumptions
in the types header.
- No newline necessary after the SPDX line
- Newline necessary after inline function definitions
- Rename native_cpuid_reg() to NATIVE_CPUID_REG(): it's a CPP macro,
whose name we capitalize in such cases.
- Prettify the CONFIG_PARAVIRT_XXL inclusion block a bit
- Standardize register references in comments to EAX/EBX/ECX/etc.,
from the hodgepodge of references.
- s/cpus/CPUs because why add noise to common acronyms?
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: John Ogness <john.ogness@linutronix.de>
Cc: "Ahmed S. Darwish" <darwi@linutronix.de>
Cc: x86-cpuid@lists.linux.dev
Link: https://lore.kernel.org/r/20250317221824.3738853-4-mingo@kernel.org
|
|
In preparation for future commits where CPUID headers will be expanded,
refactor the CPUID header <asm/cpuid.h> into:
asm/cpuid/
├── api.h
└── types.h
Move the CPUID data structures into <asm/cpuid/types.h> and the access
APIs into <asm/cpuid/api.h>. Let <asm/cpuid.h> be just an include of
<asm/cpuid/api.h> so that existing call sites do not break.
Suggested-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Ahmed S. Darwish <darwi@linutronix.de>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: "H. Peter Anvin" <hpa@zytor.com>
Cc: John Ogness <john.ogness@linutronix.de>
Cc: "Ahmed S. Darwish" <darwi@linutronix.de>
Cc: x86-cpuid@lists.linux.dev
Link: https://lore.kernel.org/r/20250317221824.3738853-2-mingo@kernel.org
|
