| Age | Commit message (Collapse) | Author |
|---|
|
Pull KVM updates from Paolo Bonzini:
"ARM:
- pKVM state isolation: years after pKVM started being enabled
upstream, we finally get the basics in the form of state isolation.
The vcpu state is now off-limit for the host for protected VMs
- pKVM also grows a hypervisor-private memory allocator, allowing it
to deal with sub-page allocation, much like everywhere else
- pKVM also lands a trace_printk() equivalent, which hopefully will
make debugging a bit easier
- First shot at annotating portions of the KVM/arm64 code to be
extracted and built on s390. The real fun is yet to come
- Allow pre-faulting of stage-2 canonical mappings from userspace,
joining x86 and s390 in implementing this feature
- Add a reverse mapping infrastructure for shadow stage-2,
significantly improving performance when MMU notifiers kick in, and
avoiding full unmap of all shadow tables for this VM
- Next step of GICv5 enablement, adding support for the IRS block,
and providing both SPI and LPI support. This is the first point
where KVM/arm64 can support an SMP GICv5 guest
- As a preliminary to eventually enabling CCA, add a bunch of
infrastructure to KVM/arm64, splitting some of the basic operations
into individual backends (normal VM, normal pKVM VM, protected pKVM
VM). This allows CCA to later slot in without polluting the rest of
KVM with its firmware-driven nonsense
- Make ID registers immutable from the point where a vcpu is able to
run, even if it fails to run. This prevents KVM from sampling
untrusted values
- Restore some type checking to the EL2 object. KVM/arm64 originally
had some strong type checking between the kernel and the EL2
object, but lost it along the way when the private hypercall
interface was (somewhat pointlessly) converted to SMCCC. Type
checking is back, syscall style, and with added sparse goodies
- Expand the FFA support to FFA_NOTIFICATION_*, and add some
sanitisation of the protocol that pKVM relays to the "trusted"
runtime
- Make GICv3 RD assignment more robust to userspace nonsense
- Convert the remaining MMU fault handler backends to
kvm_s2_fault_vma_info and co., fixing a number of bugs in the
process
- Fix the SError/SEA selftests to be both more robust and less
braindead
- Fix a smattering of documentation bugs, some of them as a result of
our AI friends spitting out credible assertions based on
non-existing symbols
- The usual collection of low impact bug fixes (SMCCC, timers, PMU)
Loongarch:
- Add separate vmid support
- Simplify MSI interrupt injection
- Add crash hypercall support
- Allow to set pv_feature until vCPU run
- Enable HW page table walk automatically in selftests
s390:
- rework page allocation for secure guest variable storage
- Selftests and documentation improvements
- Replace get_zeroed_page with kzalloc
- Fix corner cases of floating interrupt injection
- Initial arm64 on s390 support
x86:
- Clean up the handling of vcpu->arch.xen.long_mode to be consistent
and correctly handle 32-bit/64-bit alignment, and various other bug
fixes have accumulated over the months since v1
- Grab KVM's SRCU "lock" in kvm_zap_gfn_range() to fix a recurring
bug where callers fail to do so
- Fix a variety of issues in the pre-fault code that are mostly
benign in the current code base, but will show up in big ways
if/when the pre-faulting logic is used to implementing
prefetch-on-fault for the TDP MMU
- For PREEMPT_RT=y kernels, drop the rmaps' individual bit-spinlock
and just walk the rmaps' with mmu_lock taken. On the write side
(which always uses mmu_lock), the bit-spinlock was assuming that
preemption was disabled, but that is *not* true when rwlocks become
sleepable
- Prepare a memory_fault exit (paired with -EFAULT) when KVM hits an
error PFN, e.g. so that userspace can resolve transient issues
instead of killing the guest. Convert a variety of WARN()+EFAULT
paths into KVM_BUG_ON()+EIO so that KVM can hard commit to filling
memory_fault on -EFAULTs that occur in the page fault path
- Fix a NX hugepage un-accounting bug where KVM would always uncharge
the shadow MMU, even when handling TDP MMU pages
- Mask off the address space ID when checking for illegal usage of
kvm_set_internal_memslot() to prevent false negatives on the sanity
check
- Explicitly zero reserved flags when delivering a SynIC message to
avoid leaking a byte of stack information to the guest
- Rework TDX's handling of the canonical MMU root level to not rely
on userspace to fill CPUID correctly, as doing so lets userspace
trivially trigger a KVM_BUG_ON()
- Reject KVM_SET_NESTED_STATE if userspace "restores" L1 with
EFER.LMA=1 && EFER.LME=0, which confuses KVM into trying to
configure an 64-bit MMU without PAE
- Ignore EFER.LMA (clear it from KVM's cached save state) if
EFER.LME=0 when preparing L2 state, to fix another way to confuse
KVM's MMU (and because real hardware also ignores EFER.LMA)
- Improve KVM's sanity checks for a misconfigured MMU to (hopefully)
detect all of the ways KVM can botch EFER.LMA vs. CR4.PAE
- Remove the "full" out-param from kvm_x86_ops.flush_tlb_gva() as the
param is only used to avoid what effectively amounts to a single
memory write
- Fix a mostly-benign bug where KVM can write through a guest's
read-only mapping due to not checking the error code when
expediting emulated MMIO page faults
- Honor the guest's view of the "EFER.LMSLE must be zero" defeature
so that userspace can provide migration compatibility between Rome
and Milan+
- Pend a KVM_REQ_EVENT request when an exception is morphed into a
nested VM-Exit, otherwise KVM can fail to actually deliver the
nested VM-Exit, e.g. when injecting a page fault
- Add support for PML on SVM
- Add support for virtualizing Bus Lock Detect on SVM, and fix a
number of DR6 bugs and warts along the way
- Fix a variety of warts in KVM's handling of guest TSC, kvmclock,
and timekeeping in general, as a (big) first step towards
addressing several longstanding problems with the kvmclock
implementation. Most notably, fix a precision issue where
get_kvmclock_ns() computed the clock from the *host* TSC without
applying guest TSC scaling, causing systemic drift from the values
the guest computes from its own TSC
- Rip out KVM's shadow timekeeping structures, now that ktime
snapshots are powerful enough to replace it
- Misc fixes and cleanups
x86 (AMD):
- Fix a benign-in-practice bug where KVM unintentionally sets a
non-zero error code when delivering exceptions to SEV-ES+ guests
- Force a new ASID in both VMCBs on pCPU migration to fix a bug where
KVM can run a VMCB with a stale ASID due to the tracking being
per-CPU, not per-VMCB
- Return INVALID_INPUT to the guest instead of exiting to userspace
with -EIO if KVM can't access "guest request" buffer, i.e. try to
keep the guest alive if the guest or userspace goofs
x86 (Intel):
- Enable Notify VM-Exits (now called Instruction Timeout in the SDM)
and Bus Lock VM-Exits for TDX guests when they're enabled by
userspace (KVM forgot to wire up the TDX Module knobs)
- Move all common (between VMX and TDX) APIs/helpers to
common.h/main.c, and change all namespace prefixes from "vmx" to
"vt"
- Fix a thankfully benign bug (due to other areas of KVM being
hardened) where userspace can bypass INIT_MEM_REGION's address wrap
check by providing a nr_pages that will overflow a u64 when
converted to bytes, and fix an also- benign off-by-one bug in the
check (inclusive vs. exclusive)
- Set the vmcs.GUEST_ACTIVITY_STATE back to ACTIVE when userspace
sets KVM's mp_state to anything other than HALTED, e.g. so that a
vCPU doesn't get stuck in HLT if userspace emulates RESET
guest_memfd:
- Take the invalidate lock when unbinding a dying file to fix a race
with kvm_gmem_invalidate_end() that results in KVM leaving the MMU
invalidation count elevated. Not taking the lock in the first place
was pure misguided paranoid
- When binding a memslot, purge the bindings if storing the new
memslot fails, as Xarray's "store range" API can subtly fail
partway through
- Establish memslot<=>guest_memfd bindings *after* the memslot is
ready, as the memslot becomes reachable the instant the binding is
installed
- Don't grab a page refcount when reloading an SNP VMSA and stop
returning a "struct page" from guest_memfd's PFN lookup altogether,
to ensure that KVM doesn't hold any unnecessary refcounts (the
lifetime of the page is ensured by MMU notifiers and similar
hooks). Preventing unnecessary refcounts greatly simplifies
supporting in-place SHARED=>PRIVATE conversions
- Add support for in-place PRIVATE<=>SHARED conversion, i.e. allow a
single guest_memfd instance to be used for both PRIVATE and SHARED
variations of CoCo guests' memory (though not at the same time).
This eliminates the need to double allocate guest memory for TDX
and SNP, and is basically a hard requirement for pKVM, which relies
on being able to preserve memory contents on conversions
- Hack-a-fix a deadlock bug in ->error_remove_folio() by returning
-EBUSY if the filemap invalidate lock is contended. Punt on a
proper fix for the moment, as a proper fix likely involves
reworking truncate_error_folio()
- Add a pile of selftests for in-place conversions
- Misc guest_memfd cleanups
Generic:
- Move some configuration items to preprocessor symbols, to help with
arm-on-s390's need to compile the same code twice with different
values
- Allow KVM implementations to choose alternative device names, also
for s390's support of ARM virtualmachines
- Switch the KVM-VFIO interface and external consumers over to
standard file-based reference counting, eliminating all external
KVM symbol exports
Selftests:
- Rework KVM selftest's handling of memory regions all so that PPC
can force its page table allocations to be naturally aligned
- PPC support
- Fix emulation of VMPTRST and VMPTRLD when eVMCS is in-use to match
real Hyper-V's behavior
- Saturate L2's TSC frequency if stacking L1 and L2 scaling
multipliers exceeds what hardware supports, as attempting to run L2
with an illegal multiplier will either cause VM-Entry to fail (VMX)
or trigger a #GP when writing MSR_AMD64_TSC_RATIO (SVM). The #GP is
thankfully benign in the grand scheme as even "unsafe" WRMSRs have
exception fixup
- Rework KVM selftest's handling of memory regions so that PPC can
force its page table allocations to be naturally aligned
- Clean up selftest's VMX instruction helpers, and drop an absurd
number of completely useless return values, e.g. functions that
unconditionally return %true and then have every caller assert
success
- Add selftests support for Zhaoxin CPUs
- Overhaul the TSC scaling test to add a regression test for the
saturation behavior, and to increase its coverage in general
- Fix selftests usage various NUMA syscalls to account for an
undocumented, but known since 2007, off-by-one kernel bug. Add
helpers to try and avoid re-introducing the same flaws in the
future
- Automatically run xAPIC IPI migration test (between NUMA nodes)
when possible, as the entire point of the test is verify KVM's
handling of page migration of the APIC backing page
- Misc cleanups"
* tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (506 commits)
KVM: always make in-place conversion a compile-time constant when possible
KVM: x86: leave the legacy private attribute in for TDX and SEV
KVM: split CONFIG_KVM_PRIVATE_MEMATTR_VM out of KVM_VM_MEMORY_ATTRIBUTES
selftests: kvm: remove useless comma operator
arm64: Use proper include variant
MAINTAINERS: Add entry for ARM on s390 KVM
KVM: s390: arm64: Integrate arm on s390
KVM: s390: arm64: Implement basic page fault handler
KVM: s390: arm64: Implement vCPU IOCTLs
KVM: s390: arm64: Implement vm/vcpu create destroy
KVM: s390: arm64: Implement required functions
KVM: s390: Add basic arm64 kvm module
s390/hwcaps: Report SAE support as hwcap
KVM: s390: arm64: Introduce host definitions
s390: Introduce Start Arm Execution instruction
KVM: s390: Use arm64 code
s390/tools: Use arm64 headers
KVM: s390: Use dedicated function for migration mode
s390/Kconfig: remove PCI dependency from HAS_IOMEM
KVM: s390: Kick PV cpus at the right time for service irqs
...
|
|
Each kernel make invocation begins with ~30 compiler and linker runs each
of which performs duplicate probe for a number of compiler and linker
options.
This is useless work - the compiler and its version is known, so use these
to determine which options are available, once.
Using the newly introduced cc-option-str, track the result of each probe in
CC_OPT_*, LD_OPT_* variables and have Makefile's simply append the symbol.
Some care has to be taken around negative options, i.e. -Wno-xxx, as gcc
accepts an unknown -Wno- without complaint and only complains later, so
instead probe the positive form for these cases.
Use CC_HAS_* For those probes that gate a Makefile conditional rather than
a simple option, as per the established pattern for this.
This change avoids a subtle issue with ppc32 (big endian) - clang doesn't
accept -fno-stack-clash-protection on this sub-target, so keep this option
in the Makefile.
With gcc and clang, defconfig and allmodconfig, the recorded command
lines are unchanged, a build with nothing to do rebuilds nothing and W=1
continues to function correctly.
Doing this improves all builds, but has a particularly positive impact on
no-op builds (builds where nothing has changed).
Whole build, 128-thread Threadripper 9980X, best of N runs:
before after delta
-------------------------------
x86 defconfig, no-op make, gcc 0.78s 0.60s -0.18s (-23%)
x86 defconfig, no-op make, clang 0.94s 0.66s -0.28s (-30%)
x86 defconfig, touch mm/vma.c, gcc 8.6s 8.5s -0.16s (-2%)
x86 defconfig, touch mm/vma.c, clang 8.0s 7.7s -0.24s (-3%)
x86 defconfig, clean, clang 29.8s 29.4s -0.37s (-1%)
x86 allmodconfig, no-op make, gcc 12.9s 12.7s -0.23s (-2%)
x86 allmodconfig, no-op make, clang 14.2s 13.9s -0.29s (-2%)
x86 allmodconfig, touch mm/vma.c, clang 32.2s 31.9s -0.30s (-1%)
Assisted-by: LLM
Signed-off-by: Lorenzo Stoakes (ARM) <ljs@kernel.org>
Link: https://patch.msgid.link/20260923-build-speedup-v4-12-73128809a4a4@kernel.org
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
|
|
The custom lib-%.o rule reuses rule_cc_o_c, which hooks the source
checker only for C=1, and that only when the object is rebuilt. The
C=2 hook, cmd_force_checksrc, hangs off the standard %.o rule, so
"make C=2" silently skips the two libfdt objects. Call
cmd_force_checksrc after the compile rule, as the standard rule does.
Fixes: aacd149b6238 ("arm64: head: avoid relocating the kernel twice for KASLR")
Reviewed-by: Marc Zyngier <maz@kernel.org>
Signed-off-by: Fuad Tabba <fuad.tabba@linux.dev>
Link: https://patch.msgid.link/20260901140326.3812068-7-fuad.tabba@linux.dev
Signed-off-by: Marc Zyngier <maz@kernel.org>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux
Pull arm64 updates from Catalin Marinas:
"A quick summary: perf support for Branch Record Buffer Extensions
(BRBE), typical PMU hardware updates, small additions to MTE for
store-only tag checking and exposing non-address bits to signal
handlers, HAVE_LIVEPATCH enabled on arm64, VMAP_STACK forced on.
There is also a TLBI optimisation on hardware that does not require
break-before-make when changing the user PTEs between contiguous and
non-contiguous.
More details:
Perf and PMU updates:
- Add support for new (v3) Hisilicon SLLC and DDRC PMUs
- Add support for Arm-NI PMU integrations that share interrupts
between clock domains within a given instance
- Allow SPE to be configured with a lower sample period than the
minimum recommendation advertised by PMSIDR_EL1.Interval
- Add suppport for Arm's "Branch Record Buffer Extension" (BRBE)
- Adjust the perf watchdog period according to cpu frequency changes
- Minor driver fixes and cleanups
Hardware features:
- Support for MTE store-only checking (FEAT_MTE_STORE_ONLY)
- Support for reporting the non-address bits during a synchronous MTE
tag check fault (FEAT_MTE_TAGGED_FAR)
- Optimise the TLBI when folding/unfolding contiguous PTEs on
hardware with FEAT_BBM (break-before-make) level 2 and no TLB
conflict aborts
Software features:
- Enable HAVE_LIVEPATCH after implementing arch_stack_walk_reliable()
and using the text-poke API for late module relocations
- Force VMAP_STACK always on and change arm64_efi_rt_init() to use
arch_alloc_vmap_stack() in order to avoid KASAN false positives
ACPI:
- Improve SPCR handling and messaging on systems lacking an SPCR
table
Debug:
- Simplify the debug exception entry path
- Drop redundant DBG_MDSCR_* macros
Kselftests:
- Cleanups and improvements for SME, SVE and FPSIMD tests
Miscellaneous:
- Optimise loop to reduce redundant operations in contpte_ptep_get()
- Remove ISB when resetting POR_EL0 during signal handling
- Mark the kernel as tainted on SEA and SError panic
- Remove redundant gcs_free() call"
* tag 'arm64-upstream' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux: (93 commits)
arm64/gcs: task_gcs_el0_enable() should use passed task
arm64: Kconfig: Keep selects somewhat alphabetically ordered
arm64: signal: Remove ISB when resetting POR_EL0
kselftest/arm64: Handle attempts to disable SM on SME only systems
kselftest/arm64: Fix SVE write data generation for SME only systems
kselftest/arm64: Test SME on SME only systems in fp-ptrace
kselftest/arm64: Test FPSIMD format data writes via NT_ARM_SVE in fp-ptrace
kselftest/arm64: Allow sve-ptrace to run on SME only systems
arm64/mm: Drop redundant addr increment in set_huge_pte_at()
kselftest/arm4: Provide local defines for AT_HWCAP3
arm64: Mark kernel as tainted on SAE and SError panic
arm64/gcs: Don't call gcs_free() when releasing task_struct
drivers/perf: hisi: Support PMUs with no interrupt
drivers/perf: hisi: Relax the event number check of v2 PMUs
drivers/perf: hisi: Add support for HiSilicon SLLC v3 PMU driver
drivers/perf: hisi: Use ACPI driver_data to retrieve SLLC PMU information
drivers/perf: hisi: Add support for HiSilicon DDRC v3 PMU driver
drivers/perf: hisi: Simplify the probe process for each DDRC version
perf/arm-ni: Support sharing IRQs within an NI instance
perf/arm-ni: Consolidate CPU affinity handling
...
|
|
In preparation for adding Clang sanitizer coverage stack depth tracking
that can support stack depth callbacks:
- Add the new top-level CONFIG_KSTACK_ERASE option which will be
implemented either with the stackleak GCC plugin, or with the Clang
stack depth callback support.
- Rename CONFIG_GCC_PLUGIN_STACKLEAK as needed to CONFIG_KSTACK_ERASE,
but keep it for anything specific to the GCC plugin itself.
- Rename all exposed "STACKLEAK" names and files to "KSTACK_ERASE" (named
for what it does rather than what it protects against), but leave as
many of the internals alone as possible to avoid even more churn.
While here, also split "prev_lowest_stack" into CONFIG_KSTACK_ERASE_METRICS,
since that's the only place it is referenced from.
Suggested-by: Ingo Molnar <mingo@kernel.org>
Link: https://lore.kernel.org/r/20250717232519.2984886-1-kees@kernel.org
Signed-off-by: Kees Cook <kees@kernel.org>
|
|
%.pi.o files are built as prerequisites of other objects.
There is no need to use extra-y, which is planned for deprecation.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Link: https://lore.kernel.org/r/20250602180937.528459-1-masahiroy@kernel.org
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
|
|
Now Kbuild provides reasonable defaults for objtool, sanitizers, and
profilers.
Remove redundant variables.
Note:
This commit changes the coverage for some objects:
- include arch/mips/vdso/vdso-image.o into UBSAN, GCOV, KCOV
- include arch/sparc/vdso/vdso-image-*.o into UBSAN
- include arch/sparc/vdso/vma.o into UBSAN
- include arch/x86/entry/vdso/extable.o into KASAN, KCSAN, UBSAN, GCOV, KCOV
- include arch/x86/entry/vdso/vdso-image-*.o into KASAN, KCSAN, UBSAN, GCOV, KCOV
- include arch/x86/entry/vdso/vdso32-setup.o into KASAN, KCSAN, UBSAN, GCOV, KCOV
- include arch/x86/entry/vdso/vma.o into GCOV, KCOV
- include arch/x86/um/vdso/vma.o into KASAN, GCOV, KCOV
I believe these are positive effects because all of them are kernel
space objects.
Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
Tested-by: Roberto Sassu <roberto.sassu@huawei.com>
|
|
The asm code that creates the initial ID map is rather intricate and
hard to follow. This is problematic because it makes adding support for
things like LPA2 or WXN more difficult than necessary. Also, it is
parameterized like the rest of the MM code to run with a configurable
number of levels, which is rather pointless, given that all AArch64 CPUs
implement support for 48-bit virtual addressing, and that many systems
exist with DRAM located outside of the 39-bit addressable range, which
is the only smaller VA size that is widely used, and we need additional
tricks to make things work in that combination.
So let's bite the bullet, and rip out all the asm macros, and fiddly
code, and replace it with a C implementation based on the newly added
routines for creating the early kernel VA mappings. And while at it,
create the initial ID map based on 48-bit virtual addressing as well,
regardless of the number of configured levels for the kernel proper.
Note that this code may execute with the MMU and caches disabled, and is
therefore not permitted to make unaligned accesses. This shouldn't
generally happen in any case for the algorithm as implemented, but to be
sure, let's pass -mstrict-align to the compiler just in case.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20240214122845.2033971-66-ardb+git@google.com
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
|
|
The asm version of the kernel mapping code works fine for creating a
coarse grained identity map, but for mapping the kernel down to its
exact boundaries with the right attributes, it is not suitable. This is
why we create a preliminary RWX kernel mapping first, and then rebuild
it from scratch later on.
So let's reimplement this in C, in a way that will make it unnecessary
to create the kernel page tables yet another time in paging_init().
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20240214122845.2033971-63-ardb+git@google.com
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
|
|
Once we update the early kernel mapping code to only map the kernel once
with the right permissions, we can no longer perform code patching via
this mapping.
So move this code to an earlier stage of the boot, right after applying
the relocations.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20240214122845.2033971-54-ardb+git@google.com
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
|
|
We will want to parse the ID register overrides even earlier, so that we
can take them into account before creating the kernel mapping. So
migrate the code and make it work in the context of the early C runtime.
We will move the invocation to an earlier stage in a subsequent patch.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20240214122845.2033971-49-ardb+git@google.com
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
|
|
Now that we have a mini C runtime before the kernel mapping is up, we
can move the non-trivial relocation processing code out of head.S and
reimplement it in C.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20240214122845.2033971-48-ardb+git@google.com
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
|
|
We will add some code under pi/ that contains global variables that
should not end up in __initdata, as they will not be writable via the
initial ID map. So only rely on objcopy for making the libfdt code
__init, and use explicit annotations for the rest.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20240214122845.2033971-47-ardb+git@google.com
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
|
|
The mini C runtime runs before relocations are processed, and so it
cannot rely on statically initialized pointer variables.
Add a check to ensure that such code does not get introduced by
accident, by going over the relocations in each object, identifying the
ones that operate on data sections that are part of the executable
image, and raising an error if any relocations of type R_AARCH64_ABS64
exist. Note that such relocations are permitted in other places (e.g.,
debug sections) and will never occur in compiler generated code sections
when using the small code model, so only check sections that have
SHF_ALLOC set and SHF_EXECINSTR cleared.
To accommodate cases where statically initialized symbol references are
unavoidable, introduce a special case for ELF input data sections that
have ".rodata.prel64" in their names, and in these cases, instead of
rejecting any encountered ABS64 relocations, convert them into PREL64
relocations, which don't require any runtime fixups. Note that the code
in question must still be modified to deal with this, as it needs to
convert the 64-bit signed offsets into absolute addresses before use.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20240214122845.2033971-46-ardb+git@google.com
Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
|
|
In subsequent patches, mark portions of the early C code will be marked
as __init. Unfortunarely, __init implies __latent_entropy, and this
would result in the early C code being instrumented in an unsafe manner.
Disable the latent entropy plugin for the early C code.
Acked-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20231129111555.3594833-44-ardb@google.com
Signed-off-by: Will Deacon <will@kernel.org>
|
|
Enable asynchronous unwind table generation for both the core kernel as
well as modules, and emit the resulting .eh_frame sections as init code
so we can use the unwind directives for code patching at boot or module
load time.
This will be used by dynamic shadow call stack support, which will rely
on code patching rather than compiler codegen to emit the shadow call
stack push and pop instructions.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Reviewed-by: Sami Tolvanen <samitolvanen@google.com>
Tested-by: Sami Tolvanen <samitolvanen@google.com>
Link: https://lore.kernel.org/r/20221027155908.1940624-2-ardb@kernel.org
Signed-off-by: Will Deacon <will@kernel.org>
|
|
Currently, when KASLR is in effect, we set up the kernel virtual address
space twice: the first time, the KASLR seed is looked up in the device
tree, and the kernel virtual mapping is torn down and recreated again,
after which the relocations are applied a second time. The latter step
means that statically initialized global pointer variables will be reset
to their initial values, and to ensure that BSS variables are not set to
values based on the initial translation, they are cleared again as well.
All of this is needed because we need the command line (taken from the
DT) to tell us whether or not to randomize the virtual address space
before entering the kernel proper. However, this code has expanded
little by little and now creates global state unrelated to the virtual
randomization of the kernel before the mapping is torn down and set up
again, and the BSS cleared for a second time. This has created some
issues in the past, and it would be better to avoid this little dance if
possible.
So instead, let's use the temporary mapping of the device tree, and
execute the bare minimum of code to decide whether or not KASLR should
be enabled, and what the seed is. Only then, create the virtual kernel
mapping, clear BSS, etc and proceed as normal. This avoids the issues
around inconsistent global state due to BSS being cleared twice, and is
generally more maintainable, as it permits us to defer all the remaining
DT parsing and KASLR initialization to a later time.
This means the relocation fixup code runs only a single time as well,
allowing us to simplify the RELR handling code too, which is not
idempotent and was therefore required to keep track of the offset that
was applied the first time around.
Note that this means we have to clone a pair of FDT library objects, so
that we can control how they are built - we need the stack protector
and other instrumentation disabled so that the code can tolerate being
called this early. Note that only the kernel page tables and the
temporary stack are mapped read-write at this point, which ensures that
the early code does not modify any global state inadvertently.
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20220624150651.1358849-21-ardb@kernel.org
Signed-off-by: Will Deacon <will@kernel.org>
|
