efi/libstub: Add generic support for parsing mem_encrypt=

commit 7205f06e847422b66c1506eee01b9998ffc75d76 upstream.

Parse the mem_encrypt= command line parameter from the EFI stub if
CONFIG_ARCH_HA

efi/libstub: Add generic support for parsing mem_encrypt=

commit 7205f06e847422b66c1506eee01b9998ffc75d76 upstream.

Parse the mem_encrypt= command line parameter from the EFI stub if
CONFIG_ARCH_HAS_MEM_ENCRYPT=y, so that it can be passed to the early
boot code by the arch code in the stub.

This avoids the need for the core kernel to do any string parsing very
early in the boot.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Tested-by: Tom Lendacky <thomas.lendacky@amd.com>
Link: https://lore.kernel.org/r/20240227151907.387873-16-ardb+git@google.com
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

show more ...

x86/efistub: Avoid placing the kernel below LOAD_PHYSICAL_ADDR

[ Upstream commit 2f77465b05b1270c832b5e2ee27037672ad2a10a ]

The EFI stub's kernel placement logic randomizes the physical placement
o

x86/efistub: Avoid placing the kernel below LOAD_PHYSICAL_ADDR

[ Upstream commit 2f77465b05b1270c832b5e2ee27037672ad2a10a ]

The EFI stub's kernel placement logic randomizes the physical placement
of the kernel by taking all available memory into account, and picking a
region at random, based on a random seed.

When KASLR is disabled, this seed is set to 0x0, and this results in the
lowest available region of memory to be selected for loading the kernel,
even if this is below LOAD_PHYSICAL_ADDR. Some of this memory is
typically reserved for the GFP_DMA region, to accommodate masters that
can only access the first 16 MiB of system memory.

Even if such devices are rare these days, we may still end up with a
warning in the kernel log, as reported by Tom:

swapper/0: page allocation failure: order:10, mode:0xcc1(GFP_KERNEL|GFP_DMA), nodemask=(null),cpuset=/,mems_allowed=0

Fix this by tweaking the random allocation logic to accept a low bound
on the placement, and set it to LOAD_PHYSICAL_ADDR.

Fixes: a1b87d54f4e4 ("x86/efistub: Avoid legacy decompressor when doing EFI boot")
Reported-by: Tom Englund <tomenglund26@gmail.com>
Closes: https://bugzilla.kernel.org/show_bug.cgi?id=218404
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>

show more ...

arm64: libstub: Move KASLR handling functions to kaslr.c

This prepares for riscv to use the same functions to handle the pĥysical
kernel move when KASLR is enabled.

Signed-off-by: Alexandre Ghiti <

arm64: libstub: Move KASLR handling functions to kaslr.c

This prepares for riscv to use the same functions to handle the pĥysical
kernel move when KASLR is enabled.

Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Acked-by: Ard Biesheuvel <ardb@kernel.org>
Tested-by: Conor Dooley <conor.dooley@microchip.com>
Tested-by: Song Shuai <songshuaishuai@tinylab.org>
Reviewed-by: Sami Tolvanen <samitolvanen@google.com>
Tested-by: Sami Tolvanen <samitolvanen@google.com>
Link: https://lore.kernel.org/r/20230722123850.634544-4-alexghiti@rivosinc.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>

show more ...

efi/libstub: Add limit argument to efi_random_alloc()

x86 will need to limit the kernel memory allocation to the lowest 512
MiB of memory, to match the behavior of the existing bare metal KASLR
phys

efi/libstub: Add limit argument to efi_random_alloc()

x86 will need to limit the kernel memory allocation to the lowest 512
MiB of memory, to match the behavior of the existing bare metal KASLR
physical randomization logic. So in preparation for that, add a limit
parameter to efi_random_alloc() and wire it up.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Link: https://lore.kernel.org/r/20230807162720.545787-22-ardb@kernel.org

show more ...

x86/efistub: Perform 4/5 level paging switch from the stub

In preparation for updating the EFI stub boot flow to avoid the bare
metal decompressor code altogether, implement the support code for
swi

x86/efistub: Perform 4/5 level paging switch from the stub

In preparation for updating the EFI stub boot flow to avoid the bare
metal decompressor code altogether, implement the support code for
switching between 4 and 5 levels of paging before jumping to the kernel
proper.

Reuse the newly refactored trampoline that the bare metal decompressor
uses, but relies on EFI APIs to allocate 32-bit addressable memory and
remap it with the appropriate permissions. Given that the bare metal
decompressor will no longer call into the trampoline if the number of
paging levels is already set correctly, it is no longer needed to remove
NX restrictions from the memory range where this trampoline may end up.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Acked-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Link: https://lore.kernel.org/r/20230807162720.545787-17-ardb@kernel.org

show more ...

efi/libstub: Implement support for unaccepted memory

UEFI Specification version 2.9 introduces the concept of memory
acceptance: Some Virtual Machine platforms, such as Intel TDX or AMD
SEV-SNP, req

efi/libstub: Implement support for unaccepted memory

UEFI Specification version 2.9 introduces the concept of memory
acceptance: Some Virtual Machine platforms, such as Intel TDX or AMD
SEV-SNP, requiring memory to be accepted before it can be used by the
guest. Accepting happens via a protocol specific for the Virtual
Machine platform.

Accepting memory is costly and it makes VMM allocate memory for the
accepted guest physical address range. It's better to postpone memory
acceptance until memory is needed. It lowers boot time and reduces
memory overhead.

The kernel needs to know what memory has been accepted. Firmware
communicates this information via memory map: a new memory type --
EFI_UNACCEPTED_MEMORY -- indicates such memory.

Range-based tracking works fine for firmware, but it gets bulky for
the kernel: e820 (or whatever the arch uses) has to be modified on every
page acceptance. It leads to table fragmentation and there's a limited
number of entries in the e820 table.

Another option is to mark such memory as usable in e820 and track if the
range has been accepted in a bitmap. One bit in the bitmap represents a
naturally aligned power-2-sized region of address space -- unit.

For x86, unit size is 2MiB: 4k of the bitmap is enough to track 64GiB or
physical address space.

In the worst-case scenario -- a huge hole in the middle of the
address space -- It needs 256MiB to handle 4PiB of the address
space.

Any unaccepted memory that is not aligned to unit_size gets accepted
upfront.

The bitmap is allocated and constructed in the EFI stub and passed down
to the kernel via EFI configuration table. allocate_e820() allocates the
bitmap if unaccepted memory is present, according to the size of
unaccepted region.

Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20230606142637.5171-4-kirill.shutemov@linux.intel.com

show more ...

efi: fix missing prototype warnings

The cper.c file needs to include an extra header, and efi_zboot_entry
needs an extern declaration to avoid these 'make W=1' warnings:

drivers/firmware/efi/libstu

efi: fix missing prototype warnings

The cper.c file needs to include an extra header, and efi_zboot_entry
needs an extern declaration to avoid these 'make W=1' warnings:

drivers/firmware/efi/libstub/zboot.c:65:1: error: no previous prototype for 'efi_zboot_entry' [-Werror=missing-prototypes]
drivers/firmware/efi/efi.c:176:16: error: no previous prototype for 'efi_attr_is_visible' [-Werror=missing-prototypes]
drivers/firmware/efi/cper.c:626:6: error: no previous prototype for 'cper_estatus_print' [-Werror=missing-prototypes]
drivers/firmware/efi/cper.c:649:5: error: no previous prototype for 'cper_estatus_check_header' [-Werror=missing-prototypes]
drivers/firmware/efi/cper.c:662:5: error: no previous prototype for 'cper_estatus_check' [-Werror=missing-prototypes]

To make this easier, move the cper specific declarations to
include/linux/cper.h.

Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Acked-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>

show more ...

efi/zboot: arm64: Grab code size from ELF symbol in payload

Instead of relying on a dodgy dd hack to copy the image code size from
the uncompressed image's PE header to the end of the compressed ima

efi/zboot: arm64: Grab code size from ELF symbol in payload

Instead of relying on a dodgy dd hack to copy the image code size from
the uncompressed image's PE header to the end of the compressed image,
let's grab the code size from the symbol that is injected into the ELF
object by the Kbuild rules that generate the compressed payload.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
Acked-by: Mark Rutland <mark.rutland@arm.com>

show more ...

efi/libstub: Use relocated version of kernel's struct screen_info

In some cases, we expose the kernel's struct screen_info to the EFI stub
directly, so it gets populated before even entering the ker

efi/libstub: Use relocated version of kernel's struct screen_info

In some cases, we expose the kernel's struct screen_info to the EFI stub
directly, so it gets populated before even entering the kernel. This
means the early console is available as soon as the early param parsing
happens, which is nice. It also means we need two different ways to pass
this information, as this trick only works if the EFI stub is baked into
the core kernel image, which is not always the case.

Huacai reports that the preparatory refactoring that was needed to
implement this alternative method for zboot resulted in a non-functional
efifb earlycon for other cases as well, due to the reordering of the
kernel image relocation with the population of the screen_info struct,
and the latter now takes place after copying the image to its new
location, which means we copy the old, uninitialized state.

So let's ensure that the same-image version of alloc_screen_info()
produces the correct screen_info pointer, by taking the displacement of
the loaded image into account.

Reported-by: Huacai Chen <chenhuacai@loongson.cn>
Tested-by: Huacai Chen <chenhuacai@loongson.cn>
Link: https://lore.kernel.org/linux-efi/20230310021749.921041-1-chenhuacai@loongson.cn/
Fixes: 42c8ea3dca094ab8 ("efi: libstub: Factor out EFI stub entrypoint into separate file")
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>

show more ...

efi/libstub: smbios: Drop unused 'recsize' parameter

We no longer use the recsize argument for locating the string table in
an SMBIOS record, so we can drop it from the internal API.

Signed-off-by:

efi/libstub: smbios: Drop unused 'recsize' parameter

We no longer use the recsize argument for locating the string table in
an SMBIOS record, so we can drop it from the internal API.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>

show more ...

arm64: efi: Use SMBIOS processor version to key off Ampere quirk

Instead of using the SMBIOS type 1 record 'family' field, which is often
modified by OEMs, use the type 4 'processor ID' and 'process

arm64: efi: Use SMBIOS processor version to key off Ampere quirk

Instead of using the SMBIOS type 1 record 'family' field, which is often
modified by OEMs, use the type 4 'processor ID' and 'processor version'
fields, which are set to a small set of probe-able values on all known
Ampere EFI systems in the field.

Fixes: 550b33cfd4452968 ("arm64: efi: Force the use of ...")
Tested-by: Andrea Righi <andrea.righi@canonical.com>
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>

show more ...

efi: zboot: Use EFI protocol to remap code/data with the right attributes

Use the recently introduced EFI_MEMORY_ATTRIBUTES_PROTOCOL in the zboot
implementation to set the right attributes for the c

efi: zboot: Use EFI protocol to remap code/data with the right attributes

Use the recently introduced EFI_MEMORY_ATTRIBUTES_PROTOCOL in the zboot
implementation to set the right attributes for the code and data
sections of the decompressed image, i.e., EFI_MEMORY_RO for code and
EFI_MEMORY_XP for data.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>

show more ...

efi/libstub: Add memory attribute protocol definitions

EFI_MEMORY_ATTRIBUTE_PROTOCOL servers as a better alternative to
DXE services for setting memory attributes in EFI Boot Services
environment. T

efi/libstub: Add memory attribute protocol definitions

EFI_MEMORY_ATTRIBUTE_PROTOCOL servers as a better alternative to
DXE services for setting memory attributes in EFI Boot Services
environment. This protocol is better since it is a part of UEFI
specification itself and not UEFI PI specification like DXE
services.

Add EFI_MEMORY_ATTRIBUTE_PROTOCOL definitions.
Support mixed mode properly for its calls.

Tested-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Evgeniy Baskov <baskov@ispras.ru>
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>

show more ...

arm64: efi: Limit allocations to 48-bit addressable physical region

The UEFI spec does not mention or reason about the configured size of
the virtual address space at all, but it does mention that a

arm64: efi: Limit allocations to 48-bit addressable physical region

The UEFI spec does not mention or reason about the configured size of
the virtual address space at all, but it does mention that all memory
should be identity mapped using a page size of 4 KiB.

This means that a LPA2 capable system that has any system memory outside
of the 48-bit addressable physical range and follows the spec to the
letter may serve page allocation requests from regions of memory that
the kernel cannot access unless it was built with LPA2 support and
enables it at runtime.

So let's ensure that all page allocations are limited to the 48-bit
range.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>

show more ...

efi: random: combine bootloader provided RNG seed with RNG protocol output

Instead of blindly creating the EFI random seed configuration table if
the RNG protocol is implemented and works, check whe

efi: random: combine bootloader provided RNG seed with RNG protocol output

Instead of blindly creating the EFI random seed configuration table if
the RNG protocol is implemented and works, check whether such a EFI
configuration table was provided by an earlier boot stage and if so,
concatenate the existing and the new seeds, leaving it up to the core
code to mix it in and credit it the way it sees fit.

This can be used for, e.g., systemd-boot, to pass an additional seed to
Linux in a way that can be consumed by the kernel very early. In that
case, the following definitions should be used to pass the seed to the
EFI stub:

struct linux_efi_random_seed {
u32 size; // of the 'seed' array in bytes
u8 seed[];
};

The memory for the struct must be allocated as EFI_ACPI_RECLAIM_MEMORY
pool memory, and the address of the struct in memory should be installed
as a EFI configuration table using the following GUID:

LINUX_EFI_RANDOM_SEED_TABLE_GUID 1ce1e5bc-7ceb-42f2-81e5-8aadf180f57b

Note that doing so is safe even on kernels that were built without this
patch applied, but the seed will simply be overwritten with a seed
derived from the EFI RNG protocol, if available. The recommended seed
size is 32 bytes, and seeds larger than 512 bytes are considered
corrupted and ignored entirely.

In order to preserve forward secrecy, seeds from previous bootloaders
are memzero'd out, and in order to preserve memory, those older seeds
are also freed from memory. Freeing from memory without first memzeroing
is not safe to do, as it's possible that nothing else will ever
overwrite those pages used by EFI.

Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com>
[ardb: incorporate Jason's followup changes to extend the maximum seed
size on the consumer end, memzero() it and drop a needless printk]
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>

show more ...

efi: libstub: Add mixed mode support to command line initrd loader

Now that we have support for calling protocols that need additional
marshalling for mixed mode, wire up the initrd command line loa

efi: libstub: Add mixed mode support to command line initrd loader

Now that we have support for calling protocols that need additional
marshalling for mixed mode, wire up the initrd command line loader.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>

show more ...

efi: libstub: Permit mixed mode return types other than efi_status_t

Rework the EFI stub macro wrappers around protocol method calls and
other indirect calls in order to allow return types other tha

efi: libstub: Permit mixed mode return types other than efi_status_t

Rework the EFI stub macro wrappers around protocol method calls and
other indirect calls in order to allow return types other than
efi_status_t. This means the widening should be conditional on whether
or not the return type is efi_status_t, and should be omitted otherwise.

Also, switch to _Generic() to implement the type based compile time
conditionals, which is more concise, and distinguishes between
efi_status_t and u64 properly.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>

show more ...

efi: libstub: Implement devicepath support for initrd commandline loader

Currently, the initrd= command line option to the EFI stub only supports
loading files that reside on the same volume as the

efi: libstub: Implement devicepath support for initrd commandline loader

Currently, the initrd= command line option to the EFI stub only supports
loading files that reside on the same volume as the loaded image, which
is not workable for loaders like GRUB that don't even implement the
volume abstraction (EFI_SIMPLE_FILE_SYSTEM_PROTOCOL), and load the
kernel from an anonymous buffer in memory. For this reason, another
method was devised that relies on the LoadFile2 protocol.

However, the command line loader is rather useful when using the UEFI
shell or other generic loaders that have no awareness of Linux specific
protocols so let's make it a bit more flexible, by permitting textual
device paths to be provided to initrd= as well, provided that they refer
to a file hosted on a EFI_SIMPLE_FILE_SYSTEM_PROTOCOL volume. E.g.,

initrd=PciRoot(0x0)/Pci(0x3,0x0)/HD(1,MBR,0xBE1AFDFA,0x3F,0xFBFC1)/rootfs.cpio.gz

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>

show more ...

efi: libstub: use EFI_LOADER_CODE region when moving the kernel in memory

The EFI spec is not very clear about which permissions are being given
when allocating pages of a certain type. However, it

efi: libstub: use EFI_LOADER_CODE region when moving the kernel in memory

The EFI spec is not very clear about which permissions are being given
when allocating pages of a certain type. However, it is quite obvious
that EFI_LOADER_CODE is more likely to permit execution than
EFI_LOADER_DATA, which becomes relevant once we permit booting the
kernel proper with the firmware's 1:1 mapping still active.

Ostensibly, recent systems such as the Surface Pro X grant executable
permissions to EFI_LOADER_CODE regions but not EFI_LOADER_DATA regions.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>

show more ...

arm64: efi: Force the use of SetVirtualAddressMap() on Altra machines

Ampere Altra machines are reported to misbehave when the SetTime() EFI
runtime service is called after ExitBootServices() but be

arm64: efi: Force the use of SetVirtualAddressMap() on Altra machines

Ampere Altra machines are reported to misbehave when the SetTime() EFI
runtime service is called after ExitBootServices() but before calling
SetVirtualAddressMap(). Given that the latter is horrid, pointless and
explicitly documented as optional by the EFI spec, we no longer invoke
it at boot if the configured size of the VA space guarantees that the
EFI runtime memory regions can remain mapped 1:1 like they are at boot
time.

On Ampere Altra machines, this results in SetTime() calls issued by the
rtc-efi driver triggering synchronous exceptions during boot. We can
now recover from those without bringing down the system entirely, due to
commit 23715a26c8d81291 ("arm64: efi: Recover from synchronous
exceptions occurring in firmware"). However, it would be better to avoid
the issue entirely, given that the firmware appears to remain in a funny
state after this.

So attempt to identify these machines based on the 'family' field in the
type #1 SMBIOS record, and call SetVirtualAddressMap() unconditionally
in that case.

Tested-by: Alexandru Elisei <alexandru.elisei@gmail.com>
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>

show more ...

efi/arm64: libstub: Split off kernel image relocation for builtin stub

The arm64 build of the EFI stub is part of the core kernel image, and
therefore accesses section markers directly when it needs

efi/arm64: libstub: Split off kernel image relocation for builtin stub

The arm64 build of the EFI stub is part of the core kernel image, and
therefore accesses section markers directly when it needs to figure out
the size of the various section.

The zboot decompressor does not have access to those symbols, but
doesn't really need that either. So let's move handle_kernel_image()
into a separate file (or rather, move everything else into a separate
file) so that the zboot build does not pull in unused code that links to
symbols that it does not define.

While at it, introduce a helper routine that the generic zboot loader
will need to invoke after decompressing the image but before invoking
it, to ensure that the I-side view of memory is consistent.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>

show more ...

efi: libstub: Factor out EFI stub entrypoint into separate file

In preparation for allowing the EFI zboot decompressor to reuse most of
the EFI stub machinery, factor out the actual EFI PE/COFF entr

efi: libstub: Factor out EFI stub entrypoint into separate file

In preparation for allowing the EFI zboot decompressor to reuse most of
the EFI stub machinery, factor out the actual EFI PE/COFF entrypoint
into a separate file.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>

show more ...

efi: libstub: Move screen_info handling to common code

Currently, arm64, RISC-V and LoongArch rely on the fact that struct
screen_info can be accessed directly, due to the fact that the EFI stub
and

efi: libstub: Move screen_info handling to common code

Currently, arm64, RISC-V and LoongArch rely on the fact that struct
screen_info can be accessed directly, due to the fact that the EFI stub
and the core kernel are part of the same image. This will change after a
future patch, so let's ensure that the screen_info handling is able to
deal with this, by adopting the arm32 approach of passing it as a
configuration table. While at it, switch to ACPI reclaim memory to hold
the screen_info data, which is more appropriate for this kind of
allocation.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>

show more ...

efi/libstub: measure EFI LoadOptions

The EFI TCG spec, in §10.2.6 "Measuring UEFI Variables and UEFI GPT
Data", only reasons about the load options passed to a loaded image in
the context of boot op

efi/libstub: measure EFI LoadOptions

The EFI TCG spec, in §10.2.6 "Measuring UEFI Variables and UEFI GPT
Data", only reasons about the load options passed to a loaded image in
the context of boot options booted directly from the BDS, which are
measured into PCR #5 along with the rest of the Boot#### EFI variable.

However, the UEFI spec mentions the following in the documentation of
the LoadImage() boot service and the EFI_LOADED_IMAGE protocol:

The caller may fill in the image’s "load options" data, or add
additional protocol support to the handle before passing control to
the newly loaded image by calling EFI_BOOT_SERVICES.StartImage().

The typical boot sequence for Linux EFI systems is to load GRUB via a
boot option from the BDS, which [hopefully] calls LoadImage to load the
kernel image, passing the kernel command line via the mechanism
described above. This means that we cannot rely on the firmware
implementing TCG measured boot to ensure that the kernel command line
gets measured before the image is started, so the EFI stub will have to
take care of this itself.

Given that PCR #5 has an official use in the TCG measured boot spec,
let's avoid it in this case. Instead, add a measurement in PCR #9 (which
we already use for our initrd) and extend it with the LoadOptions
measurements

Co-developed-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
Signed-off-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>

show more ...

efi: libstub: install boot-time memory map as config table

Expose the EFI boot time memory map to the kernel via a configuration
table. This is arch agnostic and enables future changes that remove t

efi: libstub: install boot-time memory map as config table

Expose the EFI boot time memory map to the kernel via a configuration
table. This is arch agnostic and enables future changes that remove the
dependency on DT on architectures that don't otherwise rely on it.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>

show more ...