Just another blog (Posts about linux)

Secure Boot on Gentoo with shim & GRUB

Gabriele Svelto — Thu, 18 Jul 2024 15:38:23 GMT

Getting Secure Boot to work on Gentoo has traditionally been tricky, due to the widespread use of custom kernels and the absence of pre-signed boot loaders like those used by the mainstream binary Linux distributions. Since the required information is spread through the handbook and the wiki I decided to write one easy-to-follow tutorial instead, in order to make this information a bit more accessible.

There are several ways to make Secure Boot work, the goal being that every executable loaded by the system during boot is signed and can be verified by the one loading it. The method I've chosen uses the shim bootloader to launch a standalone installation of GRUB.

The way the boot chain will work is the following:

Your machine's UEFI firmware will load the shim bootloader, verifying its signature using the pre-loaded Microsoft-provided key.
The shim bootloader will load a GRUB standalone executable which contains everything needed by GRUB to run: its modules, configuration file, fonts and themes. This executable will be signed with a key we'll generate and load into the Machine Key Owner list, a user-managed list of keys. The shim bootloader will also set GRUB's shim_lock option which will inform GRUB to verify all the files it loads.
The GRUB standalone executable will thus launch a signed Linux kernel. This will either be a Gentoo binary distribution kernel - in which case we'll also load Gentoo binary distribution key in the MOK to verify it - or a custom kernel which will be signed with the same key we'll have used to sign GRUB.
The Linux kernel will optionally enforce that the modules it loads are also signed.
In a dual-boot system GRUB will also be able to chain-load the Microsoft Windows bootloader or other signed UEFI executables, never breaking the Secure Boot chain.

Note that you can follow this procedure in place of following the Configuring the bootloader chapter of the Gentoo handbook, or do it on an already existing installation. You don't need to turn off Secure Boot for the procedure to work. In fact, if you're installing Gentoo using a live distribution that supports Secure Boot, you can do the entire installation without ever turning it off.

Preparing the system

First of all we need to mount the EFI boot partition. This is a FAT-formatted partition that you'll have made during the partitioning step of a Gentoo installation, or was already present if you're installing Gentoo alongside Windows or another Linux distribution using UEFI boot. This guide assumes that this partition will be mounted under the /boot/efi mount-point, so you'll have something like this in /etc/fstab:

/dev/sda1       /boot/efi       vfat            defaults            0 0

Go on and mount the partition if it hasn't been mounted already:

# mount /boot/efi

Setting up the signing keys

It's now time to generate the keys that we'll use to sign GRUB. We'll generate an RSA-2048 certificate in PEM format which will be used to sign GRUB (as well as the kernel and its modules if you're building it from source):

# openssl req -new -nodes -utf8 -sha256 -x509 -outform PEM \
    -out /root/secureboot/MOK.pem -keyout /root/secureboot/MOK.pem \
    -subj "/CN=<your name here>/"

Note that it is good practice to keep this certificate offline, but for simplicity this guide assumes that it is under /root/secureboot/. You can always move it to a removable drive later.

Now we also need the certificate in binary DER format. This version of the certificate will be loaded into the MOK list.

# openssl x509 -in /root/secureboot/MOK.pem -outform DER -out /root/secureboot/MOK.cer

Now modify your make.conf so that the secureboot USE flag is enabled and the SECUREBOOT_SIGN_KEY, SECUREBOOT_SIGN_CERT, MODULES_SIGN_KEY and MODULES_SIGN_CERT variables point to the certificate in the PEM format.

# USE flags
USE=".. secureboot .."

# Secure Boot signing keys
SECUREBOOT_SIGN_KEY="/root/secureboot/MOK.pem"
SECUREBOOT_SIGN_CERT="/root/secureboot/MOK.pem"
MODULES_SIGN_KEY="/root/secureboot/MOK.pem"
MODULES_SIGN_CERT="/root/secureboot/MOK.pem"

Configuring and installing the required packages

Now let's install the packages we'll use: we need the sys-boot/efibootmgr package to add new boot entries, the sys-boot/mokutil package to load our keys into the Machine Owner Key list and the sys-boot/shim package that contains the signed shim bootloader. The sys-boot/mokutil package is currently marked as unstable so we'll unmask it first.

# echo "sys-boot/mokutil ~amd64" >> /etc/portage/package.accept_keywords
# echo emerge --ask sys-boot/efibootmgr sys-boot/mokutil sys-boot/shim

I also recommend rebuilding the sys-apps/kmod package with the pkcs7 USE flag, so that the modinfo command will show you the signatures in the kernel modules.

# echo "sys-apps/kmod pkcs7" >> /etc/portage/package.use
# emerge --ask --newuse --oneshot sys-apps/kmod

Installing the kernel

Now that the keys have been set up it's time to make sure that the kernel image is signed so that it can be verified by GRUB when loading it. The procedure is different depending on the type of kernel you're using.

If you're using Gentoo's binary distribution kernel (via the sys-kernel/gentoo-kernel-bin package) refer to the Gentoo binary distribution kernel instructions
If you're using a custom kernel built using the sys-kernel/gentoo-kernel package use the Gentoo kernel built from sources instructions
Finally if you're using a custom kernel built by hand (sys-kernel/gentoo-sources, sys-kernel/vanilla-sources or anything else that's completely built from source) use the Custom kernel instructions

Gentoo binary distribution kernel

Gentoo binary distribution kernels are already signed with Gentoo's binary distribution key. The public key, which we'll need to verify the signature, is installed alongside the kernel. So proceed to install it as usual:

# emerge --ask sys-kernel/gentoo-kernel-bin

Now we need to import the key in the MOK list. The key is stored under certs/signing_key.x509 in the kernel sources installation directory. To load it into the MOK list use the following command:

# mokutil --import /usr/src/linux-<version>-gentoo-dist/certs/signing_key.x509

mokutil will ask for a password to enroll the key. This will be used only once after you have rebooted the system during the enrollment process, it can be discarded afterwards.

If mokutil complains about the key already being in the keyring you can force it to be loaded like this (this might happen if a firmware update wiped the MOK list and you need to re-enroll the key).

# mokutil --ignore-keyring --import /usr/src/linux-<version>-gentoo-dist/certs/signing_key.x509

That's it, you can now move on to the Installing the shim and GRUB bootloaders section.

Gentoo kernel built from sources

When building the kernel using the sys-kernel/gentoo-kernel package the kernel will be automatically signed with the keys that we've set up in the Setting up the signing keys section. To sign the loadable modules too set the modules-sign USE flag in make.conf:

USE=".. modules-sign .."

This will instruct the sys-kernel/gentoo-kernel package to also sign all the modules using the aformentioned keys. Additionally, all packages that build third-party modules will automatically sign them.

Now install the kernel as usual:

# emerge --ask sys-kernel/gentoo-kernel

That's it, you can now move on to the Installing the shim and GRUB bootloaders section.

Custom kernel

When building your own kernel you'll need to specify the module signing key in your .config file and force signature checks on all loaded modules:

CONFIG_MODULE_SIG_FORCE=y
CONFIG_MODULE_SIG_KEY="/root/secureboot/MOK.pem"

Now build and install the kernel as usual

# make
# make install
# make modules_install

The modules will be signed automatically but the kernel image needs to be signed manually:

# sbsign --key /root/secureboot/MOK.key --cert /root/secureboot/MOK.crt /boot/vmlinuz-<version>

That's it, you can now move on to the Installing the shim and GRUB bootloaders section.

Installing the shim and GRUB bootloaders

Now let's add a folder for our bootloader to the EFI partition and move the shim bootloader inside of it.

# mkdir --parents /boot/efi/EFI/gentoo
# cp /usr/share/shim/mmx64.efi /boot/efi/EFI/gentoo/
# cp /usr/share/shim/BOOTX64.EFI /boot/efi/EFI/gentoo/

Notice how the shim bootloader is made up of two executables: the actual bootloader BOOTX64.EFI and a tool to manipulate the MOK list (mmx64.efi). The latter will be used during the first reboot to enroll our keys.

The next step is to install GRUB. Make sure that you've got UEFI support enabled by setting the GRUB_PLATFORMS variable in make.conf:

GRUB_PLATFORMS="efi-64"

If you want to dual-boot Windows - or other Linux distributions - enable the mount USE flag so that GRUB's OS prober will be able to find them.

# echo "sys-boot/grub mount" >> /etc/portage/package.use

Now install the GRUB package:

emerge --ask sys-boot/grub

Now adjust the /etc/default/grub configuration file with the options suitable for your machine. In case you want to dual-boot Windows you'll need to explicitly set the GRUB_DISABLE_OS_PROBER option:

# Enable OS prober
GRUB_DISABLE_OS_PROBER=false

We can now generate GRUB's configuration and the unsigned standalone bootloader executable. It's worth spending a few words on why we'll opt for a standalone bootloader instead of GRUB's traditional installation (done using the grub-install tool).

By default GRUB's configuration will use the shim_lock verifier. This is a mechanism that causes the shim bootloader to inform GRUB that it's being loaded in a Secure Boot environment and thus it's now GRUB's turn to validate the boot chain. When this happens GRUB will verify all the executable files it loads - including its modules. A traditional GRUB installation will store all these files separately and require a separate GPG signature for each of them.

Adding and maintaining these signatures is an unwieldy and error-prone excercise and requires a separate GPG key in addition to the ones we've already generated. A standalone installation on the other hand produces a single executable which contains a memdisk holding all the modules, fonts, themes as well as the configuration. This executable needs to be signed only once, greatly reducing the maintainance burden.

With that said let's proceed. We'll first generate GRUB's configuration using grub-mkconfig then use grub-mkstandalone to produce the actual bootloader:

# grub-mkconfig -o /boot/grub/grub.cfg
# grub-mkstandalone --output /boot/efi/EFI/gentoo/grubx64.efi \
    --directory /usr/lib/grub/x86_64-efi --sbat /usr/share/grub/sbat.csv \
    --format x86_64-efi "/boot/grub/grub.cfg=/boot/grub/grub.cfg"

It's worth breaking down the options we're passing to grub-mkstandalone to explain what's going on here:

--output specifies the path of the bootloader executable we'll be generating
--directory indicates where GRUB will find the modules that will be included in the bootloader executable, /usr/lib/grub/x86_64-efi is the default location for these modules on Gentoo
--sbat /usr/share/grub/sbat.csv specifies the Secure Boot Advanced Targeting metadata to use. This is a list of EFI progams and their minimum versions which are allowed to run. It's used to prevent outdated EFI progams - such as buggy/compromised bootloaders - from being used. Since we don't need to add anything there we use the default version that comes with Gentoo's GRUB installation.
--format x86_64-efi specifies the bootloader executable format, that is 64-bit EFI.
"/boot/grub/grub.cfg=/boot/grub/grub.cfg" will make grub-mkstandalone use the grub.cfg file we've generated to populate the /boot/grub/grub.cfg file contained in the bundled memdisk. The memdisk is a small tarball or SquashFS image that GRUB will use to store all the files that go into a regular installation. By default GRUB uses the /boot/grub prefix when installing these files, which is why it expects grub.cfg to be there.

At this point we've got ourselves an unsigned bootloader with everything we need to boot our system, time to sign it:

# sbsign --cert /root/secureboot/MOK.pem --key /root/secureboot/MOK.pem \
    --output /boot/efi/EFI/gentoo/grubx64.efi /boot/efi/EFI/gentoo/grubx64.efi

Note that we've placed the signed GRUB bootloader next to the shim bootloader. It needs to be called grubx64.efi because that's what the shim bootloader expects to find.

Importing the key in the MOK list

It's now time to import the key we've used to sign GRUB (and possibly the kernel too) into the MOK list. This is a two step process, the first part is to import the key:

# mokutil --import /root/secureboot/MOK.cer

mokutil will ask for a password to enroll the key. This will be used only once after you have rebooted the system during the enrollment process, it can be discarded afterwards.

# mokutil --ignore-keyring --import /root/secureboot/MOK.cer

The second part will happen upon the next reboot. Remember the mmx64.efi file we've put in the EFI parition alongside the shim bootloader? Upon being loaded, shim will notice that we're trying to import a key in the MOK and launch it to do the actual enrollment.

Creating a new EFI boot entry

The last step is to create a new EFI boot entry for the shim bootloader. Note that we don't need an entry for GRUB, because it will be loaded via SHIM.

The boot entry can be created with this command:

# efibootmgr --disk <disk_with_efi_partition> --part <partition_number> --create -L "shim" -l '\EFI\gentoo\BOOTX64.EFI'

disk_with_efi_partition is the disk which holds the EFI partition, and partition_number is the number of the EFI partition in the GPT table. So, if your EFI partition is on the /dev/sda1 device you'll create the boot entry with:

# efibootmgr --disk /dev/sda --part 1 --create -L "shim" -l '\EFI\gentoo\BOOTX64.EFI'

This command should also set the new boot entry as the default one, double-check it with:

# efibootmgr
BootCurrent: 0003
Timeout: 0 seconds
BootOrder: 0003,0000,0017,0018,0019,001A,001B,001C,001D,001E,001F,0024,0002
...
Boot0003* shim  HD(1,GPT,e8389b70-d497-40e7-94a5-0b4a48732aa0,0x800,0x82000)/File(\EFI\gentoo\BOOTX64.EFI)
...

Notice how the BootOrder variable starts with 0003 which corresponds to the shim entry in this example. If this is not the case adjust the boot order using:

# efibootmgr --bootorder <shim_entry>,<other_entry>,...

Reboot & enrolling the key in the MOK list

You can now reboot your machine. Upon the next reboot the shim bootloader will notice that you tried to enroll a new key and load the Shim UEFI key management executable. It will look like this (I apologize for the horrible Moiré artifacts on the pictures):

After pressing a key you'll be presented with the key management menu, choose the Enroll MOK entry:

You will presented with the list of keys you enrolled. One if you only added yours or two if you also added the Gentoo kernel distribution key. You can view the keys to ensure they're what you expect:

Here's the information about the selected key:

Once you've verified you're enrolling the proper keys choose Continue:

You will be asked if you want to enroll the key(s), choose Yes:

You will now be asked for the password you used when you enrolled the key using mokutil. Input the password, you won't need it anymore once this is done:

Once you've entered the password it's time to reboot:

Conclusion

You're done! Your machine should now reboot using the shim bootloader which will in turn load GRUB. If you had other operating systems such as Windows these should appear in the menu, as well as other UEFI executables like memtest86+ for example. Booting any of the entries should work and provide a validated boot chain. Once you've booted back into Gentoo you can always verify that this is the case by using mokutil again:

# mokutil --sb-state
SecureBoot enabled

Updating and troubleshooting

You might wonder about how to deal updates to the packages we've used when setting everything up. Here's how.

Kernel updates

If you're using a custom kernel you'll have to execute some of the steps above every time you install (or remove) a new kernel. In particular you'll have to regenerate the grub.cfg file, rebuild the GRUB bootloader standalone image and copy it over to your EFI partition.

If you're using a distribution kernel there's a simpler way. You can instruct the sys-kernel/installkernel package to regenerate the GRUB signed UEFI executable automatically every time a new kernel is installed by using a plug-in script. Here's instructions for installing them depending on the init system you're using. Be sure to adjust the GRUB_CFG, EFI_PARTITION and GRUB_UEFI environment variable if you're using different paths respectively for your grub.cfg file, EFI boot partition or the location of the GRUB EFI executable.

OpenRC

Copy the following script into /etc/kernel/postinst.d/92-grub-mkstandalone-secureboot.install then make it executable:

#!/usr/bin/env bash

# Copyright 2024-2025 Gabriele Svelto

# This script must be installed under /etc/kernel/postinst.d, it will
# run after the 91-grub-mkconfig.install script and generate a signed
# stand-alone GRUB image suitable for booting on UEFI systems with
# Secure Boot enabled.
#
# This script is meant to be executed by the traditional installkernel
# tool and will run only when the systemd USE flag is disabled or
# SYSTEMD_KERNEL_INSTALL=0 is set in the environment.

: "${GRUB_CFG:=/boot/grub/grub.cfg}"
: "${EFI_PARTITION:=/boot/efi}"
: "${GRUB_UEFI:=/boot/efi/EFI/gentoo/grubx64.efi}"

# familiar helpers, we intentionally don't use Gentoo functions.sh
die() {
  echo -e " ${NOCOLOR-\e[1;31m*\e[0m }${*}" >&2
  exit 1
}

einfo() {
  echo -e " ${NOCOLOR-\e[1;32m*\e[0m }${*}" >&2
}

ewarn() {
  echo -e " ${NOCOLOR-\e[1;33m*\e[0m }${*}" >&2
}

eerror() {
  echo -e " ${NOCOLOR-\e[1;31m*\e[0m }${*}" >&2
}

main() {
  # re-define for subst to work
  [[ -n ${NOCOLOR+yes} ]] && NOCOLOR=

  # do nothing if somehow GRUB is not installed
  [[ -x $(command -v grub-mkstandalone) ]] || { ewarn "grub-mkstandalone command not available" && exit 0; }

  [[ ${EUID} -eq 0 ]] || die "Please run this script as root"

  # look for SECUREBOOT_SIGN_CERT and SECUREBOOT_SIGN_KEY values in make.conf
  eval $(grep -s -h SECUREBOOT_SIGN_CERT "/etc/make.conf" "/etc/portage/make.conf")
  eval $(grep -s -h SECUREBOOT_SIGN_KEY "/etc/make.conf" "/etc/portage/make.conf")

  # check that the Secure Boot signing certificate and key are present
  [[ -n "${SECUREBOOT_SIGN_CERT}" ]] || die "SECUREBOOT_SIGN_CERT environment variable is not set"
  [[ -f "${SECUREBOOT_SIGN_CERT}" ]] || die "Secure boot certificate file ${SECUREBOOT_SIGN_CERT} is not present"
  [[ -n "${SECUREBOOT_SIGN_KEY}" ]] || die "SECUREBOOT_SIGN_KEY environment variable is not set"
  [[ -f "${SECUREBOOT_SIGN_KEY}" ]] || die "Secure boot certificate key ${SECUREBOOT_SIGN_KEY} is not present"

  if [[ -f ${GRUB_UEFI} ]]; then
    einfo "Backing up existing grub EFI binary as ${GRUB_UEFI}~"
    cp "${GRUB_UEFI}"{,~} || die "Failed to save existing EFI binary"
  fi

  # Mount the EFI partition if it was specified
  if [[ -n ${EFI_PARTITION} ]] && ! mountpoint -q "${EFI_PARTITION}"; then
    ewarn "EFI partition ${EFI_PARTITION} does not appear to be mounted"
  fi

  einfo "Generating new GRUB EFI image as ${GRUB_UEFI}"
  local dname="${GRUB_UEFI%/*}"
  mkdir -vp "${dname}" || die "Failed to mkdir ${dname}"
  # Exit non-fatally to ensure emerge does not fail completely in containers
  grub-mkstandalone -o "${GRUB_UEFI}" -d "/usr/lib/grub/x86_64-efi" --sbat "/usr/share/grub/sbat.csv" --format "x86_64-efi" "/boot/grub/grub.cfg=${GRUB_CFG}" || eerror "grub-mkstandalone failed"
  sbsign --cert "${SECUREBOOT_SIGN_CERT}" --key "${SECUREBOOT_SIGN_KEY}" --output "${GRUB_UEFI}" "${GRUB_UEFI}" || eerror "sbsign failed"
}

main

systemd

Copy the following script into /etc/kernel/install.d/92-grub-mkstandalone-secureboot.install then make it executable:

#!/usr/bin/env bash

# Copyright 2024-2025 Gabriele Svelto

# This script must be installed under /etc/kernel/install.d, it will
# run after 91-grub-mkconfig.install and generate a signed stand-alone
# GRUB image suitable for booting on UEFI systems with Secure Boot
# enabled.
#
# This script is executed by systemd's kernel-install, NOT by the
# traditional installkernel. I.e. this plugin is run when the systemd
# USE flag is enabled or SYSTEMD_KERNEL_INSTALL=1 is set in the
# environment.

COMMAND="${1}"

: "${GRUB_CFG:=/boot/grub/grub.cfg}"
: "${EFI_PARTITION:=/boot/efi}"
: "${GRUB_UEFI:=/boot/efi/EFI/gentoo/grubx64.efi}"

if [[ ${KERNEL_INSTALL_LAYOUT} != "grub" ]]; then
  exit 0
fi

if [[ ${COMMAND} == add || ${COMMAND} == remove ]]; then
  # do nothing if somehow GRUB is not installed
  if ! command -v grub-mkstandalone >/dev/null; then
    [[ ${KERNEL_INSTALL_VERBOSE} == 1 ]] && echo \
      "grub-mkstandalone command not available"
    exit 0
  fi

  # look for SECUREBOOT_SIGN_CERT and SECUREBOOT_SIGN_KEY values in make.conf
  eval $(grep -s -h SECUREBOOT_SIGN_CERT "/etc/make.conf" "/etc/portage/make.conf")
  eval $(grep -s -h SECUREBOOT_SIGN_KEY "/etc/make.conf" "/etc/portage/make.conf")

  # check that the Secure Boot signing certificate and key are present
  [[ -n "${SECUREBOOT_SIGN_CERT}" ]] || { echo "SECUREBOOT_SIGN_CERT environment variable is not set" && exit 1; }
  [[ -f "${SECUREBOOT_SIGN_CERT}" ]] || { echo "Secure boot certificate file ${SECUREBOOT_SIGN_CERT} is not present" && exit 1; }
  [[ -n "${SECUREBOOT_SIGN_KEY}" ]] || { echo "SECUREBOOT_SIGN_KEY environment variable is not set" && exit 1; }
  [[ -f "${SECUREBOOT_SIGN_KEY}" ]] || { echo "Secure boot certificate key ${SECUREBOOT_SIGN_KEY} is not present" && exit 1; }

  if [[ -f ${GRUB_UEFI} ]]; then
    [[ ${KERNEL_INSTALL_VERBOSE} == 1 ]] && echo \
      "Backing up existing grub EFI binary as ${GRUB_UEFI}~"
    cp "${GRUB_UEFI}"{,~} || { echo "Failed to save existing EFI binary" && exit 1; }
  fi

  [[ ${KERNEL_INSTALL_VERBOSE} == 1 ]] && echo \
    "Generating new GRUB EFI image as ${GRUB_UEFI}"
  dname="${GRUB_UEFI%/*}"
  mkdir -p "${dname}" || { echo "Failed to mkdir ${dname}" && exit 1; }
  grub-mkstandalone -o "${GRUB_UEFI}" -d "/usr/lib/grub/x86_64-efi" --sbat "/usr/share/grub/sbat.csv" --format "x86_64-efi" "/boot/grub/grub.cfg=${GRUB_CFG}" || { echo "grub-mkstandalone failed" && exit 1; }
  sbsign --cert "${SECUREBOOT_SIGN_CERT}" --key "${SECUREBOOT_SIGN_KEY}" --output "${GRUB_UEFI}" "${GRUB_UEFI}" || { echo "sbsign failed" && exit 1; }
fi

Removing old kernels

Removing old kernels can be done via the app-admin/eclean-kernel package. It plays nice with the kernel installation scripts, so if you've added the update scripts described in the paragraph above, it will regenerate the GRUB configuration and signed EFI image every time you remove some kernels. For example, let's say you have three kernels installed and install a new one. After rebooting your machine running the following command will prune the oldest kernel:

# eclean-kernel --num 3 --ask

Once the command is finished your GRUB EFI image will have the new configuration files with only the most three recent kernels and will have already been re-signed and installed.

GRUB updates

When the sys-boot/grub package is updated you don't necessarily need to update your installation, though it's recommended to do so. Re-create the GRUB configuration, standalone image and sign it again:

# grub-mkconfig -o /boot/grub/grub.cfg
# grub-mkstandalone --output /boot/efi/EFI/gentoo/grubx64.efi \
    --directory /usr/lib/grub/x86_64-efi --sbat /usr/share/grub/sbat.csv \
    --format x86_64-efi "/boot/grub/grub.cfg=/boot/grub/grub.cfg"
# sbsign --cert /root/secureboot/MOK.pem --key /root/secureboot/MOK.pem \
    --output /boot/efi/EFI/gentoo/grubx64.efi /boot/efi/EFI/gentoo/grubx64.efi

shim bootloader updates & troubleshooting

If the sys-boot/shim package is updated it is highly recommended to also update your installation. Failing to do so might lead to an unbootable system, as older versions of the package are progressively marked as unsafe, and new UEFI firmwares will refuse to boot them. To update the shim bootloader first update the package then manually copy the new version to the EFI partition:

# cp /usr/share/shim/mmx64.efi /boot/efi/EFI/gentoo/
# cp /usr/share/shim/BOOTX64.EFI /boot/efi/EFI/gentoo/

In particular you'll have to update the shim bootloader if you get a boot failure with the following error message:

Verifying shim SBAT data failed: security policy violation

You won't be able to boot with the existing bootloader and you'll have to either temporarily disable Secure Boot or boot with another bootloader to fix the issue. Updating shim will solve the problem.

motherboard UEFI firmware updates & troubleshooting

Updating the motherboard UEFI firmware sometimes clears the MOK list. If that's the case your system will fail to boot and you will have to enroll your key again. Follow the steps in Reboot & enrolling the key in the MOK list to enroll your key once more.

Similarly sometimes the UEFI boot list will be cleared, removing the shim boot entry. If it happens repeat the steps in Creating a new EFI boot entry to create a new one.

BitLocker recovery

If you have a dual-boot system with Windows and BitLocker enable then this setup will probably trigger a BitLocker recovery procedure every time you change either a kernel or GRUB configuration. Either change will cause your GRUB EFI image to change, and it appears that Windows forces the recovery procedure when that happens. Inserting the BitLocker recovery key is sufficent to get the system back to normal but it can be a bit annoying.

Gentoo 6.1.x generic kernel configuration

Gabriele Svelto — Sun, 26 Nov 2023 07:43:54 GMT

The 6.1.x branch of the Linux kernel has been marked as stable in Gentoo for a few months already and I was busy enough that I forgot to publish an updated generic kernel configuration for it. As with my previous posts this configuration is largely based on the Fedora kernel with some Gentoo-specific tweaks. It supports practically every bit of hardware in existence and enables a lot of bleeding-edge kernel functionality.

The only few notable changes compared to the Fedora kernel are the following:

The binfmt_misc module is baked in for convenience
The NVMe core modules are baked in so you can boot from an NVMe drive without having to use an initrd
The CONFIG_ACPI_EXTLOG option is enabled. This is useful if you're using rasdaemon to monitor ECC memory
The boot logo is disabled
Kernel debugging is disabled
The CONFIG_GENTOO_KERNEL_SELF_PROTECTION option is enabled. This implicitly enables an additional set of security features for hardening the kernel
The CONFIG_GENTOO_PRINT_FIRMWARE_INFO option is enabled, it prints out the firmwares that are loaded into various bits of hardware. This is useful if you want to reduce the amount of files installed sys-kernel/linux-firmware package

Note that the RTC time based on NTP synchronization is enabled so - if you're using OpenRC - you don't need the hwclock service (but you can use osclock instead if some other service requires the clock facility).

Additionally note that this kernel configuration is for use with OpenRC. If you're using systemd you'll have to remove the CONFIG_GENTOO_LINUX_INIT_SCRIPT=y line from the configuration file and add CONFIG_GENTOO_LINUX_INIT_SYSTEMD=y instead.

Now that zstd is well supported in many packages I have enabled it for compressing kernel modules. The only thing you need to make sure to use it is that the sys-apps/kmod package has the zstd USE flag set so that it can handle the compressed modules. If for some reason you don't want modules to be compressed set CONFIG_MODULE_COMPRESS_ZSTD=n and CONFIG_MODULE_COMPRESS_NONE=y respectively.

To use this configuration file install the latest stable sys-kernel/gentoo-sources package (6.1.x), copy the configuration file under /usr/src/linux/ and rename it to .config then proceed to build and install the kernel as usual.

Gentoo 6.1.x kernel configuration file

Gentoo 5.15.x generic kernel configuration

Gabriele Svelto — Tue, 11 Jan 2022 08:12:32 GMT

The 5.15.11 Linux kernel has been marked as stable in Gentoo a few weeks ago and I've wanted to put out a new generic kernel configuration but I kept struggling with a really silly issue adapting it. As with my previous posts this configuration is largely based on the Fedora kernel with some Gentoo-specific tweaks. It supports practically every bit of hardware in existence and enables a lot of bleeding-edge kernel functionality.

The only few notable changes compared to the Fedora kernel are the following:

The binfmt_misc module is baked in for convenience
The NVMe core modules are baked in so you can boot from an NVMe drive without having to use an initrd
The CONFIG_ACPI_EXTLOG option is enabled. This is useful if you're using rasdaemon to monitor ECC memory
The boot logo is disabled
Kernel debugging is disabled
The CONFIG_GENTOO_KERNEL_SELF_PROTECTION option is enabled. This implicitly enables an additional set of security features for hardening the kernel
The CONFIG_GENTOO_PRINT_FIRMWARE_INFO option is enabled, it prints out the firmwares that are loaded into various bits of hardware. This is useful if you want to reduce the amount of files installed sys-kernel/linux-firmware package

Note that the RTC time based on NTP synchronization is enabled (and it's finally become the default in many other Linux distros) so you don't need the hwclock service (but you can use osclock instead if some other service requires the clock facility).

For maximum compatibility I haven't enabled kernel compression in this configuration, but I suggest using CONFIG_MODULE_COMPRESS_ZSTD=y as it provides significant space savings while having effectively no impact on load times. Just make sure that the sys-apps/kmod package has the zstd USE flag set so that it can handle the compressed modules.

To use this configuration file install the latest stable sys-kernel/gentoo-sources package (5.15.x), copy the configuration file under /usr/src/linux/ and rename it to .config then proceed to build and install the kernel as usual.

Gentoo 5.15.x kernel configuration file

Gentoo 5.10.x generic kernel configuration

Gabriele Svelto — Mon, 05 Apr 2021 13:21:12 GMT

The 5.10.27 Linux kernel has been marked as stable in Gentoo a few days ago and I've just updated my generic kernel configuration file to support it. As with my previous posts this configuration is based on the Fedora kernel with some Gentoo-specific tweaks. It supports practically every bit of hardware in existence and enables a lot of bleeding-edge kernel functionality.

The only few notable changes compared to the Fedora kernel are the following:

The binfmt_misc module is baked in for convenience
The NVMe core modules are baked in so you can boot from an NVMe drive without having to use an initrd
RTC time based on NTP synchronization is enabled so you don't need the hwclock service (but you can use osclock instead if some other service requires the clock facility)
The CONFIG_ACPI_EXTLOG option is enabled. This is useful if you're using rasdaemon to monitor ECC memory
The boot logo is disabled
Kernel debugging is disabled

Note that this kernel configuration is for use with OpenRC. If you're using systemd you'll have to remove the CONFIG_GENTOO_LINUX_INIT_SCRIPT=y line from the configuration file and add CONFIG_GENTOO_LINUX_INIT_SYSTEMD=y instead.

To use it install the latest stable sys-kernel/gentoo-sources package (5.10.x), copy the configuration file under /usr/src/linux/ and rename it to .config then proceed to build and install the kernel as usual.

Gentoo 5.10.x kernel configuration file

Monitoring ECC memory on Linux with rasdaemon

Gabriele Svelto — Thu, 13 Feb 2020 22:31:44 GMT

If you have a workstation built around an AMD Ryzen/Threadripper or Intel Xeon processor chances are you're using ECC memory. ECC memory is a worthy investment to improve the reliability of your machine and if properly monitored will allow you to spot memory problems before they become catastrophic.

On recent Linux kernels the rasdaemon tools can be used to monitor ECC memory and report both correctable and uncorrectable memory errors. As we'll see with a little bit of tweaking it's also possible to know exactly which DIMM is experiencing the errors.

Installing rasdaemon

First of all you'll need to intall rasdeamon, it's packaged for most Linux distributions:

Debian/Ubuntu
```
# apt-get install rasdaemon
```
Fedora
```
# dnf install rasdaemon
```
openSUSE
```
# zypper install rasdaemon
```
Gentoo

The package is currently marked as unstable so you'll need to unmask it first:
```
# echo "app-admin/rasdaemon ~amd64" >> /etc/portage/package.keywords
```
Then I recommend enabling sqlite support, this makes rasdaemon record events to disk and is particularly useful for machines that get rebooted often:
```
# echo "app-admin/rasdaemon sqlite" >> /etc/portage/packages.use
```
Finally install rasdaemon itself:
```
emerge rasdaemon
```

Configuring rasdaemon

Then we'll setup rasdaemon to launch at startup and to record events to an on-disk sqlite database.

Note that when booting with Secure Boot enabled, using the kernel lockdown facility in confidentiality mode will prevent rasdaemon from running. To use rasdaemon you'll have to use a different lockdown mode, disable lockdown entirely or disable Secure Boot. You'll find more information in the Troubleshooting section.

Debian/Ubuntu/Fedora/openSUSE and other systemd-based distros
```
# systemctl enable rasdaemon
# systemctl start rasdaemon
```
Gentoo with OpenRC

Add the following line to /etc/conf.d/rasdaemon:
```
RASDAEMON_ARGS=--record
```
Add rasdaemon to the default run-level and start it
```
# rc-config add rasdaemon default
# rc-config start rasdaemon
```

Configuring DIMM labels

At this point rasdaemon should already be running on your system. You can now use the ras-mc-ctl tool to query the errors that have been detected. From now on I will use data from my machine to give an example of the output.

# ras-mc-ctl --error-count
Label                 CE      UE
mc#0csrow#2channel#0  0   0
mc#0csrow#2channel#1  0   0
mc#0csrow#3channel#1  0   0
mc#0csrow#3channel#0  0   0

The CE column represents the number of corrected errors for a given DIMM, UE represents uncorrectable errors that were detected. The label on the left shows the EDAC path under /sys/devices/system/edac/mc/ of every DIMM.

This is not very readable. Since the kernel has no idea of the physical layout of your motherboard it will print the EDAC paths instead of the names of the DIMM slots. We can confirm that the labels are missing with this command:

# ras-mc-ctl --print-labels
ras-mc-ctl: Error: No dimm labels for ASUSTeK COMPUTER INC. model PRIME B450-PLUS

To identify which DIMM slot corresponds to which EDAC path you will have to reboot your system with only one DIMM inserted, write down the name of the slot you insterted it in and then printing out the paths with ras-mc-ctl --error-count.

In my case this was the mapping:

mc#0csrow#0channel#0  DIMM_A1
mc#0csrow#0channel#1  DIMM_A2
mc#0csrow#1channel#1  DIMM_A2
mc#0csrow#1channel#0  DIMM_A1
mc#0csrow#2channel#0  DIMM_B1
mc#0csrow#2channel#1  DIMM_B2
mc#0csrow#3channel#1  DIMM_B2
mc#0csrow#3channel#0  DIMM_B1

Note that there's more than one path per DIMM label, that's fine.

With this data at hand create a text file under /etc/ras/dimm_labels.d/. You will need to fill it up with the mapping data in the following format:

Vendor: <motherboard vendor name>
Model: <motherboard model name>
  <label>: <mc>.<row>.<channel>

You can obtain the motherboard vendor and model name with the following command:

# sudo ras-mc-ctl --mainboard
ras-mc-ctl: mainboard: ASUSTeK COMPUTER INC. model PRIME B450-PLUS

The label lines take a string (the name of the physical DIMM slot), then the numbers in the EDAC path corresponding to the physical slot. You can put more than one label entry per line by separating them with a semicolon. If a given label is associated with more than one EDAC path you can add the separate <mc>.<row>.<channel> sequences by separating them with a comma.

In my case the resulting file (/etc/ras/dimm_labels.d/asus) looks like this:

Vendor: ASUSTeK COMPUTER INC.
Model: PRIME B450-PLUS
  DIMM_A1:  0.0.0, 0.1.0;    DIMM_A2:   0.0.1, 0.1.1;
  DIMM_B1:  0.2.0, 0.3.0;    DIMM_B2:   0.2.1, 0.3.1;

You can find another example of this, with configuration entries for a bunch of other motherboards, in the edac-utils repo.

Once the file is ready it's time to load the labels in the kernel with the following command:

# ras-mc-ctl --register-labels

Printing out labels and error counts will now use the physical DIMM slot names. This is much better if you need to figure out which of your DIMMs is faulty and needs to be replaced:

# ras-mc-ctl --print-labels
LOCATION                            CONFIGURED LABEL     SYSFS CONTENTS
                                    DIMM_A1              0:0:0 missing
                                    DIMM_A2              0:0:1 missing
                                    DIMM_A1              0:1:0 missing
                                    DIMM_A2              0:1:1 missing
mc0 csrow 2 channel 0               DIMM_B1              DIMM_B1
mc0 csrow 2 channel 1               DIMM_B2              DIMM_B2
mc0 csrow 3 channel 0               DIMM_B1              DIMM_B1
mc0 csrow 3 channel 1               DIMM_B2              DIMM_B2

# ras-mc-ctl --error-count
Label   CE      UE
DIMM_B2 0       0
DIMM_B1 0       0
DIMM_B1 0       0
DIMM_B2 0       0

To persist the DIMM names across reboots load the rac-mc-ctl service at startup:

Debian/Ubuntu/Fedora and other systemd-based distros

# systemctl enable ras-mc-ctl
# systemctl start ras-mc-ctl

Gentoo with OpenRC

# rc-config add ras-mc-ctl default
# rc-config start ras-mc-ctl

You're done! After rebooting your system rasdaemon will be continually running and recording errors. You can use ras-mc-ctl to print out a summary of all the errors that have been seen and recorded. Since the counts are stored on disk they will be persisted across reboots. Here's some example output from my machine:

# ras-mc-ctl --summary
Memory controller events summary:
  Corrected on DIMM Label(s): 'DIMM_B1' location: 0:2:0:-1 errors: 5

PCIe AER events summary:
  1 Uncorrected (Non-Fatal) errors: BIT21

No Extlog errors.

No devlink errors.
Disk errors summary:
  0:0 has 6646 errors
No MCE errors.

Troubleshooting

ras-mc-ctl --status prints out ras-mc-ctl: drivers are not loaded

For rasdaemon to work the EDAC kernel drivers for your particular machine need to be loaded. They are usually loaded automatically at boot. You can check out which ones are loaded with this command:
```
# lsmod | grep edac
amd64_edac_mod         32768  0
edac_mce_amd           28672  1 amd64_edac_mod
```
If the EDAC drivers haven't been loaded automatically either your kernel doesn't provide one for your machine or you need to manually load it. Check the EDAC kernel documentation for more details.
rasdaemon fails to start, complaining it can't access the debugfs filesystem

You're likely using the kernel lockdown module in confidentiality mode. When Secure Boot is enabled this will prevent rasdaemon from reading the files it needs to gather its statistics. rasdaemon can work with kernel lockdown when using the integrity mode. To switch to integrity mode add the lockdown=integrity option to the Linux kernel command line in your boot loader.

When using GRUB this can usually be achieved by editing /etc/default/grub and changing the GRUB_CMDLINE_LINUX_DEFAULT variable to include the option, e.g.:
```
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash lockdown=integrity"
```
Keep in mind that integrity mode is less strict than confidentiality mode, as it permits userspace applications to access a fair amount of information that lives in the kernel. This might not be suitable for some deployments - such as those that must run untrusted userspace code.

Gentoo 5.4.x generic kernel configuration

Gabriele Svelto — Fri, 20 Dec 2019 08:28:11 GMT

While the 5.4.x Linux kernel hasn't been marked as stable in Gentoo yet I've updated my generic kernel configuration file to match it. As with the previous config files for 4.14.x and 4.19.x this configuration is based on the Fedora kernel with some Gentoo-specific tweaks. It supports practically every bit of hardware in existence and enables a lot of bleeding-edge kernel functionality. The downside is that building it will take a while and the modules will occupy quite a bit of storage.

To use it install the appropriate sys-kernel/gentoo-sources package (5.4.x), copy the configuration file under /usr/src/linux/ and rename it to .config then proceed to build and install the kernel as usual.

Gentoo 5.4.x kernel configuration file

Gentoo 4.19.x generic kernel configuration

Gabriele Svelto — Sun, 24 Feb 2019 20:10:28 GMT

The 4.19.23 Linux kernel has been marked as stable in Gentoo a few days ago and I've just updated my generic kernel configuration file to match it. As with the the 4.14.x configuration I posted a while ago this configuration is based on the Fedora kernel with some Gentoo-specific tweaks. It supports practically every bit of hardware in existence and enables a lot of bleeding-edge kernel functionality. The downside is that building it will take a while and the modules will occupy quite a bit of storage.

To use it install the latest stable sys-kernel/gentoo-sources package (4.19.x), copy the configuration file under /usr/src/linux/ and rename it to .config then proceed to build and install the kernel as usual.

Gentoo 4.19.x kernel configuration file

Setting the compose key on Xfce

Gabriele Svelto — Wed, 03 Oct 2018 11:21:27 GMT

The compose key is a handy tool to generate characters that aren't available on your keyboard. On Xfce there isn't a readily accessible way to set it, but it can be done rather easily from the Settings Editor:

Launch the Settings Editor from Applications > Settings > Settings Editor or via the terminal by executing the xfce4-settings-editor command
Select the keyboard-layout channel
Look for the Compose property under Default > XkbOptions > Compose
To enable the compose key you have to enter one of the following values in the Compose property:

Compose key

Value

Right Win

compose:rwin

Left Win

compose:lwin

Right Ctrl

compose:rctrl

Left Ctrl

compose:lctrl

Right Alt

compose:ralt

Compose key	Value
Right Win	compose:rwin
Left Win	compose:lwin
Right Ctrl	compose:rctrl
Left Ctrl	compose:lctrl
Right Alt	compose:ralt

The resulting setting should look like this (I'm using the right Windows key in this example):

Gentoo 4.14.x generic kernel configuration

Gabriele Svelto — Thu, 13 Sep 2018 12:24:52 GMT

While the Gentoo Handbook contains almost every step needed into making a working Gentoo installation the kernel configuration step can be quite confusing for a new user. Enabling proper hardware support and turning on all the useful features can be daunting if you're not a developer or simply haven't encountered the kernel configuration before.

A good way around it is to use a generic kernel. genkernel provides a way to build a default kernel but I often find its default configuration to be either out-of-date or missing some important bit.

So, if you want to get started on Gentoo quickly you might as well use my kernel configuration which is based on the Fedora kernel and as such follows an everything-and-the-kitchen-sink approach. It supports practically every bit of hardware out there, will work on desktop PCs, laptops and servers, and includes important security features such as KPTI. The downside is that it's very large and will take a long time to compile.

To use it install the latest stable sys-kernel/gentoo-sources package (4.14.x), copy the configuration file under /usr/src/linux/ and rename it to .config then proceed to build the kernel as usual.

Gentoo 4.14.x kernel configuration file