Markus Wernig

UNIX/Network Security Engineer
CCSA, CCSE
CISSP


PGP key transition note
GPG Key
 (in use after Aug. 9 2013)

old GPG Key
 (in use up to Aug. 9 2013)

Personal | Professional | IT related | Writings de

A customized binary Gentoo kernel package

After having set up a build server/binhost for use with my Gentoo desktop machines, there was one piece of software left that still needed to be compiled on the client machines: the kernel.

While Gentoo does have a binary kernel package nowadays, this "distribution kernel" shares one characteristic with the kernels of all other binary GNU/Linux distributions: It is very generic and has almost every possible feature/option enabled. This makes sense, as it must be able to run on any supported hardware, and people should not need to go back to compiling their own kernels as soon as they want to use a more uncommon option or piece of hardware.

But that characteristic (or the lack of it, more precisely) is exactly why I am using Gentoo: because it gives me fine-grained control over the software I'm using and over which features this software has and does not have. So using the "distribution kernel" does not do for me, I had to find a different approach. One that would combine the convenience of binary packaging with the type of control that I've grown to appreciate over the years. The same, basically, that I had achieved with the binhost, where all packages are built with the exact flags that I want to use.

So I started hacking together a "custom-kernel-bin" ebuild that would allow me to build a custom kernel binary package, based on the gentoo-sources kernel source package, from which I have built every of my kernels in the last 15 years.

The following assumes that you have a build environment for binary packages already set up. In my case this build environment (or rather its PKGDIR the binary packages are installed into) also serves as binhost, from where the clients download their binary xpak files, but that is out of scope here and not relevant for the kernel package.

Important notes

  • This package now allows building a kernel with or without initrd.
    The "initramfs" USE flag controls this behaviour. dracut is used to build the initramfs after installation if the flag is set (which is why it is a conditional RDEPEND dependency).
  • This package does not maintain earlier kernel versions in /boot!
    Like any other package it removes the files from earlier versions of itself. It does not, however, remove kernels from /boot that were installed there manually. So it might be a good idea to keep your last manually compiled kernel there in case a new one doesn't boot.

/etc/portage/make.conf

I use the following build flags, among others:

CBUILD="x86_64-pc-linux-gnu"
CHOST="x86_64-pc-linux-gnu"
ACCEPT_KEYWORDS="amd64"

PORTDIR_OVERLAY=/usr/local/portage
CFLAGS="${COMMON_FLAGS} -march=x86-64 -mtune=generic --param l1-cache-size=32 --param l1-cache-line-size=64
 --param l2-cache-size=8192"
FEATURES="-sandbox -preserve-libs -ccache userpriv buildpkg binpkg-multi-instance"
EMERGE_DEFAULT_OPTS="--backtrack=200 --with-bdeps=y --keep-going=y --ask-enter-invalid"

Some of these may need to be adjusted for other environments. Note that this particular buildhost is set up to also use the same files locally that it installs as binpkg into $PKGDIR. This may not be possible with every CPU combination.

/usr/local/portage

This is where the ebuild is located (if PORTDIR_OVERLAY points somewhere else on your buildhost, use that instead). The following shows two ebuilds, one for kernel 5.5.9, one for 5.6.7.
The examples use the placeholder "$version" for the package version. Replace "$version" with the actual kernel version you are building.

buildsrv # ls -R1 /usr/local/portage/
/usr/local/portage/:
sys-kernel

/usr/local/portage/sys-kernel:
custom-kernel-bin

/usr/local/portage/sys-kernel/custom-kernel-bin:
custom-kernel-bin-5.5.9.ebuild
custom-kernel-bin-5.6.7.ebuild
files
metadata.xml
Manifest

/usr/local/portage/sys-kernel/custom-kernel-bin/files:
config-5.5.9-gentoo
config-5.6.7-gentoo

There are basically four files here that we need, one of which is optional.
The files need to be created by hand, relative to (in this example) /usr/local/portage/sys-kernel/custom-kernel-bin:

  • custom-kernel-bin-$version.ebuild: This is the ebuild file that controls both, the build and installation process
  • files/config-$version-gentoo: This is the kernel configuration (.config)
  • Manifest: This file contains the checksums of all involved files
  • metadata.xml (optional): This file contains additional information about the package
  1. custom-kernel-bin-$version.ebuild
    # Copyright 2021 Gentoo Authors
    # Distributed under the terms of the GNU General Public License v2
    
    EAPI=7
    
    DESCRIPTION="Custom-built amd64 kernel binpkg from gentoo-sources"
    HOMEPAGE="https://www.kernel.org"
    SRC_URI=""
    
    LICENSE="GPL-2"
    SLOT="0"
    KEYWORDS="amd64 ~amd64"
    IUSE="initramfs"
    
    DEPEND=""
    RDEPEND="
            ${DEPEND}
            !sys-kernel/gentoo-kernel:${SLOT}
            !sys-kernel/vanilla-kernel:${SLOT}
            !sys-kernel/vanilla-kernel-bin:${SLOT}
            sys-boot/grub
            initramfs? ( sys-kernel/dracut )
            "
    BDEPEND="=sys-kernel/gentoo-sources-${PV}"
    
    QA_PREBUILT='*'
    
    S=${WORKDIR}
    SRCDIR="/usr/src/linux-${PV}-gentoo"
    BUILDDIR="${T}/build-${PV}"
    
    # We set up a temp builddir, not /usr/src/linux-NNN
    src_prepare() {
            default
            cp -r ${SRCDIR}/ ${BUILDDIR}
    }
    
    # If we have a config file for the kernel (package) version ${PV}
    # that we are building for, we copy that to the temp builddir.
    # Else we try to find and use a kernel config from an earlier build.
    # Note that all emerge-specific flags from make.conf are also set.
    src_configure() {
            unset ARCH
            if [ -e "${FILESDIR}/config-${PV}-gentoo" ]; then
                    cp ${FILESDIR}/config-${PV}-gentoo ${BUILDDIR}/.config
                    cd $BUILDDIR
                    emake olddefconfig && return
            elif [ -d ${FILESDIR}/ ]; then
                    latest=`ls ${FILESDIR}/config-* | sort -V | tail -1`
                    if [ -e ${latest} ]; then
                            cp ${latest} ${BUILDDIR}/.config
                            cd $BUILDDIR
                            emake olddefconfig && return
                    fi
            fi
            # fallback
            die "Failed to configure kernel source"
    }
    
    # Compile normally.
    # Note that all emerge-specific flags from make.conf are also set.
    src_compile() {
            cd $BUILDDIR
            emake
            emake INSTALL_MOD_PATH="${T}" modules_install && return
            # fallback
            die "Failed to compile kernel source"
    }
    
    # We copy only the kernel, System.map and .config to /boot.
    # There is NO INITRD here! It is generated by dracut in the postinst step if
    # the "initramfs" USE flag is set.
    # Then copy the modules.
    # Note: The kernel sources are not required on the target system, so the various symlinks in /lib/modules/$version
    #       that usually point to /usr/src/linux are not set here. Uncomment below to change this.
    src_install() {
            dodir /boot
            insinto /boot
            newins "${BUILDDIR}/arch/x86/boot/bzImage" vmlinuz-${PV}-gentoo
            newins "${BUILDDIR}/.config" config-${PV}-gentoo
            newins "${BUILDDIR}/System.map" System.map-${PV}-gentoo
    
            rm -f ${T}/lib/modules/${PV}-gentoo/build
            rm -f ${T}/lib/modules/${PV}-gentoo/source
    #       ln -s ${SRCDIR} ${T}/lib/modules/${PV}-gentoo/build
    #       ln -s ${SRCDIR} ${T}/lib/modules/${PV}-gentoo/source
            dodir "/lib/modules/${PV}-gentoo"
            insinto "/lib/modules"
            doins -r "${T}/lib/modules/${PV}-gentoo"
    }
    
    # Generate grub config for new kernel and module maps.
    pkg_postinst() {
            if use initramfs ; then
                    elog "Creating initramfs"
                    dracut --hostonly --force --kver ${PV}-gentoo
            fi
            elog "Kernel files were installed successfully. Running grub-mkconfig -o /boot/grub/grub.cfg"
            grub-mkconfig -o /boot/grub/grub.cfg
            depmod -a ${PV}-gentoo
    }
    

  2. files/config-$version-gentoo

    Copy a .config file for the kernel $version you are building (!!!) here.

    If you don't have one (which is to be expected for a new kernel version), you can copy the .config from an earlier kernel build into the kernel source tree under /usr/src/linux-$version-gentoo and then run "make oldconfig" on the tree. Then copy the resulting .config to /usr/local/portage/sys-kernel/custom-kernel-bin/files/config-$version-gentoo.

  3. metadata.xml

    You can put much information about your custom kernel package here (see the Gentoo developer handbook). I personally found it useful to add a comment that clarifies what the initramfs USE flag is used for. This file is optional. If you have it, equery and other tools will display the information in it, else not.
    <?xml version="1.0" encoding="UTF-8"?>
    <!DOCTYPE pkgmetadata SYSTEM "http://www.gentoo.org/dtd/metadata.dtd">
    <pkgmetadata>
      <maintainer type="person">
        <email>build@some.domain</email>
        <name>Build Root</name>
      </maintainer>
      <use>
        <flag name="initramfs">Build an initramfs after installation (requires dracut)</flag>
      </use>
    </pkgmetadata>
    
    Output of equery when you add this file:
    # equery u custom-kernel-bin
    [...]
     - - initramfs : Build an initramfs after installation (requires dracut)
    
    Output of equery without this file:
    # equery u custom-kernel-bin
    [...]
     - - initramfs : <unknown>
    
  4. Manifest

    After you have created the files above, run this command:
    ebuild /usr/local/portage/sys-kernel/custom-kernel-bin/custom-kernel-bin-$version.ebuild digest
    This will generate the Manifest file.

Build the package

Run the following command:

emerge --buildpkgonly custom-kernel-bin
This will compile the kernel and create the package file in
/var/cache/binpkgs/sys-kernel/custom-kernel-bin/custom-kernel-bin-$version-1.xpak
(or wherever your $PKGDIR points to).

Install the package

There is more than one way to do this. My method uses the binhost (where the above command installed the package into $PKGDIR, which serves as my binhost)

The target machine (client) needs to have the same ebuild and Manifest files that the package was built with. Either set up a repository from which it can be downloaded with emerge --sync (out of scope here), or simply copy them to the target. Assuming that you have PORTDIR_OVERLAY=/usr/local/portage, you could eg.:

mkdir /usr/local/portage/sys-kernel/
scp -r buildhost:/usr/local/portage/sys-kernel/custom-kernel-bin /usr/local/portage/sys-kernel/
Also put this into /etc/portage/make.conf:
(all the same USE flags etc. as on the buildhost)

PORTDIR_OVERLAY=/usr/local/portage
PORTAGE_BINHOST="https://$binhost/path/to/$PKGDIR"
FEATURES="-sandbox -ccache -buildpkg preserve-libs getbinpkg"

If everything is set up correctly, emerge should find the new package
emerge -s custom-kernel-bin

*  sys-kernel/custom-kernel-bin
      Latest version available: 5.6.7
      Latest version installed: [ Not Installed ]
      Size of files: 0 KiB
      Homepage:      https://www.kernel.org
      Description:   Custom-built amd4 kernel binpkg from gentoo-sources
      License:       GPL-2

Now install the package with emerge -K custom-kernel-bin.

The "-K" makes sure it will rather fail than try to build the package if the package is not installed eg. because of some flag mismatch.

Note for initramfs:

If you want dracut to build an initramfs after installation, you need to set the initramfs USE flag for the custom-kernel-bin package on the build host and on the target system (!), i.e. the system you are installing the custom kernel to. To do so, add the following line to /etc/portage/package.use or, if that is not a file but a directory, to any file under it:

sys-kernel/custom-kernel-bin initramfs
This will install dracut and its dependencies before installing custom-kernel-bin. If you are using a binhost, make sure the dracut package (and its dependencies) are available there.

If you want to be able to install the same custom binary kernel on multiple targets with and without initrd, you need to build the kernel package twice, one time with, one time without the initramfs USE flag. Make sure that the binpkg-multi-instance flag is set in FEATURES, else the two package versions will overwrite each other.

You can customize dracut's operation locally in /etc/dracut.conf. (This might be necessary if you need to modify the modules that dracut puts into the initramfs.)


Markus Wernig

webmaster wernig net