Linux kernel compilation jots

Just a few notes to self as I compiled a kernel on a x86_64 machine, targeting an i386. Kind-of cross-compilation, but with no need for a cross compiler.

Remember to update the (extra) version number in the Makefile.

Also remember that there’s always

$ make help

and it’s very useful.

After copying a known config file to .config:

$ make ARCH=i386 oldconfig
$ time make ARCH=i386 -j 8 bzImage modules && echo Success

A lazier version is to use the “olddefconfig” and then the “bindeb-pkg” make targets instead of those above — see below. The bonus is that everything gets neatly packaged, and most of the said next becomes unnecessary.

And as root (hey, the ARCH parameter wasn’t required!):

# make modules_install INSTALL_MOD_PATH=/path/to/

(this installs into /path/to/lib/modules/{version number}, so don’t write the “/lib/modules” part)

Remember to update the symbolic links to the source directory if necessary.

Note that it’s possible to set the kernel version directly from the make command, overriding the one given in the Makefile. For example, to match the currently running version:

$ make KERNELVERSION=`uname -r` ARCH=i386 -j 8 bzImage modules && echo Success 

Be sure to check in include/generated/utsrelease.h, possibly while the kernel is compiling, that you got it right. In particular, there may be a “+” sign added.

A depmod was required on the running machine as follows (after booting with the kernel, without modules loaded), even though a depmod ran on modules_install:

# depmod -a

When hacking on the kernel sources, it can be useful to go something like

$ make ARCH=i386 SUBDIRS=drivers/pci/pcie/

in order to compile just a certain subdirectory (like “I didn’t do anything stupid, did I?”).

So nope. SUBDIRS is deprecated. Use the “M=” alternative for modules, even though SUBDIRS catches the built-in objects as well (in case they were played with too).

And it’s also possible to add the known targets, such as

$ make ARCH=i386 SUBDIRS=drivers/pci/pcie/ clean

for cleaning up before compiling etc.

Creating an initramfs file (when necessary)

For a non-running kernel, something like (needs to run as root, or it fails)

# update-initramfs -v -c -k 4.14.0-test -b .
update-initramfs: Generating ./initrd.img-4.14.0-test

(-v for verbose, not really necessary)

In theory, this should have worked as well, but it doesn’t enable the prompt for encrypted root filesystem, which is why I need the initramfs to begin with.

$ mkinitramfs -o initrd.img-4.14.0-test 4.14.0-test

Anyway, update-initramfs got me an 285 MB file, which doesn’t fit into the boot/ directory. The one that came with the distro was 28 MB.

On the other hand, if I do the same thing with the currently running kernel, the output gets small and neat. So maybe because the new kernel is much newer, and maybe because initramfs always copies a lot of modules, and not just in use, when it’s not from the running kernel.

It didn’t help bluffing mkinitramfs by renaming the directories in /lib/modules/ and run mkinitramfs as if it was on the current kernel. Exactly the same file size resulted.

So I opened the initramfs image manually (copying from myself), from within an empty directory with

$ zcat ../initrd.img-4.14.0-test | cpio -i -d -H newc --no-absolute-filenames

and looked for the large files. The directories lib/modules/4.14.0-test/kernel/drivers/{gpu,net,scsi} took ~620 MB together. So removing these three, navigating to the root of the initram filesystem and compressing it back again:

$ find . -print0 | cpio --null -ov --format=newc | gzip -9 > ../smaller-initramfs.img

which shrunk the image to 94 MB. Which is small enough. The missing modules will load as the real root filesystem is mounted, so modules that aren’t necessary for boot can be deleted this way.

Trying to obtain a smaller initramfs image with

# update-initramfs -u -b .

when the new kernel is running brought me back a 285 MB image. It’s probably a matter of the new kernel’s size. It might be necessary to write a script that removes any module not loaded when the kernel is up from initramfs’ /lib/modules. But it’s not worth the effort at the moment.

Creating headers for module compilation

This is the probably somewhat off-beat way to create the files for /usr/src/ so that kernel modules can be compiled against the running kernel. The idea is to create a .deb file for the binary of the kernel, which necessarily includes the headers, and then fetch the desired parts from that. Maybe there’s a more straightforward way, but I don’t do this often enough to look for it.

Ah, and “make headers_install” is not the answer. It install the headers used by user-space programs, not for compiling modules. Neither is “make modules_prepare”, which allows compiling the C sources against the kernel tree, but then the MODPOST stage in the compilation fails because Module.symvers is missing.

So start with this:

$ make bindeb-pkg

Or, if the kernel isn’t compiled yet, better do it like this:

$ time make -j 12 bindeb-pkg && echo Success

Note that if the kernel is assigned an EXTRAVERSION in the Makefile, it must not contain any uppercase characters nor underscores. In fact, it has to match the regular expression [-+.a-z0-9]* or else an illegal Debian package name will be created, and the finale of the build will fail.

The .deb files are put in the parent directory of the kernel tree, so be ready for that. The linux-image-*.deb file contains the Linux image, files to put in /boot as well as the kernel modules, and the linux-headers-*.deb file contains the Linux headers. Frankly speaking, it’s probably easiest to just install these files with apt or dpkg, but for control freaks like me, this is how to do it manually.

To extract the .deb file that contains the compilation headers:

$ ar x linux-headers-5.15.0_5.15.0-1_amd64.deb

Yes, that’s “ar”, not “tar”. That produces three files, among others the data.tar.xz file. First, verify what it contains:

$ tar -tJf data.tar.xz | less

and once you’ve convinced yourself that it’s OK to untar this in the target’s root directory, become root, and go

# tar -C / -xJvf data.tar.xz

The -C flag causes a chdir to root before executing the command, right? Also note that this will update the “build” symlink in /lib/modules as well.

For a simple installation, do this for the linux-headers and linux-image packages.

Compiler versions

Since the target computer’s compiler version is really old, I got this when trying to compile a module on it:

$ make
make -C /lib/modules/5.15.0/build M=/home/eli/kernelmodule modules
make[1]: Entering directory '/usr/src/linux-headers-5.15.0'
warning: the compiler differs from the one used to build the kernel
  The kernel was built by: gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
  You are using:           gcc (Debian 4.9.2-10+deb8u2) 4.9.2
  CC [M]  /home/eli/kernelmodule/themodule.o
gcc: error: unrecognized command line option ‘-mrecord-mcount’

So I said, OK, let’s compile the kernel the target machine (or a root jail with a similar environment), but I got:

***
*** Compiler is too old.
***   Your GCC version:    4.9.2
***   Minimum GCC version: 5.1.0
***

So supporting old systems with new kernels isn’t all that easy. My solution is to compile the modules on the new machine, but nevertheless use the kernel headers just generated. They are still useful in the long run.