Linux kernel compilation jots

General

These are a few random notes to self regarding kernel compilation.

The preferred vanilla kernel rep to use is Linux Stable:

$ git clone https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/

It’s often a good idea to pick a kernel version that was released a while ago, but with a high sub-subversion number. So it has been tested properly and it also has several bug fixes that were discovered down the road.

See another post of mine for avoiding “+” being added to the kernel’s version number.

Targeting i386

I compiled a kernel on a x86_64 machine, targeting an i386. Kind-of cross-compilation, but with no need for a cross compiler.

Remember to update the (extra) version number in the Makefile, but with only with [-+.a-z0-9]+ characters, or else the will be trouble with creating .deb packages (see below). Don’t forget adding a dash (“-”) at the beginning of EXTRAVERSION, or else it will be glued to the SUBLEVEL.

Also remember that there’s always

$ make help

and it’s very useful.

After copying a known config file to .config:

$ make ARCH=i386 oldconfig
$ time make ARCH=i386 -j 12 bzImage modules && echo Success

A lazier version is to use the “olddefconfig” and then the “bindeb-pkg” make targets instead of those above — see below. The bonus is that everything gets neatly packaged, and most of the said next becomes unnecessary.

And as root (hey, the ARCH parameter wasn’t required!):

# make modules_install INSTALL_MOD_PATH=/path/to/

(this installs into /path/to/lib/modules/{version number}, so don’t write the “/lib/modules” part)

Remember to update the symbolic links to the source directory if necessary.

Note that it’s possible to set the kernel version directly from the make command, overriding the one given in the Makefile. For example, to match the currently running version:

$ make KERNELVERSION=`uname -r` ARCH=i386 -j 8 bzImage modules && echo Success 

Be sure to check in include/generated/utsrelease.h, possibly while the kernel is compiling, that you got it right. In particular, there may be a “+” sign added.

A depmod was required on the running machine as follows (after booting with the kernel, without modules loaded), even though a depmod ran on modules_install:

# depmod -a

When hacking on the kernel sources, it can be useful to go something like

$ make ARCH=i386 SUBDIRS=drivers/pci/pcie/

in order to compile just a certain subdirectory (like “I didn’t do anything stupid, did I?”).

So nope. SUBDIRS is deprecated. Use the “M=” alternative for modules, even though SUBDIRS catches the built-in objects as well (in case they were played with too).

And it’s also possible to add the known targets, such as

$ make ARCH=i386 SUBDIRS=drivers/pci/pcie/ clean

for cleaning up before compiling etc.

Installing on a Debian-based distribution

That includes Ubuntu, Linux Mint and all distributions where “apt” and “dpkg” are used to manage packages.

Instead of fiddling with all the files, just create .deb packages and install them. It’s so easy, and the files get the right names and locations without any hassle.

The command is simply

$ time make bindeb-pkg && echo Success

after a successful kernel compilation. I’ve tried to do this along with the compilation (of v6.8.12), either by adding bindeb-pkg to the targets or by requesting this target only. In both cases, the build failed after a few minutes, and I had little motivation figuring out why. The only backside is that the compilation number of the used kernel becomes #2 instead of #1, but that’s really petty.

Note that if the kernel is assigned an EXTRAVERSION in the Makefile, it must not contain any uppercase characters nor underscores. In fact, it has to match the regular expression [-+.a-z0-9]* or else an illegal Debian package name will be created, and the finale of the build will fail.

It takes about 12-20 minutes, and it appears to be stuck on the way, but in the end the following files are created on the Linux kernel tree’s parent directory. For a 6.8.12-myserver kernel targeting amd64, these are the files created:

• linux-image-6.8.12-myserver_6.8.12-myserver-2_amd64.deb: The related files in /boot + modules in /lib/modules/
• linux-headers-6.8.12-myserver_6.8.12-myserver-2_amd64.deb: The headers for compiling modules
• linux-image-6.8.12-myserver-dbg_6.8.12-myserver-2_amd64.deb: Files apparently for debugging, a lot of them in /usr/lib/debug/lib/modules/
• linux-libc-dev_6.8.12-myserver-2_amd64.deb: Header files for compiling user-space interface with the kernel, under /usr/include/

Installing these first two packages with “dpkg -i” does what I consider having the kernel installed on the machine: The kernel image in /boot, the kernel modules and the headers. It’s really that simple.

Creating headers for module compilation (non-Debian machine)

This is the probably somewhat off-beat way to create the files for /usr/src/ so that kernel modules can be compiled against the running kernel. The idea is to create a .deb file for the binary of the kernel, which necessarily includes the headers, and then fetch the desired parts from that. Maybe there’s a more straightforward way, but I don’t do this often enough to look for it.

Ah, and “make headers_install” is not the answer. It install the headers used by user-space programs, not for compiling modules. Neither is “make modules_prepare”, which allows compiling the C sources against the kernel tree, but then the MODPOST stage in the compilation fails because Module.symvers is missing.

So start with creating the Debian package files with bindeb-pkg, as mentioned above.

To extract the .deb file that contains the compilation headers:

$ ar x linux-headers-5.15.0_5.15.0-1_amd64.deb

Yes, that’s “ar”, not “tar”. That produces three files, among others the data.tar.xz file. First, verify what it contains:

$ tar -tJf data.tar.xz | less

and once you’ve convinced yourself that it’s OK to untar this in the target’s root directory, become root, and go

# tar -C / -xJvf data.tar.xz

The -C flag causes a chdir to root before executing the command, right? Also note that this will update the “build” symlink in /lib/modules as well.

For a simple installation, do this for the linux-headers and linux-image packages.

Creating an initramfs file (when necessary)

For a non-running kernel, something like (needs to run as root, or it fails)

# update-initramfs -v -c -k 4.14.0-test -b .
update-initramfs: Generating ./initrd.img-4.14.0-test

(-v for verbose, not really necessary)

In theory, this should have worked as well, but it doesn’t enable the prompt for encrypted root filesystem, which is why I need the initramfs to begin with.

$ mkinitramfs -o initrd.img-4.14.0-test 4.14.0-test

Anyway, update-initramfs got me an 285 MB file, which doesn’t fit into the boot/ directory. The one that came with the distro was 28 MB.

On the other hand, if I do the same thing with the currently running kernel, the output gets small and neat. So maybe because the new kernel is much newer, and maybe because initramfs always copies a lot of modules, and not just in use, when it’s not from the running kernel.

It didn’t help bluffing mkinitramfs by renaming the directories in /lib/modules/ and run mkinitramfs as if it was on the current kernel. Exactly the same file size resulted.

So I opened the initramfs image manually (copying from myself), from within an empty directory with

$ zcat ../initrd.img-4.14.0-test | cpio -i -d -H newc --no-absolute-filenames

and looked for the large files. The directories lib/modules/4.14.0-test/kernel/drivers/{gpu,net,scsi} took ~620 MB together. So removing these three, navigating to the root of the initram filesystem and compressing it back again:

$ find . -print0 | cpio --null -ov --format=newc | gzip -9 > ../smaller-initramfs.img

which shrunk the image to 94 MB. Which is small enough. The missing modules will load as the real root filesystem is mounted, so modules that aren’t necessary for boot can be deleted this way.

Trying to obtain a smaller initramfs image with

# update-initramfs -u -b .

when the new kernel is running brought me back a 285 MB image. It’s probably a matter of the new kernel’s size. It might be necessary to write a script that removes any module not loaded when the kernel is up from initramfs’ /lib/modules. But it’s not worth the effort at the moment.

Compiler versions

Since the target computer’s compiler version is really old, I got this when trying to compile a module on it:

$ make
make -C /lib/modules/5.15.0/build M=/home/eli/kernelmodule modules
make[1]: Entering directory '/usr/src/linux-headers-5.15.0'
warning: the compiler differs from the one used to build the kernel
  The kernel was built by: gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
  You are using:           gcc (Debian 4.9.2-10+deb8u2) 4.9.2
  CC [M]  /home/eli/kernelmodule/themodule.o
gcc: error: unrecognized command line option ‘-mrecord-mcount’

So I said, OK, let’s compile the kernel the target machine (or a root jail with a similar environment), but I got:

***
*** Compiler is too old.
***   Your GCC version:    4.9.2
***   Minimum GCC version: 5.1.0
***

So supporting old systems with new kernels isn’t all that easy. My solution is to compile the modules on the new machine, but nevertheless use the kernel headers just generated. They are still useful in the long run.