Compiling your own Kernel

This article will NOT tell you how exactly to configure your system for best performance, lowest memory or smallest size. Instead we will tell you enough to get started and get exploring the Linux kernel.

A salute to the Kernel

Unless you are a hardcore Linux user, you might be a little overwhelmed by the subject of this article, but let us assure you, it's much easier than it sounds while being much more complicated than it looks!

For starters, let us give you an idea of what the kernel really is. In common usage, the "kernel" of something is its very essence; it is the idea at the very core. Quite appropriately, the Kernel of an operating system is its very core, the piece of code that interfaces between the computer's hardware and the applications running on it.

There are two basic approaches over the architecture of the kernel, on one side, there exists the "Monolithic Kernel", and on the other "Microkernel".

Monolithic kernels contain the entire OS within the kernel. This means all your driver and file-system code runs with highest privileges in the kernel's address-space.

In a microkernel-based OS, only the bare minimum code needed to run the system is part of the kernel, and the rest runs with lower privileges. This allows even system-level components such as drivers to be treated as applications, i.e. if it crashes it can simply be restarted, instead of requiring a system reboot.

It is possible to have a kernel which exhibits properties in between these two. The Windows NT kernel for example, which is used in Windows 2000 / XP / 2003 / Vista / 2008 / 7, is a hybrid kernel.

The Linux kernel itself is Monolithic; however it is highly modular and supports loading such modules at runtime.

Different Kernel architectures

< Click on Image to Enlarge >

Monolithic kernel:

In a monolithic kernel, the kernel neatly separates the hardware devices and the applications which run on top of it. The drivers are "part" of the kernel as they reside in the same memory area.

Microkernel:

In a microkernel design the kernel manages a bare minimum of the system's hardware leaving the rest to device drivers which run in the user space.

The Linux Kernel

Technically, Linux IS the kernel, and "Linux" in itself is not a very usable OS. The many Linux distributions out there are actually software running on the Linux kernel.

Being a monolithic kernel, Linux is huge. The Linux kernel is a complicated piece of coding which -as of October 2010- is over 13.5 million lines of code, spans over tens of thousands of files and supports dozens of platforms.

While the Linux kernel might not be a micro kernel it is still quite modular. The Linux kernel can be -and has been- configured to run on an embedded device with lesser than an MB of ROM and RAM! The Linux kernel source code includes many switches that can be used to configure exactly what all features are built into the kernel, and how they are to be linked to the kernel. Additionally, not all hardware that you would like your system to support needs to be built into the kernel, some of it can be built as external modules which can be loaded and unloaded by the kernel at run-time to add support for certain features.

Even so, compiling this monster is a huge and complex task, but thankfully most of it is automated, and all you have to do is type a few commands, in fact configuring how the kernel should build can be done using a GUI.

Before we dig into that, we need to understand why there is any advantage in building your own kernel.

Why compile your own kernel anyway?

As we have said, compiling your own kernel is a huge task, which might be easy but is definitely not simple. So why do it in the first place? Why not just continue using the ready-made one which came with your distribution.

Nearly all Linux distributions out there include software, and a kernel that is compiled to run on the widest variety of hardware available. Even as of now, the 32-bit versions of most Linux distributions are capable of running a Pentium or Pentium II! While you stand to gain by compiling any application for your hardware, you gain more from compiling system components such as KDE and Gnome and the kernel rather than a text editor like "gedit".

A lot of the hardware that is supported by your distribution's Kernel will never be of use to you, while you might be running hardware that is not fully supported. Such hardware may be supported in a newer version or through patches that can enable support for it.

The kernel is the most central part of the OS, and one which you cannot do without, but the only way to make your kernel run faster or use less memory is to re-compile it exactly the way you want.

Hardware support is a great place for optimizing the kernel. When configuring how your kernel is built, you will be able to control which drivers are loaded as modules, and which are compiled into the kernel. If you have a hardware device that you do not use regularly, it is best to have it as a module.

For example, if your computer only has Linux installed, you may not need support for NTFS, however you can keep it around as a module in case someone brings over a portable drive formatted in NTFS.

Next we go to the actual steps of compiling your own Linux kernel.

Compiling your own kernel

Before you begin, we need to remind you. Compiling the kernel is a long, CPU-intensive process, which requires large amounts of disk space.

Now many Linux distributions have their own version of the kernel, with their own customizations, and often the kernel build procedure can change because of this. So, instead of using your own distribution’s kernel source code we will ask you to download the latest untouched "vanilla" kernel source code from the http://www. kernel.org web site, and build that instead. Before we go ahead, download the latest kernel sources and place them in the "/usr/src" directory.

cd /usr/src sudo tar xvf /path/to/download/sources/linux-2.6.36.tar.bz2

At the time of writing this article, the latest kernel version was 2.6.36, hence we will use "/usr/src/linux-2.6.36/" as the directory containing the sources. Replace this with the directory where you extracted your sources.

STEP 1: Getting the build requirements ready

Building the Linux kernel requires quite a few packages to be installed. The usual build tools such as GCC and make are required, but the kernel has a few more dependencies for successfully compiling. A list of the tools you need will be listed in the "Documentation/Changes" file includes with the kernel sources.

Whichever distribution you use will probably include build tools for the kernel available as a package / meta-package / pattern etc. Simply install that and you should be good to go. Some features of the build process however, might have additional dependencies. For example, if you plan to use the graphical configuration tools, you need to have GTK or QT libraries installed. If you try to use these commands without the requisite packages installed they will output exactly what you need to install.

To keep everything neat and clean, we will use a separate directory for our compilation output. In our case we have created a directory called "Linux” under our home folder, but you could place this anywhere.

It is a good idea to copy out the current kernel's configuration to the one you are building to serve as a starting point, so you don't need to configure everything. To do this, enter the directory where you want to build your kernel (in our case "~/Linux" or "/home/username/ Linux"), and type the following:

zcat /proc/config.gz > .config

Most distribution kernels give access to the configuration via "/proc/config.gz", with this command we are extracting that information into a file called" .config" in the build directory. In case your distribution does not have "/proc/config.gz", you could look in "/boot" for the config file, which will likely be named "config-" followed by your kernel version. Copy this file to the build directory as ".config". If neither of these exists, you should check with your distribution's documentation for more information about how to locate this file, or simply start from scratch.

STEP 2: Configuring the kernel

This section alone can have a few thousand pages dedicated to it! We will instead give you just a singular piece of advice: If you don't understand what a setting does, don't touch it.

There are many ways to configure the kernel. The first is "wizard" -like method "make config" this will walk you through each one of the thousands of kernel configuration parameters one-by-one, we do not recommend you use this option. The second similar option is "make oldconfig", which you can use if you already copied the config of your running kernel in the last step. This will only ask you to configure settings which are not configured in the current" .config".

The ncurses-based configuration tool for the kernel

You will most likely be using one of the following three "make menuconfig", "make xconfig", or "make gconfig". The "make menuconfig" option will show you a ncurses-based console UI which will let you configure the kernel using checkboxes, radio-buttons, etc. This is a much more convenient way of configuring your kernel than the methods listed before, however even more convenient and easy to use options exist in "make xconfig" and "make gconfig". These options show a GUI based on Qt, and Gtk respectively which lets you easily browse all the options and read about them as you make your selections.

The Qt-based GUI for configuring the Linux Kernel

For each setting that you would like to configure, read the associated help text, and then enable / disable it as you find necessary.

Since we want to use a separate directory for our source and our build, we need to provide some additional commands over and above the ones we have listed above. We will add the following extra parameters to our ma ke command:

make -C /usr/src/linux-2.6.36/ O=$PWD

The reason for this is simple. The"-C" option for make tells the make command to change to the specified directory (here "/usr/src/linux-2.6.36") as before running the command. The O=$PWD parameter sets the output directory to $PWD, which is an environment variable that your shell sets as the current directory. This is a way of saying run the make command in that U usr/src/linux-2.6.36/) directory but output the stuff here. These need to be added every time.

So go ahead and run:

make -C /usr/src/linux-2.6.36/ O=$PWD xconfig

OR

make -C /usr/src/linux-2.6.36/ O=$PWD gconfig

And configure the kernel according to your liking.

STEP 3: Building the kernel

Finally, we get to build the kernel. This is as simple as issuing the make command, or in our case:

make -C /usr/src/linux-2.6.36/ O=$PWD

To take advantage of that multi-core system you have, you can issue this command with the parameter -j N, where N is the number of parallel tasks you wish to perform. This can have a significant impact on the build time. In our case the build time on an Intel Core i5 2.67 GHz quad-core system with 8 GB RAM went from 50 minutes for a single thread build to 13 minutes with 8 threads. We recommend staying within 2x the number of cores in your processor. So for a quad-core system with 4 build in parallel:

make -C /usr/src/linux-2.6.36/ O=$PWD -j4

This step will take time, depending on how fast your system is, how many cores your computer has and how many parallel tasks you enabled. This could take from a few minutes to over an hour.

STEP 4: Installing your new kernel

Till now all the commands we have run have been run as a normal user. However these final few steps need to be run as the super-user. So we assume you have switched to the super-user (using "su") for the last few steps.

Now we need to install the modules for our kernel build:

make -C /usr/src/linux-2.6.36/ O=$PWD modules_install

Finally we install the kernel with:

make -C /usr/src/linux-2.6.36/ O=$PWD install

Now we may have one final task left, to create an initial ramdisk image "initrd" for your kernel. This will again depend on the distribution you are compiling for. We need to go to the "/boot" directory and run mkinitrd . Ubuntu and some other distributions use "mkinitramfs", in this case run go to the /boot directory (still as a super-user or using "sudo"), and run:

update-initramfs -c -k vrnlinuz-2.6.36

Finally, finally, finally you need to update your boot loader, this will depend on the distribution you are using. For newer versions of Ubuntu, this will be "update-grub2" otherwise "update-grub". If your distribution uses some other boot -loader, please check the specific instructions for your boot loader ("lilo" can be updated using the "lilo" command).

It is a good idea to keep your old kernel around, at least until you are sure it is stable and all your hardware is working. You may also need to build any third partly kernel module such NVIDIA or ATI drivers for your new kernel.

Conclusion

Building you own Linux kernel has a plethora of advantages, and only some of them have been described in this article. While the information presented here is enough to get a build running, the rest of the burden is still on you. Read the "README" file which comes with the kernel sources, and refer to the extensive documentation available inside the Documentation folder included with the kernel sources for any specific configuration options. Finally, refer to your distribution’s own guide to compiling the kernel as this may have additional features / options that the "vanilla" kernel might not.

Depending on how well versed with Linux you are, reading this article will either appease your concerns about compiling your own Linux kernel, or further turn you off the concept. If you find that the kernel is harder to build than you thought, you probably need to spend some more time using Linux before you are ready to take this step. Otherwise you are probably half-way through the build process already. In that case: Good Luck!