Compiling The Linux Kernel

This is the new Kernel Section (May 2001) and at least the kernel building part is finally completed out in a nice looking manner.

The old kernel section has needed help for so long that I decided to finally redo the page correctly. Here it is, enjoy!

Required Packages

Note: The correct required packages will be installed if you choose both “Development” and “Kernel Development” during the installation of Red Hat.

Note: The following packages are listed in their order of dependency. Of course, you can install multiple rpm packages at one time in rpm, just add multiple rpm packages to the command line as follows: rpm -hUv pkg1.rpm pkg2.rpm …

Red Hat 7.0 and 7.1 Packages:

glibc-x.x.x-x.i386.rpm
ncurses-x.x.x-x.i386.rpm
kernel-headers-x.x.x-x.0.i386.rpm
kernel-source-x.x.x-x.0.i386.rpm
glibc-devel-x.x.x-x.i386.rpm
cpp-x.x.x-x.i386.rpm
make-x.x.x-x.i386.rpm
ncurses-devel-x.x.x-x.i386.rpm
libstdc++-devel-x.x.x-x.i386.rpm
gcc-c++-x.x.x-x.i386.rpm
dev86-x.x.x-x.i386.rpm (contains the x86 Assembler)

Red Hat 6.2 Packages:

glibc-2.1.3-15.i386.rpm
ncurses-5.0-11.i386.rpm
kernel-headers-2.2.14-5.0.i386.rpm
kernel-source-2.2.14-5.0.i386.rpm
glibc-devel-2.1.3-15.i386.rpm
cpp-1.1.2-30.i386.rpm
make-3.78.1-4.i386.rpm
ncurses-devel-5.0-11.i386.rpm
egcs-1.1.2-30.i386.rpm
egcs-objc-1.1.2-30.i386.rpm
dev86-0.15.0-2.i386.rpm (contains the x86 Assembler)

Red Hat Prior to version 6.2:

x86 Assembler: as86
The x86 Assembler, as86, prior to Red Hat 6.1 was found in the binutils-x.x.x-x.i386.rpm package. as86, for Red Hat 6.1 and above, is in the dev86-x.x.x-x.i386.rpm package.

Optional Packages in addition to the above required packages:

kernel-doc-2.2.14-5.0.i386.rpm <—- Optional
glibc-profile-2.1.3-15.i386.rpm <—- Optional
egcs-g77-1.1.2-30.i386.rpm <—- Optional
egcs-c++-1.1.2-30.i386.rpm <—- Optional

Selecting “kernel development” during a custom install of Red Hat 6.2 will automatically select “development” and “kernel development” and will install all of the above packages from the required list and “egcs-c++-1.1.2-30.i386.rpm” from the list of optional packages. Packages that are not installed are the following:

kernel-doc-2.2.14-5.0.i386.rpm <—- Optional
glibc-profile-2.1.3-15.i386.rpm <—- Optional
egcs-g77-1.1.2-30.i386.rpm <—- Optional

Steps to rebuild the kernel

Change directory to the Linux source directory

While building the kernel you will remain in this directory to execute each of the commands.

To modify the stock kernel on Red Hat just change directory to /usr/src/linux. Downloaded kernels do not have to be put into /usr/src.

/usr/src/linux is a soft link that points to the current Linux source directory, for example: ln -s linux-2.4.2 /usr/src/linux

Edit the EXTRAVERSION = line in the Makefile

The “EXTRAVERSION = ” line at the top of the Makefile determines the directory names, kernel names, etc. that will be used when creating a new kernel and modules. Updating this line will prevent confusion and will prevent the system from stepping on itself. By changing this line, you no longer have to move the old “/lib/modules/x.x.x” directory out of the way.

The method I commonly use for the extraversion line is to put the date and time. Here is an example of the extraversion if I were to build a new smp (smp = multiprocessing) kernel on May 18, 2001 at 4:30pm (note the leading dash, good for easier reading):

EXTRAVERSION = -20010518-1630smp

If I were building the same kernel as non-smp, I would create the extraversion line in the Makefile as follows:

EXTRAVERSION = -20010518-1630

make mrproper (first time / optional after first time)

The command “make mrproper” does a really thorough clean up job and includes removing any old .config files lying around in the source directory.

Note: You may not want to run “make mrproper” each time you try to build a kernel. I would recommend running this command at least on the first time you build a kernel.

Set up arch/i386/defconfig (optional, I recommend doing this)

The purpose of this step is to make sure that arch/i386/defconfig is populated to suit your needs.

When running “make oldconfig” it will look in your .config file (if it exists) for the answer to each configuration question. If no answer is found there, it then looks to the arch/i386/defconfig file for the answer. If no answer is found in either of these two files, then the user is prompted for the answer. Note that “make mrproper” removes your .config file. “make xconfig” should behave in the same manner as “make oldconfig” but I have not verified this yet.

Please note that any time you run “make oldconfig” you should know the contents of arch/i386/defconfig (generally is copied from the configs/ directory.), and .config should exist (if you don’t have an old .config then I recommend to copy arch/i386/defconfig to .config as follows: cp arch/i386/defconfig .config)

I highly recommend having a good arch/i386/defconfig file in place, it makes generating a good .config file much easier — without it you would have to answer lots and lots of questions when creating a new .config file in the following kernel building steps.

You can populate arch/i386/defconfig in one of at least 3 ways:

Use the existing “arch/i386/defconfig” file.

You will find that arch/i386/defconfig already exists. I have occasionally found that this file is not set up as I need it. Keep in mind that Red Hat should ship with most everything built as modules. If you find that most everything in arch/i386/defconfig is set to “no” (disabled or turned off), then I would highly recommend replacing arch/i386/defconfig as shown in one of the other methods in this step.

Use a default configuration file provided by Red Hat (Configuration file from the “configs” directory).

Red Hat provides several default configuration files which are stored in the “configs” directory. Choose one that is named appropriately and copy it to “arch/i386/defconfig”. For example: “cd /usr/src/linux-2.4; cp configs/kernel-2.4.2-2-i386.config arch/i386/defconfig”.

Use your own “.config” file.

You can take an old .config file you have been using and copy it to arch/i386/defconfig. This method could help you to make minor changes to a working kernel configuration — the cool feature about this is, if using this method, once you do a “make mrproper” and a “make oldconfig” you get the effect of a “restore my defaults”.

Generate the “.config” file

Note: Choose one of the following methods to generate the “.config” file (kernel configuration file). I suggest using the following command: yes "" | make oldconfig, or if you are interested to see what is new in your kernel, then just run “make oldconfig”.

Running “make oldconfig” will generate a new .config based on the answers found in arch/i386/defconfig. You will be prompted for answers to items that are not listed in arch/i386/defconfig.

Running “make xconfig” should do the same as “make oldconfig”; however, you will not be prompted for answers to items not listed in arch/i386/defconfig.

Configure the kernel

Now that you have your “.config” file in place we should work with it to build the kernel you want (also called configuring the kernel).

Note: Choose one of the following methods to configure the kernel.

make config

Command line kernel configuration utility. This utility goes over each question one at a time. This utility does not allow you to go to a previous question. Can be a little time consuming to answer all questions, especially if you miss the question you were looking for. This utility is probably useful as a backend to some other scripting utility. CTRL+C will abort the “make config” utility.

make menuconfig

Ncurses (ncurses = Text based graphics) kernel configuration utility. You may pick and choose which questions you will answer.

make xconfig

GUI (X-Windows) kernel configuration utility. You may pick and choose which questions you will answer.

Note: “make xconfig” should be the easiest kernel configuration utility to use.

make clean

“make clean” will clean up stale files in the kernel source directories.

make dep

“make dep” will create the dependency files needed.

Build the kernel

Note: Choose bzImage to build the kernel.

make bzImage

Builds a bzip compressed kernel image. This seems to be the most popular method.

make zImage – (will be discontinued in the future)

Builds a compressed kernel image. The resulting kernel is usually bigger than a bzImage kernel. Use of zImage is discouraged as it may be removed in the future, please use bzImage instead.

make bzdisk – (only for floppy)

Builds a bzip compressed kernel image on a floppy in /dev/fd0. There are many ways of building a bootable floppy, this is one of the ways.

make modules

This command will generate the kernel modules.

make modules_install

This command will install the kernel modules. These modules will be placed into a directory in /lib/modules. Again, the directory name that is used is determined by the “EXTRAVERSION = ” line in the Makefile, so be sure to update the EXTRAVERSION information each time you attempt to build a new kernel.

Install the kernel (/boot)

Copy and Rename: Copy and rename the kernel from arch/i386/bzImage to /boot/vmlinuz-newkernelversion. Here’s a sample of an smp kernel 2.4.2 built on May 18, 2001 at 4:30pm:


cp arch/i386/bzImage /boot/vmlinuz-2.4.2-20010518-1630smp

Create Link: You may also create a soft link for /boot/vmlinuz:


ln -s /boot/vmlinuz-2.4.2-20010518-1630smp /boot/vmlinuz

Note: If the old /boot/vmlinuz is a file (and not a symbolic link) I would rename it to vmlinuz-old.

Note: If the old /boot/vmlinuz is a symbolic link I would remove that link and then generate the new one as shown above.

Install the System.map (/boot)

System.map gets generated in the kernel source directory (the current directory – see step 1)

Copy and Rename: I would recommend to rename this file by using the same naming convention as was used for the modules directory (/lib/modules/2.4.2-x.x.x-x). Here’s a sample of a System.map file for an smp kernel 2.4.2 built on May 18, 2001 at 4:30pm:


cp System.map /boot/System.map-2.4.2-20010518-1630smp

Create Link: I would then create a soft link for /boot/System.map:


ln -s /boot/System.map-2.4.2-20010518-1630smp /boot/System.map

Note: If the old /boot/System.map is a file (and not a symbolic link) I would rename it to System.map-old.

Note: If the old /boot/System.map is a symbolic link I would remove that link and then generate the new one as shown above.

Note: System.map maps out kernel functions. It appears that this file is useful for debugging kernel problems and may not be required for correct kernel operation.

Create a new initrd and install it

The initrd contains SCSI drivers needed at boot time.

The initrd (or initial ram disk) gets loaded during the boot process at the same time the kernel loads. This gets around the chicken and the egg syndrome where the SCSI driver is on the hard drive, but you can’t access the hard drive until you load the SCSI driver.

Note: An initrd should be created only if you are loading SCSI drivers (and Compaq array drivers) as modules. If you do not have any SCSI drivers (or Compaq array drivers) being loaded as modules, they you do not need to make an initrd.

Creating an initrd is easy:

Verify the contents of /etc/conf.modules or /etc/modules.conf (depending on which version you have)

If you load your SCSI driver as a module you should have a line for your SCSI boot device similar to one of the following:

alias scsi_hostadapter cpqarray
alias scsi_hostadapter1 ncr53c8xx
alias scsi_hostadapter2 ncr53c8xx

Note: If you have multiple controllers, then the boot one should be assigned to “scsi_hostadapter” and the other controllers should each increment the count “scsi_hostadapter1” “scsi_hostadapter2” “…”.

Run the mkinitrd command

Be sure to use the same naming convention as we did for the /lib/modules, vmlinuz, and System.map:


mkinitrd -v /boot/initrd-2.4.2-20010518-1630smp.img 2.4.2-20010518-1630smp

The above command syntax is:

-v = verbose
/boot/initrd-2.4.2-20010518-1630smp.img = file to generate
2.4.2-20010518-1630smp = directory to grab modules from (/lib/modules/x.x.x-x)

You may wish to generate a soft link for initrd as well:


ln -s /boot/initrd-2.4.2-20010518-1630smp.img /boot/initrd.img

Check the contents of /etc/lilo.conf

For /etc/lilo.conf I would recommend at least 2 stanzas:

One stanza for the current (new) kernel
One stanza for the last known good kernel

Each “label=” line in /etc/lilo.conf should be unique.

These entries in /etc/lilo.conf can be made to reference soft links or made to reference actual files.

It has been recommended to have one section using soft links for vmlinuz so that you can use the “make install” routine to install a Linux kernel and it will automatically put the kernel in place, attach the soft link for vmlinuz to the new kernel in /boot, and automatically run /sbin/lilo. You will; however, still have to generate a new initrd, put it in place, edit /etc/lilo.conf, and run /sbin/lilo again. I’m not sure if “make install” puts the new System.map in place for you or not – I’ll have to check.

Here is a sample /etc/lilo.conf:

boot=/dev/sda1
map=/boot/map
install=/boot/boot.b
prompt
timeout=50
linear
default=linux
message=/boot/message

image=/boot/vmlinuz-2.2.18-1.3smp
	label=linux-221813smp
	initrd=/boot/initrd-2.2.14-5.0smp.img
	read-only
	root=/dev/sda6
	append="hda=ide-scsi"

image=/boot/vmlinuz-2.2.19-200104091621
	label=linux-2219
	read-only
	root=/dev/sda6
	append="hda=ide-scsi"

other=/dev/sda3
	label=cpqconfig

image=/boot/vmlinuz-2.4.2-2smp
	label=linux
	initrd=/boot/initrd-2.4.2-2smp.img
	read-only
	root=/dev/sda6
	append="hda=ide-scsi"

image=/boot/vmlinuz-2.4.2-2
	label=linux-up
	initrd=/boot/initrd-2.4.2-2.img
	read-only
	root=/dev/sda6
	append="hda=ide-scsi"

For the above, I would rename the section with label=linux to label=linux-old. Then I would more than likely add a new section as follows:

image=/boot/vmlinuz
	label=linux
	initrd=/boot/initrd.img
	read-only
	root=/dev/sda6
	append="hda=ide-scsi"

And we could also add a section like this:

image=/boot/vmlinuz-2.4.2-20010518-1630smp
	label=linux-2.4.2-20010518-1630smp
	initrd=/boot/initrd-2.4.2-20010518-1630smp.img
	read-only
	root=/dev/sda6
	append="hda=ide-scsi"

Verify that you have an old section pointing to your old kernel – in case your new one doesn’t boot.

Also verify that you have a new section pointing to your new kernel:

Check the “image=” line
Make sure there is not a duplicate “label=” anywhere in /etc/lilo.conf
Check the “initrd=” line
Check that “root=” is pointing to the correct partition
Check that “boot=” is pointing to the correct boot sector:

“boot=” is the very first line of /etc/lilo.conf. Make sure that “boot=/dev/sda1” is pointing to either the MBR or to the first sector of your boot partition, depending on how you configured your system.

Note: If you are moving LILO from the MBR to the boot partition you need to make sure to remove LILO from the MBR (see procedures/mbr.html for more information).

The “boot=” line determines where LILO will be installed.

Note: the “z” in vmlinuz stands for compression; therefore, vmlinux is an uncompressed kernel.

Note: Adjust the default= and timeout= values as needed.

/sbin/lilo

After verifying that /etc/lilo.conf is pointing to all the correct places, finally run /sbin/lilo.

You should now be able to reboot onto your new kernel.

Monitoring your kernel build

You may capture the output of building your kernel in several ways.

Use the “script” command.

Once script is started it will log everything to “typescript”. When you are done logging type the word exit. You may then review the contents of “typescript”.

You can run individual commands like so and have normal output and error go to separate files:

your_command 2>>kernel_err.log 1>>kernel_out.log

You can run all the commands in a for loop and have individual output/error logs generated:


cd /usr/src/linux;
vi Makefile;
for X in mrproper xconfig clean dep bzImage modules modules_install; do
    make $X 2>/tmp/kernel-${X}-error.log 1>/tmp/kernel-${X}-output.log;
    if [ $? -ne 0 ]; then
        echo Error, check error logs in /tmp
        break;
    fi
done

You can run all the commands in a for loop and have all output and errors dump to 1 output file and 1 error file:


cd /usr/src/linux;
vi Makefile;
for X in mrproper xconfig clean dep bzImage modules modules_install; do
    echo "---------------------- starting make $X --------------"
    echo "---------------------- starting make $X --------------" >2
    make $X;
    echo "---------------------- ending make $X --------------"
    echo "---------------------- ending make $X --------------" >2
    if [ $? -ne 0 ]; then
        echo Error, check error logs in /tmp
        break;
    fi
done 2>/tmp/kernel-error.log 1>/tmp/kernel-output.log

If the above sections of code need adjustment please email me contact.html. I have used them in the past; however, I am writing them here from memory.

SMP on older Compaq ProLiant Servers

Compaq’s early SystemPro/ProLiant SMP design was created long before the Intel MPS specification. Since the 2.2 and 2.4 Linux kernels only support Intel MPS architectures, these early MP servers can only be used in single processor mode:

SystemPro/XL
ProLiant 2000
ProLiant 4000
ProLiant 4500

Compaq is considering making its early MP specifications available to the public, or even adding the support to the Linux kernel. If you are interested in participating in the development of such support, or have one of these servers you would like to run Linux SMP, please send an email to John Cagle c/o linux.feedback@compaq.com. Be sure to put John Cagle’s name in the email so it will be forwarded to him.

Tested smp on PL5000, it works! 2 processors seen
dual PPRO 200, Mach ID CPQ1561, 256K secondary cache, CPU IDs 0617 and 0619, no processors mapped out, serial number D629HVN1xxxx

Other files in /boot:

Discuss these files later:
kernel.h
module-info
Discuss these parts of /etc/lilo.conf:
map=/boot/map
map seems to have something to do with the messages file – see man lilo.conf for more information.

To be completed at a later date…