Building Linux
Early in 2017, mintCast embarked on a new voyage; to explore strange new commands; to seek out new kernels and new filesystems; to boldly go where lots of people have gone before. Hopefully it won’t take us five years, but we aim to learn how to “Build Linux” for ourselves.
These pages will document our journey. We will be aided by the giants who have gone before us and laid out the path described in the Linux From Scratch (LFS) and Beyond Linux from Scratch (BLFS) documents. If we survive these two, we may add other guides as well.
And so, we depart…
Preparing The System – January 22, 2017
Decisions to be made before we begin:
- Architecture: 32-bit or 64-bit – LFS supports AMD/Intel 32-bit and 64-bit CPUs. The 64-bit build is a “pure” system in that it supports only 64-bit executables. The 32-bit system would be limited to 4GB RAM. A multi-lib system is possible, but a lot of extra work. We will build a pure 64-bit system.
- Init: Sysvinit or Systemd – While systemd is gaining ground and is in use on many commercial distributions, there is still some controversy in the Linux community. LFS supports both init systems. The LFS systemd build installs systemd along side Sysvinit so we can compare them. We will use the current stable systemd verion of LFS (Version 7.10-systemd).
- Standards: LFS follows POSIX.1-2008, Filesystem Hierarchy Standard (FHS) Version 3.0, and Linux Standard Base (LSB) Version 5.0. We will be initially building the “Core” standard defined in LSB. This will mean we complet the LFS system, then add selected modules from BLFS.
Now it’s time to get started by preparing our build environment (reference: LFS Chapter 1 and Chapter 2):
- Of course, we’re using a Linux Mint to host the initial build, specifically LMDE 2 “Betsy”.
- The initial system needs to provide a base set of programs including a compiler, linker, and shell, to build the new system. A bash script is provided in LFS Chapter.2.2 to verify that these programs are present. LMDE had a few missing, but they were easily added by apt-get.
- A new partition will be needed for the LFS build. On the LMDE system, this means rebooting to a USB Live session in order to resize the main system partition. Decided to use about 30GB for the LFS partition. The filesystem installed on the new partition is ext4.
- Set up the $LFS environment variable to point to where the new LFS partition will be mounted. Will need to remember to check this as we go through the build process.
- Mount the new partition in the place we are pointing $LFS.
Packages, Patches, and Final Preparations – February 5, 2017
Let’s get all the packages downloaded and ready to build (LFS Chapter 3) …
- Create the $LFS/sources directory, download the list of files (wget-list). Everything is fine except zlib-1.2.8.tar.xz from http://www.zlib.net. The newest release is 1.2.11, which is in the development version of LFS. Use 1.2.11, but will need to watch for this in the subsequent steps.
- No “optional patches” installed at this point.
Now to make some final preparations for the build (LFS Chapter 4) …
- Create a temporary “tools” area to put all of the tools we’re going to be building. They will not be part of the final LFS system, so they’re kept in a separate place.
- Create a non-root user to do the work so we don’t accidentally break the system!
- Set up a clean environment for the new user.
- Will not try to take advantage of multi-core processor for the build. Too confusing. Looks like 1 SBU is going to be about 2 minutes on this system.
Constructing a Temporary System – February 19, 2017
This is going to be a little more time-consuming, so it may take a while to get all the tools built (LFS Chapter 5) …
- Using the tar program, extract the package to be built. Ensure you are the lfs user when extracting the package.
- Change to the directory created when the package was extracted.
- Follow the book’s instructions for building the package.
- Change back to the sources directory.
- Delete the extracted source directory unless instructed otherwise.
- Binutils-2.27 – Pass 1 — The Binutils package contains a linker, an assembler, and other tools for handling object files. Determined 1 SBU = 2 minutes.
- GCC-6.2.0 – Pass 1 — The GCC package contains the GNU compiler collection, which includes the C and C++ compilers.
- Linux-4.7.2 API Headers — The Linux API Headers expose the kernel’s API for use by Glibc.
- Glibc-2.24 — The Glibc package contains the main C library. At this point, try to compile and link a simple C program to verify successful installation of initial toolchain. Test was successful!
- Libstdc++-6.2.0 — Libstdc++ is the standard C++ library. It is needed for the correct operation of the g++ compiler. GCC-6.2.0 was already unpacked. Created a different “build” directory as one was already present. (I forgot to delete the source directory as instructed above!)
- Binutils-2.27 – Pass 2 — Since this is the second time through, created a new build directory.
- GCC-6.2.0 – Pass 2 — Same issue with build directory. Not sure what to do with requirement to “unset any environment variables that override the default optimization flags.” Didn’t seem to have any negative effects.
- Tcl-core-8.6.6 — The Tcl package contains the Tool Command Language.
- Expect-5.45 — The Expect package contains a program for carrying out scripted dialogues with other interactive programs.
- DejaGNU-1.6 — The DejaGNU package contains a framework for testing other programs.
- Check-0.10.0 — Check is a unit testing framework for C. “Check Servant” seems to hang? Stopped “make check” and installed anyway.
- Ncurses-6.0 — The Ncurses package contains libraries for terminal-independent handling of character screens.
- Bash-4.3.30 — The Bash package contains the Bourne-Again SHell.
- Bzip2-1.0.6 — The Bzip2 package contains programs for compressing and decompressing files. Compressing text files with bzip2 yields a much better compression percentage than with the traditional gzip.
- Coreutils-8.25 — The Coreutils package contains utilities for showing and setting the basic system characteristics.
- Diffutils-3.5 — The Diffutils package contains programs that show the differences between files or directories.
- File-5.28 — The File package contains a utility for determining the type of a given file or files.
- Findutils-4.6.0 — The Findutils package contains programs to find files. These programs are provided to recursively search through a directory tree and to create, maintain, and search a database.
- Gawk-4.1.3 — The Gawk package contains programs for manipulating text files.
- Gettext-0.19.8.1 — The Gettext package contains utilities for internationalization and localization. These allow programs to be compiled with NLS (Native Language Support), enabling them to output messages in the user’s native language.
- Grep-2.25 — The Grep package contains programs for searching through files.
- Gzip-1.8 — The Gzip package contains programs for compressing and decompressing files.
- M4-1.4.17 — The M4 package contains a macro processor.
- Make-4.2.1 — The Make package contains a program for compiling packages.
- Patch-2.7.5 — The Patch package contains a program for modifying or creating files by applying a “patch” file typically created by the diff program.
- Perl-5.24.0 — The Perl package contains the Practical Extraction and Report Language.
- Sed-4.2.2 — The Sed package contains a stream editor.
- Tar-1.29 — The Tar package contains an archiving program.
- Texinfo-6.1 — The Texinfo package contains programs for reading, writing, and converting info pages.
- Util-linux-2.28.1 — The Util-linux package contains miscellaneous utility programs.
- Xz-5.2.2 — The Xz package contains programs for compressing and decompressing files. It provides capabilities for the lzma and the newer xz compression formats.
- Finishing up (Stripping and Changing Ownership) — My $LFS partition is large, so I didn’t bother with stripping. These tools are temporary, so I’m not worried about optimizing them. Get everything switched over to root ownership.
- The commands in the remainder of this book must be performed while logged in as user root and no longer as user lfs. Also, double check that $LFS is set in root’s environment.
- Backup the current /tools directory in case we need to use it again.
- DONE!
Installing Basic System Software – March 5, 2017
Now that we have a temporary tool chain built, we’re ready to start building the main system (LFS Chapter 6) …
- First note to make here is that LFS 8.0 has just been released as we’re doing this. For now, we’ll continue with LFS 7.10.
- General notes:
- Do not use optimizations as they may cause hard-to-find bugs
- Keep to the order shown in the book
- Don’t use static libraries (except for glibc and gcc)
- Preparing Virtual Kernel File Systems – Mounting and populating /dev, and mounting the Virtual Kernel File Systems. If you don’t do all the work in one session, you’ll need to repeat this before you enter the chroot environment.
- Package Management -This section provides is a roundup of the more popular techniques and how they work. For now we’ll use the “It is All in My Head!” technique.
- Entering the chroot Environment – It is time to enter the chroot environment to begin building and installing the final LFS system. It is important that all the commands throughout the remainder of this chapter and the following chapters are run from within the chroot environment.
- Creating Directories, Essential Files and Symlinks – It is time to create some structure in the LFS file system along with a number of symbolic links which will be replaced by real files throughout the course of this chapter.
Installing Basic System Software (continued) – March 19, 2017
- Building packages… again! The book is not very detailed here on exactly what we need to do. Apparently, for the rest of Chapter 6 we need to go back to the procedures we used for building everything in Chapter 5…namely: Switch to the extracted directory in /sources, then follow the instructions in the section.
- A major problem came up in 6.13. Binutils-2.27:
"The system has no more ptys. Ask your system administrator to create more."
- After quite a bit of searching, I found that I needed to create the /dev/ptmx and /dev/pts structures inside the chroot. I didn’t see anything in Chapter 6.6 about doing this inside the chroot, and I’m not sure if it needs to be done every time you enter the chroot. Just in case, I made a small shell script called “init_chroot.sh” to do it:
#!/bin/sh mknod /dev/ptmx c 5 2 chmod 666 /dev/ptmx mkdir /dev/pts chmod 755 /dev/pts mount -t devpts -o gid=5,mode=620 none /dev/pts
- Tests for GCC-6.2.0 generated a lot of “unexpected failures” in the test_summary step. Continuing in spite of the errors based on the comment in the chapter, but a source of some concern.
- Hit another problem with the 6.22 Acl-2.2.54 … “FATAL ERROR: could not find a valid Extended Attributes library”. Eventually tracked it down to an error in the way the attr package had been installed. If there is a problem with the creation of the library links, simply rerunning the installation process will not fix the problem. I needed to remove the links first with:
rm -rvf /usr/lib/libattr* rm -rvf /lib/libattr*
- to be continued…
I have been meaning to try this out for a long while now and as a long time listener to your podcast I decided I would follow along. For info, I am following v8.0 and I am doing all my compiling within an LXC container, that way there is no chance at all of screwing up my host system.
BTW – the 1.2.8 version of zlib may be found at http://www.zlib.net/fossils/
Keith from Pittsburgh