The art of cross compiling

Having a working cross toolchain is the starting point of a journey, not the end of it. At some point, you will want to begin cross compiling the various tools, applications, and libraries that you need on your target. Many of them will be open source packages, each of which has its own method of compiling, and each with its own peculiarities. There are some common build systems, including:

I will cover only the first two here since these are the ones needed for even a basic embedded Linux system. For CMake, there are some excellent resources on the CMake website referenced in the preceding point.

Simple makefiles

Some important packages are very simple to cross compile, including the Linux kernel, the U-Boot bootloader, and Busybox. For each of these, you only need to put the toolchain prefix in the make variable CROSS_COMPILE, for example arm-cortex_a8-linux-gnueabi-. Note the trailing dash -.

So, to compile Busybox, you would type:

$ make CROSS_COMPILE=arm-cortex_a8-linux-gnueabi-

Or, you can set it as a shell variable:

$ export CROSS_COMPILE=arm-cortex_a8-linux-gnueabi-
$ make

In the case of U-Boot and Linux, you also have to set the make variable ARCH to one of the machine architectures they support, which I will cover in Chapter 3, All About Bootloaders and Chapter 4, Porting and Configuring the Kernel.

Autotools

The name, Autotools, refers to a group of tools that are used as the build system in many open source projects. The components, together with the appropriate project pages, are:

The role of Autotools is to smooth over the differences between the many different types of system that the package may be compiled for, accounting for different versions of compilers, different versions of libraries, different locations of header files, and dependencies with other packages. Packages that use Autotools come with a script named configure that checks dependencies and generates makefiles according to what it finds. The configure script may also give you the opportunity to enable or disable certain features. You can find the options on offer by running ./configure --help.

To configure, build, and install a package for the native operating system, you would typically run these three commands:

$ ./configure
$ make
$ sudo make install

Autotools is able to handle cross development as well. You can influence the behavior of the configure script by setting these shell variables:

Sometimes it is sufficient to set only the CC variable, as follows:

$ CC=arm-cortex_a8-linux-gnueabihf-gcc ./configure

At other times, that will result in an error like this:

[...]
checking whether we are cross compiling... configure: error: in '/home/chris/MELP/build/sqlite-autoconf-3081101':
configure: error: cannot run C compiled programs.
If you meant to cross compile, use '--host'.
See 'config.log' for more details

The reason for the failure is that configure often tries to discover the capabilities of the toolchain by compiling snippets of code and running them to see what happens, which cannot work if the program has been cross compiled. Nevertheless, there is a hint in the error message of how to solve the problem. Autotools understands three different types of machine that may be involved when compiling a package:

So, to cross compile, you just need to override host, as follows:

$ CC=arm-cortex_a8-linux-gnueabihf-gcc \
./configure --host=arm-cortex_a8-linux-gnueabihf

One final thing to note is that the default install directory is <sysroot>/usr/local/*. You would usually install it in <sysroot>/usr/* so that the header files and libraries would be picked up from their default locations. The complete command to configure a typical Autotools package is:

$ CC=arm-cortex_a8-linux-gnueabihf-gcc \
./configure --host=arm-cortex_a8-linux-gnueabihf --prefix=/usr

The SQLite library implements a simple relational database and is quite popular on embedded devices. You begin by getting a copy of SQLite:

$ wget http://www.sqlite.org/2015/sqlite-autoconf-3081101.tar.gz
$ tar xf sqlite-autoconf-3081101.tar.gz
$ cd sqlite-autoconf-3081101

Next, run the configure script:

$ CC=arm-cortex_a8-linux-gnueabihf-gcc \
./configure --host=arm-cortex_a8-linux-gnueabihf --prefix=/usr

That seems to work! If it failed, there would be error messages printed to the terminal and recorded in config.log. Note that several makefiles have been created, so now you can build it:

$ make

Finally, you install it into the toolchain directory by setting the make variable DESTDIR. If you don't, it will try to install it into the host computer's /usr directory which is not what you want.

$ make DESTDIR=$(arm-cortex_a8-linux-gnueabihf-gcc -print-sysroot) install

You may find that final command fails with a file permissions error. A crosstool-NG toolchain will be read-only by default, which is why it is useful to set CT_INSTALL_DIR_RO to y when building it. Another common problem is that the toolchain is installed in a system directory such as /opt or /usr/local in which case you will need root permissions when running the install.

After installing, you should find that various files have been added to your toolchain:

Now you can compile programs that use sqlite3 by adding -lsqlite3 at the link stage:

$ arm-cortex_a8-linux-gnueabihf-gcc -lsqlite3 sqlite-test.c -o sqlite-test

Where, sqlite-test.c is a hypothetical program that calls SQLite functions. Since sqlite3 has been installed into the sysroot, the compiler will find the header and library files without any problem. If they had been installed elsewhere you would have to add -L<lib dir> and -I<include dir>.

Naturally, there will be runtime dependencies as well, and you will have to install the appropriate files into the target directory as described in Chapter 5, Building a Root Filesystem.

Package configuration

Tracking package dependencies is quite complex. The package configuration utility, pkg-config (http://www.freedesktop.org/wiki/Software/pkg-config) helps track which packages are installed and which compile flags each needs by keeping a database of Autotools packages in [sysroot]/usr/lib/pkgconfig. For instance, the one for SQLite3 is named sqlite3.pc and contains essential information needed by other packages that need to make use of it:

$ cat $(arm-cortex_a8-linux-gnueabihf-gcc -print-sysroot)/usr/lib/pkgconfig/sqlite3.pc
# Package Information for pkg-config
prefix=/usr
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include
Name: SQLite
Description: SQL database engine
Version: 3.8.11.1
Libs: -L${libdir} -lsqlite3
Libs.private: -ldl -lpthread
Cflags: -I${includedir}

You can use the utility pkg-config to extract information in a form that you can feed straight to gcc. In the case of a library like libsqlite3, you want to know the library name (--libs) and any special C flags (--cflags):

$ pkg-config sqlite3 --libs --cflags
Package sqlite3 was not found in the pkg-config search path.
Perhaps you should add the directory containing `sqlite3.pc'
to the PKG_CONFIG_PATH environment variable
No package 'sqlite3' found

Oops! That failed because it was looking in the host's sysroot and the development package for libsqlite3 has not been installed on the host. You need to point it at the sysroot of the target toolchain by setting the shell variable PKG_CONFIG_LIBDIR:

$ PKG_CONFIG_LIBDIR=$(arm-cortex_a8-linux-gnueabihf-gcc -print-sysroot)/usr/lib/pkgconfig \
pkg-config sqlite3 --libs --cflags
 -lsqlite3

Now the output is -lsqlite3. In this case, you knew that already, but generally you wouldn't, so this is a valuable technique. The final command to compile would be:

$ PKG_CONFIG_LIBDIR=$(arm-cortex_a8-linux-gnueabihf-gcc -print-sysroot)/usr/lib/pkgconfig \
arm-cortex_a8-linux-gnueabihf-gcc $(pkg-config sqlite3 --cflags --libs) sqlite-test.c -o sqlite-