Fun to Program -- Build system

This was originally written and created around 2013 and may require to be updated. (2021)

Build system

There are many cross-platform build systems:

Autotools for GNU, GNOME, generic C/C++/Vala/… programs
CMake for KDE programs, etc.
Python distutils for some Python programs
- Install Python source distribution (as a user)
- Generate Python source distribution (as an upstream)

Please note there are many programs which come with the plain GNU Make as the build system . The plain GNU Make is simple and works fairly well across all modern Unix-like systems.

Autotools

This is the most versatile and robust build system.

The internal of Autotools are beyond my taste for learning. (Yack! What a convoluted use of GNU m4 for such complicated tasks.)

But compared to other newer build systems, I would rather use Autotools. This is because: “better the devil you know than the devil you don’t know.”

As a user

The source package generated by the Autotools (= Autoconf + Automake + …) requires only basic tools available on any Debian systems:

Bourne shell (Dash)
Sed
Make

As a user, you only deal with files such as configure and Makefile.in generated by the Autotools. There is no need for any specialized external tools such as GNU M4 nor Perl nor Autotools.

They can be build as follows:

$ ./configure
..
$ make
..
$ sudo make install

The schematic relationship can be visualized as:

program.c --------------------------------------\
program.h --------------------------------------+--[make]--> program    
config.h.in --\                 /--> config.h --/    /
Makefile.in --+--[./configure]--+--> Makefile ------+

TIP: The above build process is true not just for source packages generated by the Autotools but for many other source packages.

Here, the template files such as Makefile.in and config.h.in have embedded strings such as @CC@ and @prefix@ and they are replaced by the name of the particular C compiler and the installation directory using the Sed program executed from the configure script. It is good idea not to mess with these directly to customize your package build process.

Instead, you have 2 basic ways to customize your package build process as a user.

Run the configure shell script with options.
Run the make command with options and environment variables.

As for the configure options, you get their good explanation as follows:

$ ./configure -h

You should check all available command line options displayed. This is because the configure script is meant to be run non-interactively.

As for the make command, since the Makefile generated by the Autotools follows GNU coding standards, please read “Makefile Conventions” in the GNU coding standards to learn how to change behavior of make.

TIP: The default installation destination is /usr/local.

As an upstream

Autotools (= Autoconf + Automake + …) is a tool to create the portable source package with minimal efforts. It is not easy to make portable across different platforms. It generates a good set of configure, Makefile.in and config.h.in required for a source package from simple data in the configure.ac and Makefile.am files using the autoreconf -i command which runs all the Autotools in the right sequences.

program.c--------------+
program.h--------------+
configure.ac --+       |           +--> configure----------V
               +--[autoreconf -i]--+--> config.h.in --[./configure]--> config.h
Makefile.am  --+                   +--> Makefile.in --[./configure]--> Makefile

configure.ac is the primarily configuration file to create configure by the Autoconf:
- The Autoconf is a M4 macro program with many pre-defined macros.
- M4 macro quotes used by the Autoconf is changed to [ ].
- M4 macro strings in configure.ac are substituted by the Autoconf.
- configure.ac can contain shell codes and they remain in configure as is.
Makefile.am is the primarily configuration file to create Makefile.in by the Automake:
- The Automake is a Perl program.
- Makefile.am defines the specific variable names and their values to denote source files and their generated files.
- Generated build dependencies are defined as suffix rules.
- Makefile.am can contain Makefile snippets and remains in Makefile as is.
Makefile is generated from Makefile.in using configure by a user.
- configure is a portable shell program.
- Makefile.in is the Makefile template.

Suppose you have hello.c.

#include <stdlib.h>
#include <stdio.h>
#include "config.h"
main()
{
    printf("Hello, world! version: %s\n", VERSION);
    return EXIT_SUCCESS;
}

Then run autoscan to generate initial template of configure.ac and edit it manually to the shape.

$ autoscan
configure.ac: warning: missing AC_CHECK_HEADERS([stdlib.h]) wanted by: hello.c:1
$ mv configure.scan configure.ac
$ vim configure.ac

Since strict GNU compliance is not easy, lets make this as a “foreign” package to avoid complication:

#                                               -*- Autoconf -*-
# Process this file with autoconf to produce a configure script.

AC_PREREQ([2.69])
AC_INIT([hello], [0.1], [osamu@debian.org])
AC_CONFIG_SRCDIR([config.h.in])
AC_CONFIG_HEADERS([config.h])

# Manually added
AM_INIT_AUTOMAKE([foreign])

# Checks for programs.
AC_PROG_CC

# Checks for libraries.

# Checks for header files.

# Checks for typedefs, structures, and compiler characteristics.

# Checks for library functions.

AC_CONFIG_FILES([Makefile])
AC_OUTPUT

Makefile.am defines programs to be built and relationship of source files in very simple way as.

bin_PROGRAMS = hello
hello_SOURCES = hello.c

Here:

List binary program names as the value of bin_PROGRAMS.
List source file names for hello as the value of hello_SOURCES.

$ ls
configure.ac
hello.c
Makefile.am

Let’s generate configure, Makefile.in and config.h.in using autoreconf -i -v -f (-f option is needed only when you are running this command second time. -v is for verbose output.):

$ autoreconf -i -v -f
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force
autoreconf: configure.ac: tracing
autoreconf: configure.ac: not using Libtool
autoreconf: running: /usr/bin/autoconf --force
autoreconf: running: /usr/bin/autoheader --force
autoreconf: running: automake --add-missing --copy --force-missing
configure.ac:10: installing './install-sh'
configure.ac:10: installing './missing'
Makefile.am: installing './depcomp'
autoreconf: Leaving directory `.'

Let’s make ./configure to change the binary program destination from /usr/local/bin to /usr/bin and to change the entire install destination from / to ./tmp for use by the binary distribution package build. Then let’s run it from ./tmp/usr/bin/hello.

$ ./configure --prefix=/usr
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for style of include used by make... GNU
checking dependency style of gcc... gcc3
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating config.h
config.status: executing depfiles commands
$ make
make[5]: Entering directory `/path/to/hello'
make  all-am
make[6]: Entering directory `/path/to/hello'
gcc -DHAVE_CONFIG_H -I.     -g -O2 -MT hello.o -MD -MP -MF .deps/hello.Tpo -c -o ...
mv -f .deps/hello.Tpo .deps/hello.Po
gcc  -g -O2   -o hello hello.o
make[6]: Leaving directory `/path/to/hello'
make[5]: Leaving directory `/path/to/hello'
$ DESTDIR=./debian/tmp make install
make[5]: Entering directory `/path/to/hello'
make[6]: Entering directory `/path/to/hello'
 /bin/mkdir -p './debian/tmp/usr/bin'
  /usr/bin/install -c hello './debian/tmp/usr/bin'
make[6]: Nothing to be done for `install-data-am'.
make[6]: Leaving directory `/path/to/hello'
make[5]: Leaving directory `/path/to/hello'
$ # now run it
$ ./debian/tmp/usr/bin/hello
Hello, world! version: 0.1

For more, Using GNU Autotools

TIP: If you find configure.in instead of configure.ac in the source tree, the source is very old. It is good idea to update the configuration files to match newer versions of the Autotools.

TIP: You should commit only hand-edited files to the upstream VCS. For Autotools, these are configure.ac, Makefile.am, */Makefile.am, AUTHORS, and NEWS(optional). Do not bother to track other generated files.

See :