make is a GNU utility used for compiling programs and is the build system that was chosen for welibc. It helps automate the build process so that you don't need to memorize the specific compiler flags you want to use, remember exactly how to link against a library, or rely on having your command history available to compile your code. make is not necessarily complicated but there are a few pitfalls to getting started. We'll go over the creation of the makefile for welibc to help guide you.

Directory Structure

In a previous post the directory structure of welibc was discussed but it was just a placeholder since no code had been written yet. Since we actually have some files now, here is what the structure looks like:

welibc/
    ├── docs
    ├── license.txt
    ├── README.md
    ├── src
    │   └── _start.s
    └── test
        └── main_test.c

All source code will be inside the src/ directory and any code used to test the library will be placed in test/. This keeps files in logically separate places so the project doesn't become cluttered with assembly, C code, headers, and tests all mixed together.

Compiling without make

We only have two files here so compiling this project should be cake, right? Well, let's consider what we need to do:

  • Generate a _start.o file from _start.s
  • Create libwelibc.a from _start.o
  • Compile, but not link, main_test.c
  • Link main_test.o with libwelibc.a to create a test_welibc program
  • Run the test

That seems manageable. Some of the compilation basics and commands for generating libraries were covered before so I won't go over them here. Instead, I'll just show the commands:

$ gcc -nostdlib -ffreestanding -c -o _start.o src/_start.s
$ ar cr libwelibc.a _start.o
$ gcc -nostdlib -ffreestanding -c -o main_test.o test/main_test.c
$ gcc -lwelibc -L. -nostdlib -ffreestanding -o test_welibc main_test.o
$ ./test_welibc

That's only five commands which isn't too many, but let's look a little more closely. First, the compile flags are copied across three commands so if we want to change them we'll need to remember to include the updated flags in all three commands. Second, if we change something in _start.s then we'll need to go back and run all five commands again; that's just hitting the up arrow and enter keys five times though. Lastly, when (not if) we add more code we'll need to be sure to include each one as appropriate which could add another entire command for each new file we need to compile for the project. Well, make can take care of these things for us in an automated fashion so that we don't have to rely on ourselves to remember each detail of the compilation process. After all, why should we do it when the computer can do it for us?

Compiling with make

I'll only go over a few of the features of make, but a deeper explanation can be read from the source, the GNU Make Manual. To compile with make you must have a makefile with rules for compiling your project which you can run from the command-line by ussuing the make command.

Comments

You can include comments in your makefile to document how things work. Comments are started with the # character and can be placed on their own line, or at the end of a line.

# This is my makefile. There are many like it, but this one is mine.

Rules

Rules are used to define something for make to do, such as creating an object file or linking an object file against a library. The name of the rule, called a target, goes at the beginning of the line and is followed by a colon. The files or other rules required to run a rule, called prerequisites, go on the same line after the colon. The commands that should be run for that rule go on the following lines and are prepended with a tab character; the combination of these commands is called the recipe.

target: prerequisites
    recipe

Variables

make lets you define variables to use throughout your makefile, these are case sensitive and have a few assignment operators, but we'll just use := and += for now. := assigns data to a variable while += appends more data to it. For the most part, you can just consider variables to be character arrays or strings. Variables are used by placing them inside of parenthesis and prepending a dollar sign, like $(myvar).

myvar := some stuff
myvar += more things

Basic Makefile

The basic version of the makefile used to replicate the steps above looks like so and is named Makefile:

# Makefile

all: _start.o libwelibc.a test_welibc

_start.o: src/_start.s
    gcc -nostdlib -ffreestanding -c -o _start.o src/_start.s

libwelibc.a: _start.o
    ar cr libwelibc.a _start.o

test_welibc: libwelibc.a test/main_test.c
    gcc -nostdlib -ffreestanding -c -o test_welibc.o test/main_test.c
    gcc -lwelibc -L. -nostdlib -ffreestanding -o test_welibc test_welibc.o
    ./test_welibc

clean:
    rm _start.o libwelibc.a test_welibc.o test_welibc

You can see that the same five commands from above are replicated here in addition to an rm command. We have separated out the commands into targets so that each rule will build a specific piece of the project.

By default, make will run the first rule that it comes across; in this makefile that would be the "all" rule. This rule only specifies prerequisites which, if not found or are older than the target, will be built before continuing into the recipe portion of the rule. So, this will run the rules for _start.o, libwelibc, and test_welibc before executing the recipe for the rule. The recipe for this rule is empty so the make process will simply end after executing the rules for each prerequisite.

The _start.o rule requires the src/_start.s file which is located in exactly the spot indicated. If the _start.o file is older the _start.s, or if _start.o doesn't exist then it will create it using the recipe. The same goes for the libwelibc.a rule and the test_welibc rule. However, for the test_welibc rule, the recipe has a few steps which are each run in order.

Since "clean" was not a prerequisite of the "all" rule, it won't be run. We can run specific rules by passing the target as an argument on the command-line. For example, if you want to run the rm command in the "clean" rule, you would run make clean and it would clean up your files for you. Pretty nice!

Basic Enhancements

There are some simple things you can do to enhance your makefile a bit. We'll go over those before getting into some more advanced things to solidify your make skills.

Using Variables

You can see that the list of files to delete and the list of prerequisites for the "all" rule are pretty similar. We should place these file names/prerequisites into a variable so we don't have to worry about updating things in two places. A standard variable name for object files would simply be OBJECTS. We'll split up the test_welibc rule into two pieces as well: one to create test_welibc.o and one to link and run the test program.

# Makefile
OBJECTS := _start.o libwelibc.a test_welibc.o

all: $(OBJECTS) test_welibc

_start.o: src/_start.s
    gcc -nostdlib -ffreestanding -c -o _start.o src/_start.s

libwelibc.a: _start.o
    ar cr libwelibc.a _start.o

test_welibc.o: test/main_test.c
    gcc -nostdlib -ffreestanding -c -o test_welibc.o test/main_test.c

test_welibc: libwelibc.a test_welibc.o
    gcc -lwelibc -L. -nostdlib -ffreestanding -o test_welibc test_welibc.o
    ./test_welibc

clean:
    rm $(OBJECTS) test_welibc

This has the same functionality except that when we add a new object file to be created we would add it to the objects variable and it automatically gets deleted without us having to manually update the "clean" rule.

The next set of variables that we'll add will cover the compiler being used and the flags passed to the compiler. It's very common to see variables named CC, CFLAGS, and LDFLAGS in a makefile. These correspond to the compiler, compiler flags, and linker flags, respectively.

# Makefile
CC      := gcc
CFLAGS  := -nostdlib -ffreestanding
LDFLAGS := -lwelibc -L.
OBJECTS := _start.o libwelibc.a test_welibc.o

all: $(OBJECTS) test_welibc

_start.o: src/_start.s
    $(CC) $(CFLAGS) -c -o _start.o src/_start.s

libwelibc.a: _start.o
    ar cr libwelibc.a _start.o

test_welibc.o: test/main_test.c
    $(CC) $(CFLAGS) -c -o test_welibc.o test/main_test.c

test_welibc: libwelibc.a test_welibc.o
    $(CC) $(LDFLAGS) $(CFLAGS) -o test_welibc test_welibc.o
    ./test_welibc

clean:
    rm $(OBJECTS) test_welibc

Ignoring Errors and PHONYs

Let's say there was an error when test_welibc.o was created, so only _start.o was made. We might want to clean up what was built, fix the error, and finally recompile. However, the rm command would fail because it can't find the rest of the object files or test_welibc. You can prepend any command with a hyphen, -, to instruct make to ignore any errors produced by that command. As well, we can add the -rf flags to rm to remove any errors related to unfound files or directories that are not empty.

Some targets will not have any prerequisites, as with our clean rule, and make provides a special directive to mark rules which are a set of actions that don't rely on specific files. This is the .PHONY directive which you give a target and make will know to always run that target even though it has no files to determine when things are out of date and need to be rebuilt. In addition to this, if we end up with a file actually named "clean" then make won't get confused; it will know that the clean rule we have is for deleting files and doesn't correspond to compiling a file named "clean". Here's the new all and clean rules:

.PHONY: all
all: $(OBJECTS) test_welibc

.PHONY: clean
clean:
    -rm -rf $(OBJECTS) test_welibc

Advanced makery

The following will assist in further automating the make process and will also clean up the makefile to make it easier to understand when changing it.

Automatic File Finding

We already know that we'll be adding more code to the project which means we'll have to update the objects variable each time and add a new rule. Instead of manually updating the objects variables we can automate finding files so we can focus our attention on creating new rules. make has a wildcard character, %, which can be combined with two built in functions, patsubst (for pattern substitution) and wildcard, to generate a list of object files from every ".s" file that is found. A new variable to indicate where our main source files would be helpful as well. The additions look like:

SRCDIR  := src
OBJECTS := $(patsubst $(SRCDIR)/%.s,%.o,$(wildcard $(SRCDIR)/*.s)) libwelibc.a

The $(wildcard $(SRCDIR)/*.s) part will match every file in the src/ directory that ends in ".s". The pattern substitution will then do a search and replace where the % character represents the filename. So, src/_start.s would become _start.o and anytime we add a new ".s" file make will find it automatically.

We can create similar variables just for the ".c" files in the test directory since they're no longer found with the changes we just made:

TSTDIR  := test
TSTOBJS := $(patsubst $(TSTDIR)/%.c,%.o,$(wildcard $(TSTDIR)/*.c))

The rule for compiling the main_test.c file needs to change slightly, along with test_welibc and the prerequisite list for libwelibc.a; here's the new makefile:

# Makefile
CC      := gcc
CFLAGS  := -nostdlib -ffreestanding
LDFLAGS := -lwelibc -L.
SRCDIR  := src
TSTDIR  := test
OBJECTS := $(patsubst $(SRCDIR)/%.s,%.o,$(wildcard $(SRCDIR)/*.s)) libwelibc.a
TSTOBJS := $(patsubst $(TSTDIR)/%.c,%.o,$(wildcard $(TSTDIR)/*.c))

.PHONY: all
all: $(OBJECTS) $(TSTOBJS) test_welibc

_start.o: src/_start.s
    $(CC) $(CFLAGS) -c -o _start.o src/_start.s

libwelibc.a: $(OBJECTS)
    ar cr libwelibc.a _start.o

main_test.o: test/main_test.c
    $(CC) $(CFLAGS) -c -o main_test.o test/main_test.c

test_welibc: libwelibc.a main_test.o
    $(CC) $(LDFLAGS) $(CFLAGS) -o test_welibc main_test.o
    ./test_welibc

.PHONY: clean
clean:
    -rm -rf $(OBJECTS) $(TSTOBJS) test_welibc

Virtual Paths and Automatic Variables

It's kind of annoying to have to specify the directory where a file exists and it clutters the rules up having more than just file names. make provides a variable that you can fill in, VPATH (remember this is case-sensitive), with colon-separated locations to search for files when they aren't found in the current directory. Sounds like a great thing to use for the src/ and test/ directories:

VPATH   := $(SRCDIR):$(TSTDIR)

Now we can simply list the filenames in the prerequisite lists, rather than prepending the directory. However, the recipe commands won't search these directories since they are passed directly to your shell for execution. To get around this, make provides some automatic variables which are expanded before passing a recipe command to the shell. The $^ variable corresponds to every prerequisite file and is updated with the path and name found using VPATH. The $@ variable corresponds to the target name and the $< variable corresponds to the first prerequisite. With these powers combined we get:

# Makefile
CC      := gcc
CFLAGS  := -nostdlib -ffreestanding
LDFLAGS := -lwelibc -L.
SRCDIR  := src
TSTDIR  := test
OBJECTS := $(patsubst $(SRCDIR)/%.s,%.o,$(wildcard $(SRCDIR)/*.s)) libwelibc.a
TSTOBJS := $(patsubst $(TSTDIR)/%.c,%.o,$(wildcard $(TSTDIR)/*.c))
VPATH   := $(SRCDIR):$(TSTDIR)

.PHONY: all
all: $(OBJECTS) $(TSTOBJS) test_welibc

_start.o: _start.s
    $(CC) $(CFLAGS) -c -o $@ $^

libwelibc.a: $(OBJECTS)
    ar cr $@ $^

main_test.o: main_test.c
    $(CC) $(CFLAGS) -c -o $@ $^

test_welibc: main_test.o libwelibc.a
    $(CC) $(LDFLAGS) $(CFLAGS) -o $@ $<
    ./test_welibc

.PHONY: clean
clean:
    -rm -rf $(OBJECTS) $(TSTOBJS) test_welibc

That simplifies the syntax quite a bit!

Wildcard Targets

We can use the same % wildcard from before to create some wildcard rules so we don't need a specific rule for each .o which gets built with the same command, $(CC) $(CFLAGS) -c -o file.o file.c. Remember that the % expands to the matched name so we can use it in the prerequisite list as well. We'll still need two rules since some objects are made from ".s" files and others from ".c" files:

# Makefile
CC      := gcc
CFLAGS  := -nostdlib -ffreestanding
LDFLAGS := -lwelibc -L.
SRCDIR  := src
TSTDIR  := test
OBJECTS := $(patsubst $(SRCDIR)/%.s,%.o,$(wildcard $(SRCDIR)/*.s)) libwelibc.a
TSTOBJS := $(patsubst $(TSTDIR)/%.c,%.o,$(wildcard $(TSTDIR)/*.c))
VPATH   := $(SRCDIR):$(TSTDIR)

.PHONY: all
all: $(OBJECTS) $(TSTOBJS) test_welibc

_%.o: _%.s
    $(CC) $(CFLAGS) -c -o $@ $^

%.o: %.c
    $(CC) $(CFLAGS) -c -o $@ $^

libwelibc.a: $(OBJECTS)
    ar cr $@ $^

test_welibc: main_test.o libwelibc.a
    $(CC) $(LDFLAGS) $(CFLAGS) -o $@ $<
    ./test_welibc

.PHONY: clean
clean:
    -rm -rf $(OBJECTS) $(TSTOBJS) test_welibc

That's not a huge improvement at the moment, but it's the other half of automatic file finding since now we have automatic rule generation for objects which don't require any special recipe.

Generic Tricks

The next few iterations of the makefile expand on the things weve already covered.

First, we'll add variables for the name of our library and the name of the test in addition to creating a "test" rule that is separate from the "all" rule so that we don't test every time we build the library. To test, you would use the make test command. Also, we'll clean up all the files before rebuilding with the "all" target by making "clean" a prerequisite for the "all" rule. The $(TSTNAME) rule uses a pattern substitution to remove $(LIBNAME) from the list of object files that are linked together. It's redundant to provide -lwelibc -L. (the $(LDFLAGS) variable) and also provide libwelibc.a in the list of files to link. Further, we want to make a distinction between them as the proper way to compile this would be to generate the test object file and then link it against our library (which is what we accomplish here).

# Makefile
CC      := gcc
CFLAGS  := -nostdlib -ffreestanding
LDFLAGS := -lwelibc -L.
SRCDIR  := src
TSTDIR  := test
LIBNAME := libwelibc.a
TSTNAME := test_welibc
OBJECTS := $(patsubst $(SRCDIR)/%.s,%.o,$(wildcard $(SRCDIR)/*.s)) $(LIBNAME)
TSTOBJS := $(patsubst $(TSTDIR)/%.c,%.o,$(wildcard $(TSTDIR)/*.c))
VPATH   := $(SRCDIR):$(TSTDIR)

.PHONY: all
all: clean $(OBJECTS)

.PHONY: test
test: $(TSTNAME)
    ./$<

_%.o: _%.s
    $(CC) $(CFLAGS) -c -o $@ $^

%.o: %.c
    $(CC) $(CFLAGS) -c -o $@ $^

$(LIBNAME): $(OBJECTS)
    ar cr $@ $^

$(TSTNAME): $(LIBNAME) $(TSTOBJS)
    $(CC) $(LDFLAGS) $(CFLAGS) -o $@ $(patsubst $(LIBNAME),,$^)

.PHONY: clean
clean:
    -rm -rf $(OBJECTS) $(TSTOBJS) test_welibc

Next, we'll add an install rule to place the final libwelibc.a static library into the /usr/lib folder on the machine, as well as an uninstall rule to remove it.

# Makefile
CC      := gcc
CFLAGS  := -nostdlib -ffreestanding
LDFLAGS := -lwelibc -L.
INSTALL := install
SRCDIR  := src
TSTDIR  := test
LIBNAME := libwelibc.a
TSTNAME := test_welibc
OBJECTS := $(patsubst $(SRCDIR)/%.s,%.o,$(wildcard $(SRCDIR)/*.s)) $(LIBNAME)
TSTOBJS := $(patsubst $(TSTDIR)/%.c,%.o,$(wildcard $(TSTDIR)/*.c))
VPATH   := $(SRCDIR):$(TSTDIR)

.PHONY: all
all: clean $(OBJECTS)

.PHONY: test
test: $(TSTNAME)
    ./$<

.PHONY: install
install: $(LIBNAME)
    $(INSTALL) $^ /usr/lib

.PHONY: uninstall
uninstall: $(LIBNAME)
    -rm -rf /usr/lib/$^

_%.o: _%.s
    $(CC) $(CFLAGS) -c -o $@ $^

%.o: %.c
    $(CC) $(CFLAGS) -c -o $@ $^

$(LIBNAME): $(OBJECTS)
    ar cr $@ $^

$(TSTNAME): $(LIBNAME) $(TSTOBJS)
    $(CC) $(LDFLAGS) $(CFLAGS) -o $@ $(patsubst $(LIBNAME),,$^)

.PHONY: clean
clean:
    -rm -rf $(OBJECTS) $(TSTOBJS) test_welibc

Lastly, the GNU Make Manual has an entire chapter dedicated to conventions that are commonly used with makefiles and is a great source of enhancements to add. I've chosen just a few to add to the last iteration of the makefile, for now.

First, we'll add INSTALL_PROGRAM and INSTALL_DATA which are to be used for installing programs and installing binaries onto a system. Along with these we'll add LIBDIR to specify the default destination which we also prepend with DESTDIR so that the command-line flag, --destdir, can be used to specify an alternate installation destination. They also instruct you to include the SHELL variable set to the bourne shell specifically. The first instance of .SUFFIXES clears out all the suffixes being searched and the second specifies which file suffixes are to be searched. This may save some time on larger projects but probably won't matter for this one. The manual also suggests placing any build programs into a variable (like ar and gcc), but not to do so with shell utilies (like rm). Finally, we end up with the following:

# Makefile
AR      := ar
CC      := gcc
CFLAGS  := -nostdlib -ffreestanding
LDFLAGS := -lwelibc -L.
INSTALL := install
INSTALL_PROGRAM := $(INSTALL)
INSTALL_DATA    := $(INSTALL) -m 644
SHELL   := /bin/sh
LIBDIR  := /usr/lib
SRCDIR  := src
TSTDIR  := test
LIBNAME := libwelibc.a
TSTNAME := test_welibc
OBJECTS := $(patsubst $(SRCDIR)/%.s,%.o,$(wildcard $(SRCDIR)/*.s)) $(LIBNAME)
TSTOBJS := $(patsubst $(TSTDIR)/%.c,%.o,$(wildcard $(TSTDIR)/*.c))
VPATH   := $(SRCDIR):$(TSTDIR)

.SUFFIXES:
.SUFFIXES: .c .o .s

.PHONY: all
all: clean $(OBJECTS)

.PHONY: test
test: $(TSTNAME)
    ./$<

.PHONY: install
install: $(LIBNAME)
    $(INSTALL) $^ $(DESTDIR)$(LIBDIR)/$(LIBNAME)

.PHONY: uninstall
uninstall: $(LIBNAME)
    -rm -rf $(DESTDIR)$(LIBDIR)/$(LIBNAME)

_%.o: _%.s
    $(CC) $(CFLAGS) -c -o $@ $^

%.o: %.c
    $(CC) $(CFLAGS) -c -o $@ $^

$(LIBNAME): $(OBJECTS)
    $(AR) cr $@ $^

$(TSTNAME): $(LIBNAME) $(TSTOBJS)
    $(CC) $(LDFLAGS) $(CFLAGS) -o $@ $(patsubst $(LIBNAME),,$^)

.PHONY: clean
clean:
    -rm -rf $(OBJECTS) $(TSTOBJS) test_welibc

Congratulations, your makefile has officially been pimped.

That's quite a bit more advanced than what we started with and we're only compiling two files! However, a lot of the changes are intended to ease the "pain" of adding more files to the library later on, as well as expanding the test suite. So, we did a bit of heavy lifting on the front-end here but hopefully this makefile will last a while with only minor tweaks along the way.

This wouldn't be complete without mentioning Paul's Rules of Makefiles which is almost always referenced when people ask make questions. I've tried to abide by the rules and I'll admit that I wanted to do things differently but sticking to the rules actually made things easier in the end.

comments powered by Disqus