Making a Makefile
tags: assembling, compiling, design, linking, make
make
is a GNU utility used for compiling programs and is the build system that
was chosen for welibc. It helps automate the build process so that you don't
need to memorize the specific compiler flags you want to use, remember exactly
how to link against a library, or rely on having your command history available
to compile your code. make
is not necessarily complicated but there are a few
pitfalls to getting started. We'll go over the creation of the makefile for
welibc to help guide you.
Directory Structure
In a previous post the directory structure of welibc was discussed but it was just a placeholder since no code had been written yet. Since we actually have some files now, here is what the structure looks like:
welibc/
├── docs
├── license.txt
├── README.md
├── src
│ └── _start.s
└── test
└── main_test.c
All source code will be inside the src/ directory and any code used to test the library will be placed in test/. This keeps files in logically separate places so the project doesn't become cluttered with assembly, C code, headers, and tests all mixed together.
Compiling without make
We only have two files here so compiling this project should be cake, right? Well, let's consider what we need to do:
- Generate a _start.o file from _start.s
- Create libwelibc.a from _start.o
- Compile, but not link, main_test.c
- Link main_test.o with libwelibc.a to create a test_welibc program
- Run the test
That seems manageable. Some of the compilation basics and commands for generating libraries were covered before so I won't go over them here. Instead, I'll just show the commands:
$ gcc -nostdlib -ffreestanding -c -o _start.o src/_start.s
$ ar cr libwelibc.a _start.o
$ gcc -nostdlib -ffreestanding -c -o main_test.o test/main_test.c
$ gcc -lwelibc -L. -nostdlib -ffreestanding -o test_welibc main_test.o
$ ./test_welibc
That's only five commands which isn't too many, but let's look a little more
closely. First, the compile flags are copied across three commands so if we want
to change them we'll need to remember to include the updated flags in all three
commands. Second, if we change something in _start.s then we'll need to go back
and run all five commands again; that's just hitting the up arrow and enter keys
five times though. Lastly, when (not if) we add more code we'll need to be sure
to include each one as appropriate which could add another entire command for
each new file we need to compile for the project. Well, make
can take care of
these things for us in an automated fashion so that we don't have to rely on
ourselves to remember each detail of the compilation process. After all, why
should we do it when the computer can do it for us?
Compiling with make
I'll only go over a few of the features of make
, but a deeper explanation can
be read from the source, the GNU Make Manual. To compile with make
you
must have a makefile with rules for compiling your project which you can run
from the command-line by ussuing the make
command.
Comments
You can include comments in your makefile to document how things work. Comments
are started with the #
character and can be placed on their own line, or at
the end of a line.
# This is my makefile. There are many like it, but this one is mine.
Rules
Rules are used to define something for make
to do, such as creating an object
file or linking an object file against a library. The name of the rule, called a
target, goes at the beginning of the line and is followed by a colon. The files
or other rules required to run a rule, called prerequisites, go on the same line
after the colon. The commands that should be run for that rule go on the
following lines and are prepended with a tab character; the combination of these
commands is called the recipe.
target: prerequisites
recipe
Variables
make
lets you define variables to use throughout your makefile, these are case
sensitive and have a few assignment operators, but we'll just use :=
and +=
for now. :=
assigns data to a variable while +=
appends more data to it. For
the most part, you can just consider variables to be character arrays or
strings. Variables are used by placing them inside of parenthesis and prepending
a dollar sign, like $(myvar)
.
myvar := some stuff
myvar += more things
Basic Makefile
The basic version of the makefile used to replicate the steps above looks like
so and is named Makefile
:
# Makefile
all: _start.o libwelibc.a test_welibc
_start.o: src/_start.s
gcc -nostdlib -ffreestanding -c -o _start.o src/_start.s
libwelibc.a: _start.o
ar cr libwelibc.a _start.o
test_welibc: libwelibc.a test/main_test.c
gcc -nostdlib -ffreestanding -c -o test_welibc.o test/main_test.c
gcc -lwelibc -L. -nostdlib -ffreestanding -o test_welibc test_welibc.o
./test_welibc
clean:
rm _start.o libwelibc.a test_welibc.o test_welibc
You can see that the same five commands from above are replicated here in
addition to an rm
command. We have separated out the commands into targets so
that each rule will build a specific piece of the project.
By default, make
will run the first rule that it comes across; in this
makefile that would be the "all" rule. This rule only specifies prerequisites
which, if not found or are older than the target, will be built before
continuing into the recipe portion of the rule. So, this will run the rules for
_start.o, libwelibc, and test_welibc before executing the recipe for the rule.
The recipe for this rule is empty so the make process will simply end after
executing the rules for each prerequisite.
The _start.o rule requires the src/_start.s file which is located in exactly the spot indicated. If the _start.o file is older the _start.s, or if _start.o doesn't exist then it will create it using the recipe. The same goes for the libwelibc.a rule and the test_welibc rule. However, for the test_welibc rule, the recipe has a few steps which are each run in order.
Since "clean" was not a prerequisite of the "all" rule, it won't be run. We can
run specific rules by passing the target as an argument on the command-line. For
example, if you want to run the rm
command in the "clean" rule, you would run
make clean
and it would clean up your files for you. Pretty nice!
Basic Enhancements
There are some simple things you can do to enhance your makefile a bit. We'll go
over those before getting into some more advanced things to solidify your make
skills.
Using Variables
You can see that the list of files to delete and the list of prerequisites for
the "all" rule are pretty similar. We should place these file
names/prerequisites into a variable so we don't have to worry about updating
things in two places. A standard variable name for object files would simply be
OBJECTS
. We'll split up the test_welibc rule into two pieces as well: one to
create test_welibc.o and one to link and run the test program.
# Makefile
OBJECTS := _start.o libwelibc.a test_welibc.o
all: $(OBJECTS) test_welibc
_start.o: src/_start.s
gcc -nostdlib -ffreestanding -c -o _start.o src/_start.s
libwelibc.a: _start.o
ar cr libwelibc.a _start.o
test_welibc.o: test/main_test.c
gcc -nostdlib -ffreestanding -c -o test_welibc.o test/main_test.c
test_welibc: libwelibc.a test_welibc.o
gcc -lwelibc -L. -nostdlib -ffreestanding -o test_welibc test_welibc.o
./test_welibc
clean:
rm $(OBJECTS) test_welibc
This has the same functionality except that when we add a new object file to be created we would add it to the objects variable and it automatically gets deleted without us having to manually update the "clean" rule.
The next set of variables that we'll add will cover the compiler being used and
the flags passed to the compiler. It's very common to see variables named CC
,
CFLAGS
, and LDFLAGS
in a makefile. These correspond to the compiler,
compiler flags, and linker flags, respectively.
# Makefile
CC := gcc
CFLAGS := -nostdlib -ffreestanding
LDFLAGS := -lwelibc -L.
OBJECTS := _start.o libwelibc.a test_welibc.o
all: $(OBJECTS) test_welibc
_start.o: src/_start.s
$(CC) $(CFLAGS) -c -o _start.o src/_start.s
libwelibc.a: _start.o
ar cr libwelibc.a _start.o
test_welibc.o: test/main_test.c
$(CC) $(CFLAGS) -c -o test_welibc.o test/main_test.c
test_welibc: libwelibc.a test_welibc.o
$(CC) $(LDFLAGS) $(CFLAGS) -o test_welibc test_welibc.o
./test_welibc
clean:
rm $(OBJECTS) test_welibc
Ignoring Errors and PHONYs
Let's say there was an error when test_welibc.o was created, so only _start.o
was made. We might want to clean up what was built, fix the error, and finally
recompile. However, the rm
command would fail because it can't find the rest
of the object files or test_welibc. You can prepend any command with a hyphen,
-
, to instruct make
to ignore any errors produced by that command. As well,
we can add the -rf
flags to rm
to remove any errors related to unfound files
or directories that are not empty.
Some targets will not have any prerequisites, as with our clean rule, and make
provides a special directive to mark rules which are a set of actions that don't
rely on specific files. This is the .PHONY
directive which you give a target
and make
will know to always run that target even though it has no files to
determine when things are out of date and need to be rebuilt. In addition to
this, if we end up with a file actually named "clean" then make
won't get
confused; it will know that the clean rule we have is for deleting files and
doesn't correspond to compiling a file named "clean". Here's the new all and
clean rules:
.PHONY: all
all: $(OBJECTS) test_welibc
.PHONY: clean
clean:
-rm -rf $(OBJECTS) test_welibc
Advanced makery
The following will assist in further automating the make
process and will also
clean up the makefile to make it easier to understand when changing it.
Automatic File Finding
We already know that we'll be adding more code to the project which means we'll
have to update the objects variable each time and add a new rule. Instead of
manually updating the objects variables we can automate finding files so we can
focus our attention on creating new rules. make
has a wildcard character, %
,
which can be combined with two built in functions, patsubst
(for pattern
substitution) and wildcard
, to generate a list of object files from every ".s"
file that is found. A new variable to indicate where our main source files would
be helpful as well. The additions look like:
SRCDIR := src
OBJECTS := $(patsubst $(SRCDIR)/%.s,%.o,$(wildcard $(SRCDIR)/*.s)) libwelibc.a
The $(wildcard $(SRCDIR)/*.s)
part will match every file in the src/ directory
that ends in ".s". The pattern substitution will then do a search and replace
where the %
character represents the filename. So, src/_start.s would become
_start.o and anytime we add a new ".s" file make
will find it automatically.
We can create similar variables just for the ".c" files in the test directory since they're no longer found with the changes we just made:
TSTDIR := test
TSTOBJS := $(patsubst $(TSTDIR)/%.c,%.o,$(wildcard $(TSTDIR)/*.c))
The rule for compiling the main_test.c file needs to change slightly, along with test_welibc and the prerequisite list for libwelibc.a; here's the new makefile:
# Makefile
CC := gcc
CFLAGS := -nostdlib -ffreestanding
LDFLAGS := -lwelibc -L.
SRCDIR := src
TSTDIR := test
OBJECTS := $(patsubst $(SRCDIR)/%.s,%.o,$(wildcard $(SRCDIR)/*.s)) libwelibc.a
TSTOBJS := $(patsubst $(TSTDIR)/%.c,%.o,$(wildcard $(TSTDIR)/*.c))
.PHONY: all
all: $(OBJECTS) $(TSTOBJS) test_welibc
_start.o: src/_start.s
$(CC) $(CFLAGS) -c -o _start.o src/_start.s
libwelibc.a: $(OBJECTS)
ar cr libwelibc.a _start.o
main_test.o: test/main_test.c
$(CC) $(CFLAGS) -c -o main_test.o test/main_test.c
test_welibc: libwelibc.a main_test.o
$(CC) $(LDFLAGS) $(CFLAGS) -o test_welibc main_test.o
./test_welibc
.PHONY: clean
clean:
-rm -rf $(OBJECTS) $(TSTOBJS) test_welibc
Virtual Paths and Automatic Variables
It's kind of annoying to have to specify the directory where a file exists and
it clutters the rules up having more than just file names. make
provides a
variable that you can fill in, VPATH
(remember this is case-sensitive), with
colon-separated locations to search for files when they aren't found in the
current directory. Sounds like a great thing to use for the src/ and test/
directories:
VPATH := $(SRCDIR):$(TSTDIR)
Now we can simply list the filenames in the prerequisite lists, rather than
prepending the directory. However, the recipe commands won't search these
directories since they are passed directly to your shell for execution. To get
around this, make
provides some automatic variables which are expanded before
passing a recipe command to the shell. The $^
variable corresponds to every
prerequisite file and is updated with the path and name found using VPATH. The
$@
variable corresponds to the target name and the $<
variable corresponds
to the first prerequisite. With these powers combined we get:
# Makefile
CC := gcc
CFLAGS := -nostdlib -ffreestanding
LDFLAGS := -lwelibc -L.
SRCDIR := src
TSTDIR := test
OBJECTS := $(patsubst $(SRCDIR)/%.s,%.o,$(wildcard $(SRCDIR)/*.s)) libwelibc.a
TSTOBJS := $(patsubst $(TSTDIR)/%.c,%.o,$(wildcard $(TSTDIR)/*.c))
VPATH := $(SRCDIR):$(TSTDIR)
.PHONY: all
all: $(OBJECTS) $(TSTOBJS) test_welibc
_start.o: _start.s
$(CC) $(CFLAGS) -c -o $@ $^
libwelibc.a: $(OBJECTS)
ar cr $@ $^
main_test.o: main_test.c
$(CC) $(CFLAGS) -c -o $@ $^
test_welibc: main_test.o libwelibc.a
$(CC) $(LDFLAGS) $(CFLAGS) -o $@ $<
./test_welibc
.PHONY: clean
clean:
-rm -rf $(OBJECTS) $(TSTOBJS) test_welibc
That simplifies the syntax quite a bit!
Wildcard Targets
We can use the same %
wildcard from before to create some wildcard rules so we
don't need a specific rule for each .o which gets built with the same command,
$(CC) $(CFLAGS) -c -o file.o file.c
. Remember that the %
expands to the
matched name so we can use it in the prerequisite list as well. We'll still
need two rules since some objects are made from ".s" files and others from ".c"
files:
# Makefile
CC := gcc
CFLAGS := -nostdlib -ffreestanding
LDFLAGS := -lwelibc -L.
SRCDIR := src
TSTDIR := test
OBJECTS := $(patsubst $(SRCDIR)/%.s,%.o,$(wildcard $(SRCDIR)/*.s)) libwelibc.a
TSTOBJS := $(patsubst $(TSTDIR)/%.c,%.o,$(wildcard $(TSTDIR)/*.c))
VPATH := $(SRCDIR):$(TSTDIR)
.PHONY: all
all: $(OBJECTS) $(TSTOBJS) test_welibc
_%.o: _%.s
$(CC) $(CFLAGS) -c -o $@ $^
%.o: %.c
$(CC) $(CFLAGS) -c -o $@ $^
libwelibc.a: $(OBJECTS)
ar cr $@ $^
test_welibc: main_test.o libwelibc.a
$(CC) $(LDFLAGS) $(CFLAGS) -o $@ $<
./test_welibc
.PHONY: clean
clean:
-rm -rf $(OBJECTS) $(TSTOBJS) test_welibc
That's not a huge improvement at the moment, but it's the other half of automatic file finding since now we have automatic rule generation for objects which don't require any special recipe.
Generic Tricks
The next few iterations of the makefile expand on the things weve already covered.
First, we'll add variables for the name of our library and the name of the test
in addition to creating a "test" rule that is separate from the "all" rule so
that we don't test every time we build the library. To test, you would use the
make test
command. Also, we'll clean up all the files before rebuilding with
the "all" target by making "clean" a prerequisite for the "all" rule. The
$(TSTNAME)
rule uses a pattern substitution to remove $(LIBNAME)
from the
list of object files that are linked together. It's redundant to provide
-lwelibc -L.
(the $(LDFLAGS)
variable) and also provide libwelibc.a in the
list of files to link. Further, we want to make a distinction between them as
the proper way to compile this would be to generate the test object file and
then link it against our library (which is what we accomplish here).
# Makefile
CC := gcc
CFLAGS := -nostdlib -ffreestanding
LDFLAGS := -lwelibc -L.
SRCDIR := src
TSTDIR := test
LIBNAME := libwelibc.a
TSTNAME := test_welibc
OBJECTS := $(patsubst $(SRCDIR)/%.s,%.o,$(wildcard $(SRCDIR)/*.s)) $(LIBNAME)
TSTOBJS := $(patsubst $(TSTDIR)/%.c,%.o,$(wildcard $(TSTDIR)/*.c))
VPATH := $(SRCDIR):$(TSTDIR)
.PHONY: all
all: clean $(OBJECTS)
.PHONY: test
test: $(TSTNAME)
./$<
_%.o: _%.s
$(CC) $(CFLAGS) -c -o $@ $^
%.o: %.c
$(CC) $(CFLAGS) -c -o $@ $^
$(LIBNAME): $(OBJECTS)
ar cr $@ $^
$(TSTNAME): $(LIBNAME) $(TSTOBJS)
$(CC) $(LDFLAGS) $(CFLAGS) -o $@ $(patsubst $(LIBNAME),,$^)
.PHONY: clean
clean:
-rm -rf $(OBJECTS) $(TSTOBJS) test_welibc
Next, we'll add an install rule to place the final libwelibc.a static library into the /usr/lib folder on the machine, as well as an uninstall rule to remove it.
# Makefile
CC := gcc
CFLAGS := -nostdlib -ffreestanding
LDFLAGS := -lwelibc -L.
INSTALL := install
SRCDIR := src
TSTDIR := test
LIBNAME := libwelibc.a
TSTNAME := test_welibc
OBJECTS := $(patsubst $(SRCDIR)/%.s,%.o,$(wildcard $(SRCDIR)/*.s)) $(LIBNAME)
TSTOBJS := $(patsubst $(TSTDIR)/%.c,%.o,$(wildcard $(TSTDIR)/*.c))
VPATH := $(SRCDIR):$(TSTDIR)
.PHONY: all
all: clean $(OBJECTS)
.PHONY: test
test: $(TSTNAME)
./$<
.PHONY: install
install: $(LIBNAME)
$(INSTALL) $^ /usr/lib
.PHONY: uninstall
uninstall: $(LIBNAME)
-rm -rf /usr/lib/$^
_%.o: _%.s
$(CC) $(CFLAGS) -c -o $@ $^
%.o: %.c
$(CC) $(CFLAGS) -c -o $@ $^
$(LIBNAME): $(OBJECTS)
ar cr $@ $^
$(TSTNAME): $(LIBNAME) $(TSTOBJS)
$(CC) $(LDFLAGS) $(CFLAGS) -o $@ $(patsubst $(LIBNAME),,$^)
.PHONY: clean
clean:
-rm -rf $(OBJECTS) $(TSTOBJS) test_welibc
Lastly, the GNU Make Manual has an entire chapter dedicated to conventions that are commonly used with makefiles and is a great source of enhancements to add. I've chosen just a few to add to the last iteration of the makefile, for now.
First, we'll add INSTALL_PROGRAM
and INSTALL_DATA
which are to be used for
installing programs and installing binaries onto a system. Along with these
we'll add LIBDIR
to specify the default destination which we also prepend with
DESTDIR
so that the command-line flag, --destdir
, can be used to specify an
alternate installation destination. They also instruct you to include the
SHELL
variable set to the bourne shell specifically. The first instance of
.SUFFIXES
clears out all the suffixes being searched and the second specifies
which file suffixes are to be searched. This may save some time on larger
projects but probably won't matter for this one. The manual also suggests
placing any build programs into a variable (like ar
and gcc
), but not to do
so with shell utilies (like rm
). Finally, we end up with the following:
# Makefile
AR := ar
CC := gcc
CFLAGS := -nostdlib -ffreestanding
LDFLAGS := -lwelibc -L.
INSTALL := install
INSTALL_PROGRAM := $(INSTALL)
INSTALL_DATA := $(INSTALL) -m 644
SHELL := /bin/sh
LIBDIR := /usr/lib
SRCDIR := src
TSTDIR := test
LIBNAME := libwelibc.a
TSTNAME := test_welibc
OBJECTS := $(patsubst $(SRCDIR)/%.s,%.o,$(wildcard $(SRCDIR)/*.s)) $(LIBNAME)
TSTOBJS := $(patsubst $(TSTDIR)/%.c,%.o,$(wildcard $(TSTDIR)/*.c))
VPATH := $(SRCDIR):$(TSTDIR)
.SUFFIXES:
.SUFFIXES: .c .o .s
.PHONY: all
all: clean $(OBJECTS)
.PHONY: test
test: $(TSTNAME)
./$<
.PHONY: install
install: $(LIBNAME)
$(INSTALL) $^ $(DESTDIR)$(LIBDIR)/$(LIBNAME)
.PHONY: uninstall
uninstall: $(LIBNAME)
-rm -rf $(DESTDIR)$(LIBDIR)/$(LIBNAME)
_%.o: _%.s
$(CC) $(CFLAGS) -c -o $@ $^
%.o: %.c
$(CC) $(CFLAGS) -c -o $@ $^
$(LIBNAME): $(OBJECTS)
$(AR) cr $@ $^
$(TSTNAME): $(LIBNAME) $(TSTOBJS)
$(CC) $(LDFLAGS) $(CFLAGS) -o $@ $(patsubst $(LIBNAME),,$^)
.PHONY: clean
clean:
-rm -rf $(OBJECTS) $(TSTOBJS) test_welibc
Congratulations, your makefile has officially been pimped.
That's quite a bit more advanced than what we started with and we're only compiling two files! However, a lot of the changes are intended to ease the "pain" of adding more files to the library later on, as well as expanding the test suite. So, we did a bit of heavy lifting on the front-end here but hopefully this makefile will last a while with only minor tweaks along the way.
This wouldn't be complete without mentioning Paul's Rules of Makefiles
which is almost always referenced when people ask make
questions. I've tried
to abide by the rules and I'll admit that I wanted to do things differently but
sticking to the rules actually made things easier in the end.