12 KiB
ngen
Build file generator (engine) for the Ninja build system.
Licensed under the GPLv3.
Methodology and Overview
The problem with existing meta build systems---or makefile generators---like Meson and CMake is that they are needlessly complex for small-to-medium-sized C/C++ projects. These projects are just large enough that hand-writing a makefile would be painful, but trying to configure a large meta build system would be just as painful. It is unacceptable that building a simple project with 10 .c files and a library requires knowledge of a unique, obscure configuration language which is constantly changing.
ngen aims to make generating build files for these small C/C++ projects as simple as possible by using a basic key-value configuration file using patterns that any experienced programmer should be familiar with. In doing so, ngen fills the gap between writing your own makefile and wrangling with CMakeLists.txt.
ngen generates files for the small and modern Ninja
build system. "Where other build systems are high-level languages Ninja aims to
be an assembler," according to Ninja's website. It can be thought of as a
simpler, faster replacement for the classic make. It is used by default by
Meson; CMake can also be configured to use ninja as a backend. Ninja is used by
used by Google to build Chromium, v8, etc., and it is also used to build LLVM.
(All this to say, Ninja is commonly used and it is likely installed on your
system already.)
Building
Build: cargo build --release
Install: sudo sh install.sh
Uninstall: sudo sh uninstall.sh
Usage
What follows is a tutorial of how to set up ngen for an existing executable project.
Basics
The first thing you will need is an ngen.toml file. This is what will specify
all of the build parameters, such as compilation flags, source files to track,
etc. Start by creating this file and opening it in your editor.
The simplest valid ngen.toml just lists the source files you want to track.
Lets say you have a executable project that looks like this:
example
├── ngen.toml
└── src
├── foobar.c
├── functions.c
├── include
│ ├── foobar.h
│ ├── functions.h
│ ├── main.h
│ └── util.h
├── main.c
└── util.c
In your ngen.toml, write the following:
[targets.main]
sources = [
"src/main.c",
"src/util.c",
"src/functions.c",
"src/foobar.c",
]
The sources key is a list of strings, each specifying a single source
file name.
Now run ngen. This will generate a build.ninja file in the current working
directory. You won't ever have to touch this file; that's what ngen is for. You
also won't ever have to run ngen yourself again (unless your build.ninja
gets deleted); Ninja will take care of regenerating the build file if
ngen.toml changes.
With your build.ninja generated, run ninja on the command line. That's it!
Your project is now built, you will find the executable at build/main/a.out.
Remember, you can also freely add and remove files from the above list without
running ngen again: Ninja will regenerate the build.ninja for you.
Now, while this is functional, it isn't very useful. It is very likely that you will want to specify a compiler (gcc/clang), pass some flags, link some libraries into your final executable, and definitely name your program something other than "a.out." ngen makes these things dead simple, too.
The outfile key is a string that specifies the name of the file produced
by the linker (see below).
Lets set this to "example."
outfile = "example"
The compiler key is a string that specifies the program that will be used
to turn .c files into .o files.
By default, if not specified, compiler is set to "cc," which on Linux systems
should be a C compiler. It should be noted that the compiler you choose must
support the -MD and -MF flags to generate dependency files (both gcc and
clang support this). Lets say we want to use "gcc." Add the following line to
your ngen.toml:
compiler = "gcc"
The compiler_flags key is a list of strings that contains the arguments
to be passed to the compiler during the compilation of each source file.
It is not necessary to add the -c or -o outfile flags, ngen will take care
of this for you. For example, add the following to your ngen.toml:
compiler_flags = ["-Wall", "-Wextra -O2"]
The linker key is a string that specifies the program that will be used to
combine the .o files into the final outfile.
If the linker key is not found, it will be set to the value of compiler. For
this example, we don't have to change anything here.
The linker_flags key is a list of strings that contains the arguments to
be passed to the linker during the linking of the outfile.
Library flags (-lm, -lyourlib) should NOT be included here. This is for
linker options, not libraries. The syntax is the same as compiler_flags. There
is nothing we have to set here.
The linker_libs key is a list of strings that contains the link library
arguments to be linked to the outfile.
THIS is where library flags (-lm, -lyourlib) go. Lets say our example
project needs the math library:
linker_libs = ["-lm"]
Now, our ngen.toml looks like this:
[targets.main]
outfile = "example"
compiler = "gcc"
compiler_flags = ["-Wall", "-Wextra -O2"]
linker_libs = ["-lm"]
sources = [
"src/main.c",
"src/util.c",
"src/functions.c",
"src/foobar.c",
]
This is a much more realistic looking project. Once again, any changes to any of
these values will be automatically picked up by Ninja and accounted for in the
build. Running ninja -v immediately after saving ngen.toml should show that
the options you set were recognized, and your files were rebuilt accordingly.
Now, this is a good start. But, it often the case that, in a project, you want to have multiple different build targets, or configurations, that build the project with slightly different parameters; for example, it is common to have a "debug" build target that is unoptimized and includes debugging symbols, and a "release" build that is optimized at compile time. ngen is designed to make this configuration as easy as possible.
Targets
When we were specifying parameters above, we were doing so in the main target
[targets.main]. A "target" is a self-contained build process that builds its
outfile from the sources and other paramenters provided. We could have named
this target whatever we wanted, but the target named main is special, as we shall
presently see. Lets create a new target called debug. Add the following to your ngen.toml:
[targets.debug]
outfile = "example_dbg"
compiler_flags = ["-g"]
What's going on here? How does debug know what files to operate on, what
compiler to use, etc? Well, the main target is special: all targets inherit
the parameters set in the main target. Inheritance works according to two simple
rules: arrays append, strings supercede.
The first thing that happens is debug takes on all the same parameters from
main. Then, ngen reads the outfile key in debug. Becuase outfile is a string,
debug.outfile is overwritten as the value specified, in this case
"example_dbg". On the other hand, since compiler_flags is a list, the elements
sepecified in debug.compiler_flags are appended to the list of flags specified
in main. So in this case, the effective value of debug.compiler_flags is
["-Wall", "-Wextra -O2", "-g"].
For our debug build, we probably don't want the -O2 flag---optimizations
should only happen in a "release" type build. So, lets remove the -O2 flag
from main and place it in a new target called release. Our ngen.toml
should now look like this:
[targets.main]
outfile = "example"
compiler = "gcc"
compiler_flags = ["-Wall", "-Wextra"]
linker_libs = ["-lm"]
sources = [
"src/main.c",
"src/util.c",
"src/functions.c",
"src/foobar.c",
]
[targets.debug]
outfile = "example_dbg"
compiler_flags = ["-g"]
[targets.release]
compiler_flags = ["-O2"]
This brings up an important design pattern you should keep in mind when writing
your ngen.toml: the special main target should only contain the largest
subset of all your build parameters. Additonal targets should add specific
parameters for specific use cases, as we saw in the above example.
In a nutshell, inheritance allows you to easily create multiple targets with
small variations, without having to rewrite the same thing over and over again.
You can always disable inheritance using the opts.inherit = false key on
targets that you do not want to inherit from main. You can also change the
parent target that a target inherits from using the opts.inherit_from = "target" key, replacing target with the name of the desired parent target.
Save ngen.toml, and try running ninja -v debug or ninja -v release. You
should see that each of these targets uses the parameters that we specified with
inheritance. Outfiles for a given target are always placed in
build/<target_name>/, as you can see with tree build. This keeps things
organized, and also means we don't have to specify a different outfile name for
each target (I did above just to show you how strings are replaced in the
inheritance system).
Note that by default, running ninja alone with no target specifed will run
every single target it finds. You can change this behavor by adding the
opts.default = true key to the targets you want to be built when Ninja is invoked
with no arguments. Say that this example project is under active developemnt,
and you will be building the debug target alot. You can add the opts.default = true flag to the [debug] table, and now running ninja by itself will only
build the debug target. You can still build the release and main targets by
running ninja release and ninja main.
It is easy to see how powerful this simple configuration file is already. However, ngen has a few more features that you may find useful.
Configuration
The config table is where you can specify certain options which change the way
ngen behaves. One useful feature is generating a compile_commands.json file for
the clangd LSP.
To enable the generation of compile_commands.json, simply add the following
line to your ngen.toml:
[config]
compile_commands = true
The next time you run Ninja, ngen will automatically generate
build/compile_commands.json where it can be picked up by clangd. Ninja will
make sure that this file is kept up to date as well. Just set the option in your
ngen.toml and forget about it.
By default, the compile_commands.json will be generated according to the build
specs of the main target. To change which target it is generated for, use the
config.compile_commands_target key. For example,
[config]
compile_commands = true
compile_commands_target = "debug"
will generate the the compile_commands for the debug target.
Our final ngen.toml looks like this:
[config]
compile_commands = true
compile_commands_target = "debug"
[targets.main]
outfile = "example"
compiler = "gcc"
compiler_flags = ["-Wall", "-Wextra"]
linker_libs = ["-lm"]
sources = [
"src/main.c",
"src/util.c",
"src/functions.c",
"src/foobar.c",
]
[targets.debug]
opts.default = true
outfile = "example_dbg"
compiler_flags = ["-g"]
[targets.release]
compiler_flags = ["-O2"]