ngen/README.md

# ngen
Build file generator (engine) for the [Ninja build
system](https://ninja-build.org/).

Licensed under the GPLv3.

## Methodology and Overview

The problem with existing meta build systems---or makefile generators---like
[Meson](https://mesonbuild.com/) and [CMake](https://cmake.org/) is that they
are needlessly complex for small-to-medium-sized C/C++ projects. These projects
are just large enough that hand-writing a makefile would be painful, but trying
to configure a large meta build system would be just as painful. It is
unacceptable that building a simple project with 10 .c files and a library
requires knowledge of a unique, obscure configuration language which is
constantly changing.

*ngen* aims to make generating build files for these small C/C++ projects as
simple as possible by using a basic key-value configuration file using patterns
that any experienced programmer should be familiar with. In doing so, ngen fills
the gap between writing your own makefile and wrangling with CMakeLists.txt. 

ngen generates files for the small and modern [Ninja](https://ninja-build.org/)
build system. "Where other build systems are high-level languages Ninja aims to
be an assembler," according to Ninja's website. It can be thought of as a
simpler, faster replacement for the classic `make`. It is used by default by
Meson; CMake can also be configured to use ninja as a backend. Ninja is used by
used by Google to build Chromium, v8, etc., and it is also used to build LLVM.
(All this to say, Ninja is commonly used and it is likely installed on your
system already.)

## Building

Build: `cargo build --release`

Install: `sudo sh install.sh`

Uninstall: `sudo sh uninstall.sh`

## Usage
What follows is a tutorial of how to set up ngen for an existing executable
project.

### Basics

The first thing you will need is an `ngen.toml` file. This is what will specify
all of the build parameters, such as compilation flags, source files to track,
etc. Start by creating this file and opening it in your editor.

The simplest valid `ngen.toml` just lists the source files you want to track.
Lets say you have a executable project that looks like this:

```txt
example
├── ngen.toml
└── src
    ├── foobar.c
    ├── functions.c
    ├── include
    │   ├── foobar.h
    │   ├── functions.h
    │   ├── main.h
    │   └── util.h
    ├── main.c
    └── util.c
```

In your `ngen.toml`, write the following:

```toml
[targets.main]
sources = [
  "src/main.c",
  "src/util.c",
  "src/functions.c",
  "src/foobar.c",
]
```

**The `sources` key is a *list* of *strings*, each specifying a single source
file name.**

Now run `ngen`. This will generate a `build.ninja` file in the current working
directory. You won't ever have to touch this file; that's what ngen is for. You
also won't ever have to run `ngen` yourself again (unless your `build.ninja`
gets deleted); Ninja will take care of regenerating the build file if
`ngen.toml` changes.

With your `build.ninja` generated, run `ninja` on the command line. That's it!
Your project is now built, you will find the executable at `build/main/a.out`.
Remember, you can also freely add and remove files from the above list without
running `ngen` again: Ninja will regenerate the `build.ninja` for you.

Now, while this is functional, it isn't very useful. It is very likely that you
will want to specify a compiler (gcc/clang), pass some flags, link some
libraries into your final executable, and definitely name your program something
other than "a.out." ngen makes these things dead simple, too.

**The `outfile` key is a *string* that specifies the name of the file produced
by the `linker` (see below).**

Lets set this to "example."

```toml
outfile = "example"
```

**The `compiler` key is a *string* that specifies the program that will be used
to turn .c files into .o files.**

By default, if not specified, `compiler` is set to "cc," which on Linux systems
should be a C compiler. It should be noted that the compiler you choose must
support the `-MD` and `-MF` flags to generate dependency files (both gcc and
clang support this). Lets say we want to use "gcc." Add the following line to
your `ngen.toml`:

```toml
compiler = "gcc"
```

**The `compiler_flags` key is a *list* of *strings* that contains the arguments
to be passed to the `compiler` during the compilation of each `source` file.**

It is not necessary to add the `-c` or `-o outfile` flags, ngen will take care
of this for you. For example, add the following to your `ngen.toml`:

```toml
compiler_flags = ["-Wall", "-Wextra -O2"]
```

**The `linker` key is a *string* that specifies the program that will be used to
combine the .o files into the final `outfile`.**

If the `linker` key is not found, it will be set to the value of `compiler`. For
this example, we don't have to change anything here.

**The `linker_flags` key is a *list* of *strings* that contains the arguments to
be passed to the `linker` during the linking of the `outfile`.**

Library flags (`-lm`, `-lyourlib`) should NOT be included here. This is for
linker options, not libraries. The syntax is the same as `compiler_flags`. There
is nothing we have to set here.

**The `linker_libs` key is a *list* of *strings* that contains the link library
arguments to be linked to the `outfile`.**

THIS is where library flags (`-lm`, `-lyourlib`) go. Lets say our example
project needs the math library:

```toml
linker_libs = ["-lm"]
```

Now, our `ngen.toml` looks like this:

```toml
[targets.main]
outfile = "example"
compiler = "gcc"
compiler_flags = ["-Wall", "-Wextra -O2"]
linker_libs = ["-lm"]
sources = [
  "src/main.c",
  "src/util.c",
  "src/functions.c",
  "src/foobar.c",
]
```

This is a much more realistic looking project. Once again, any changes to any of
these values will be automatically picked up by Ninja and accounted for in the
build. Running `ninja -v` immediately after saving `ngen.toml` should show that
the options you set were recognized, and your files were rebuilt accordingly.

Now, this is a good start. But, it often the case that, in a project, you want
to have multiple different build *targets,* or configurations, that build the
project with slightly different parameters; for example, it is common to have a
"debug" build target that is unoptimized and includes debugging symbols, and a
"release" build that is optimized at compile time. ngen is designed to make this
configuration as easy as possible.

### Targets

When we were specifying parameters above, we were doing so in the main target
`[targets.main]`. A "target" is a self-contained build process that builds its
`outfile` from the `sources` and other paramenters provided. We could have named
this target whatever we wanted, but the target named `main` is special, as we shall
presently see. Lets create a new target called `debug`. Add the following to your `ngen.toml`:

```toml
[targets.debug]
outfile = "example_dbg"
compiler_flags = ["-g"]
```

What's going on here? How does `debug` know what files to operate on, what
compiler to use, etc? Well, the `main` target is special: all targets *inherit*
the parameters set in the main target. Inheritance works according to two simple
rules: **a**rrays **a**ppend, **s**trings **s**upercede.

The first thing that happens is `debug` takes on all the same parameters from
main. Then, ngen reads the outfile key in `debug`. Becuase outfile is a string,
`debug.outfile` is overwritten as the value specified, in this case
"example\_dbg". On the other hand, since compiler\_flags is a list, the elements
sepecified in `debug.compiler_flags` are appended to the list of flags specified
in `main`. So in this case, the effective value of `debug.compiler_flags` is
`["-Wall", "-Wextra -O2", "-g"]`. 

For our debug build, we probably don't want the `-O2` flag---optimizations
should only happen in a "release" type build. So, lets remove the `-O2` flag
from `main` and place it in a new target called `release`. Our `ngen.toml`
should now look like this:

```toml
[targets.main]
outfile = "example"
compiler = "gcc"
compiler_flags = ["-Wall", "-Wextra"]
linker_libs = ["-lm"]
sources = [
  "src/main.c",
  "src/util.c",
  "src/functions.c",
  "src/foobar.c",
]

[targets.debug]
outfile = "example_dbg"
compiler_flags = ["-g"]

[targets.release]
compiler_flags = ["-O2"]
```

This brings up an important design pattern you should keep in mind when writing
your `ngen.toml`: the special `main` target should only contain the largest
subset of all your build parameters. Additonal targets should add specific
parameters for specific use cases, as we saw in the above example.

In a nutshell, inheritance allows you to easily create multiple targets with
small variations, without having to rewrite the same thing over and over again.
You can always disable inheritance using the `opts.inherit = false` key on
targets that you do not want to inherit from `main`. You can also change the
parent target that a target inherits from using the `opts.inherit_from =
"target"` key, replacing `target` with the name of the desired parent target.

Save `ngen.toml`, and try running `ninja -v debug` or `ninja -v release`. You
should see that each of these targets uses the parameters that we specified with
inheritance. Outfiles for a given target are always placed in
`build/<target_name>/`, as you can see with `tree build`. This keeps things
organized, and also means we don't have to specify a different outfile name for
each target (I did above just to show you how strings are replaced in the
inheritance system).

Note that by default, running `ninja` alone with no target specifed will run
*every single target* it finds. You can change this behavor by adding the
`opts.default = true` key to the targets you want to be built when Ninja is invoked
with no arguments. Say that this example project is under active developemnt,
and you will be building the `debug` target alot. You can add the `opts.default =
true` flag to the `[debug]` table, and now running `ninja` by itself will only
build the `debug` target. You can still build the release and main targets by
running `ninja release` and `ninja main`.

It is easy to see how powerful this simple configuration file is already.
However, ngen has a few more features that you may find useful.

### Configuration

The `config` table is where you can specify certain options which change the way
ngen behaves. One useful feature is generating a compile\_commands.json file for
the `clangd` LSP.

To enable the generation of compile\_commands.json, simply add the following
line to your `ngen.toml`:

```toml
[config]
compile_commands = true
```

The next time you run Ninja, ngen will automatically generate
`build/compile_commands.json` where it can be picked up by clangd. Ninja will
make sure that this file is kept up to date as well. Just set the option in your
ngen.toml and forget about it.

By default, the compile\_commands.json will be generated according to the build
specs of the `main` target. To change which target it is generated for, use the
`config.compile_commands_target` key. For example,

```toml
[config]
compile_commands = true
compile_commands_target = "debug"
```

will generate the the compile\_commands for the `debug` target.

Our final `ngen.toml` looks like this:
```toml
[config]
compile_commands = true
compile_commands_target = "debug"

[targets.main]
outfile = "example"
compiler = "gcc"
compiler_flags = ["-Wall", "-Wextra"]
linker_libs = ["-lm"]
sources = [
  "src/main.c",
  "src/util.c",
  "src/functions.c",
  "src/foobar.c",
]

[targets.debug]
opts.default = true
outfile = "example_dbg"
compiler_flags = ["-g"]

[targets.release]
compiler_flags = ["-O2"]
```

## Advanced Usage

Lets say that your project also has a library in the `lib` folder, with two
files, `lib.c` and `functions.c`. You want to compile these into a
`libexample.a` file so that you can link them to your main executable. Add a new
target called `lib`:

```toml
[targets.lib]
opts.inherit = false
outfile = "libexample.a"
compiler = "gcc"
linker = "ar"
linker_flags = "r"
sources = [
  "lib/lib.c",
  "lib/functions.c",
]
```

If you try to run this as is, you will see that there is a problem with the link
step. ngen constructs commands according to a certain format to make
configuration easier. The format for the link step is `{linker} {linker_flags}
-o {outfile} {objects} {linker_libs}`. This means that currently, the above
configuration is trying to run `ar r -o libexample.a <object files>`. `ar` does
not take an `-o` flag. We can change this format string to properly produce the
library. Add the following entry to the `lib` target above:

```toml
opts.link_cmd_fmt = "{linker} {linker_flags} {outfile} {objects} {linker_libs}"
```

By removing the `-o` flag, the library should "link" correctly now. But, we
could simplify this configuration slightly by removing some of those unnecessary
format placeholders. Here is an alternate form that should work exactly the same:

```toml
[targets.lib]
opts.inherit = false
opts.link_cmd_fmt = "ar r {outfile} {objects}"
outfile = "libexample.a"
compiler = "gcc"
sources = [
  "lib/lib.c",
  "lib/functions.c",
]
```

An option `opts.compile_cmd_fmt` also exists. Its default is `{compiler}
{compiler_flags} -MD -MF {depfile} -o {object} -c {source}`. Note that if you
remove the `-MD -MF {depfile}` component, ninja will have no way to discover
what header files your source file depends on.