A compiler course, written while the compiler is written

Writing a Compiler in C

I am writing a programming language called izvor in C, and this is the course I wish I had when I started. It goes from an empty file to a compiler that turns source code into a real binary. No parser generator, no LLVM, no libraries. I show you every line and tell you what it does, then give you a command to run so you know it worked.

One rule I hold myself to

A chapter only goes up once the code behind it is in the repo, builds with warnings treated as errors, and passes its tests on Linux and macOS. None of it is made up for the page. When a chapter says a command prints something, that is what it actually printed on my machine.

Before you start

What you need
  • A C compiler. The course uses clang. On a Mac you already have it. On Linux, install clang and make.
  • A terminal, and make. Nothing else. No package manager, no dependencies, no build tool to learn.
  • No compiler knowledge. Chapter one starts at what a token is.
  • A little C. You should have seen a struct, a pointer and a switch before. You do not have to be good at them. I explain the parts that matter when they show up.
Getting the code

Typing it yourself is the whole point, but you can also clone the finished repo and read along:

$ git clone https://github.com/levimackay/izvor $ cd izvor $ make test

Every chapter tells you which file to open and where in it the code goes, so either way works.

The chapters

This is the order I actually built it in. The chapters marked planned are not written yet, because I have not written that code yet either. I am not going to run ahead of the compiler and guess.

  1. 01

    Characters into tokens

    The lexer. What a token is, why it borrows the source instead of copying it, and how to scan numbers, operators and garbage without ever crashing.

    Read →
  2. 02

    Tokens into a tree

    A tagged union for the syntax tree, then recursive descent: one function per precedence level, so that 1 + 2 * 3 comes out as seven and not nine.

    Read →
  3. 03

    Errors a person can read

    A compiler is mostly a tool for telling people what is wrong. Turning a byte offset into a line, a column and a caret, then pinning every message with golden tests and throwing random input at the whole thing.

    Read →
  4. 04

    Names and keywords

    Teaching the lexer to read let x = 10. Maximal munch, why letter breaks the obvious approach, and why comparing a keyword needs its length checked first.

    Read →
  5. 05

    Statements and variables

    Where a program stops being one expression and becomes a sequence of things that happen. Statement nodes, an environment, and the difference between a name at parse time and a name at run time.

    Planned
  6. 06

    Types, and inferring them

    Every expression gets a type before anything runs. let age = 22 works out to an integer on its own; saying so explicitly has to agree.

    Planned
  7. 07

    Functions and control flow

    Calls, parameters, returns, if and loops, and scope becoming a stack of environments rather than one.

    Planned
  8. 08

    Semantic analysis

    The pass that catches undefined names, assignment to a constant, and wrong argument counts. Plus error recovery, so one broken file reports every problem in it instead of only the first.

    Planned
  9. 09

    Generating C

    Walking the typed tree and emitting C source. Why a readable C backend is a better first target than machine code or LLVM.

    Planned
  10. 10

    A native executable

    Driving clang from inside the compiler, handling its failures as your own errors, and finally producing a binary that runs.

    Planned
  11. 11

    Arrays, structs and a runtime

    Aggregates, and the smallest runtime that can support them.

    Planned
  12. 12

    Concurrency you cannot get wrong

    parallel blocks where the programmer never touches a thread, and shared mutable state inside one is a compile error rather than a silent race.

    Planned

What you end up with

A compiler you could sit down and explain to somebody. That is the goal. It is also why I give you the lines and then walk through what each one does, instead of handing over a finished repo for you to stare at. I kept the chapters short. Each one is about one sitting.

How big this actually gets

izvor is a small language and nobody should ship production code with it. It does work end to end though. Source text goes in one side, a native binary comes out the other, and you get real error messages in between. I would rather finish something small than abandon something huge.