No code in this step. Before I change Env, I want you to see exactly why it
has to change, with a program that looks completely reasonable.
Chapter five gave every program one Env: a flat array of up to 256
Bindings, searched from the front. That was the right call at the time. It
said plainly that a hash table wasn't worth it yet, and for a program with no functions,
there is exactly one set of names in play, so one table is enough. Now watch it break.
fn double(n: int) -> int { let result = n * 2; return result;} let result = 100;print double(5);print result;
Nothing here is a typo. result on line 2 and result on line 6
were never meant to be the same variable. One holds a doubled number inside a function,
the other holds a total at the top of the program. A person reading this sees two
unrelated uses of a common word, in two different places, and expects them to stay out of
each other's way.
A single flat Env cannot tell those apart. There is one array, and both
let statements try to write into it. If env_define refuses a
duplicate name, calling double fails with result already defined,
which is true of the table and false of the program: the reader never redefined anything,
the language just ran out of room to keep two results straight. If
env_define instead overwrote the old entry, line 8 would print 10
instead of 100, silently, because calling a completely unrelated function
reached out and clobbered a variable it was never given.
Either failure is the same root cause: one namespace for a program that now has more than
one place variables can live. A function's parameters and locals need their own space,
separate from whatever called it, and a block inside an if needs its own
space too, separate from the function around it. That is what a scope stack buys you, and
it is also why it waited until this chapter. Chapter five had no functions and no blocks,
so it had exactly one scope, always. There was nothing to stack.
Checkpoint
No build to run. Just be able to say, out loud, which result chapter
five's Env would actually store.
One field, one struct that needs a name it never had, one changed signature, and one function that grows a loop. A second new function joins the file too, for a reason chapter five's design never had to face.
src/env.h & src/env.c · here is what chapter five actually left you, quoted exactly
#include <stdbool.h> #define ENV_MAX 256 typedef struct { const char *name; int length; long value; bool mutable;} Binding; typedef struct { Binding items[ENV_MAX]; int count;} Env; void env_init(Env *env);Binding *env_find(Env *env, const char *name, int length);bool env_define(Env *env, const char *name, int length, long value, bool mutable);
Note the name: it is env_find, not env_lookup, and there is no
fourth function for assignment. Chapter five's exec_statement handles
x = 5; by calling env_find for the binding and checking
->mutable directly, inline, rather than through a separate helper. That logic
moves into eval unchanged in step 12. There is nothing named
env_assign to update, because it was never written.
typedef struct { ... } Env; declares an anonymous struct and gives the
typedef name to it afterward. That works fine right up until a field inside the struct
needs to refer to the struct's own type, which is exactly what parent is
about to do. An anonymous struct has no name to write inside its own braces. The fix is
the one-word change below: give the struct a tag, struct Env, so
struct Env *parent; has something to point at before the typedef finishes.
src/env.h · the typedef gets a tag, and the struct gets a new field directly below count
typedef struct Env {Was typedef struct {. The tag, Env, is what parent below needs to exist.
Binding items[ENV_MAX]; int count; struct Env *parent;A scope either has an outer scope or it doesn't. The global scope's parent is NULL, and that's how you'll know you've climbed as far as you can go.
} Env;src/env.h · the declarations at the bottom
void env_init(Env *env, Env *parent);The one signature that changes. Every call site now has to say what this scope sits inside of.
Binding *env_find(Env *env, const char *name, int length);Binding *env_find_local(Env *env, const char *name, int length);New. Searches only env itself, never env->parent. Keep reading before step 12, where the difference between this and env_find is the difference between a real bug and a fixed one.
bool env_define(Env *env, const char *name, int length, long value, bool mutable);src/env.c · env_init, then env_find, quoted exactly as chapter five wrote it
void env_init(Env *env) { env->count = 0;} Binding *env_find(Env *env, const char *name, int length) { for (int i = env->count - 1; i >= 0; i--) { if (env->items[i].length == length && memcmp(env->items[i].name, name, (size_t)length) == 0) { return &env->items[i]; } } return NULL;}src/env.c · the edit
void env_init(Env *env, Env *parent) { env->count = 0; env->parent = parent;} Binding *env_find_local(Env *env, const char *name, int length) {This is chapter five's env_find, renamed, with not one byte of its body touched. It only ever looks at env's own array.
for (int i = env->count - 1; i >= 0; i--) { if (env->items[i].length == length && memcmp(env->items[i].name, name, (size_t)length) == 0) { return &env->items[i]; } } return NULL;} Binding *env_find(Env *env, const char *name, int length) {The name chapter five used now means "search this scope and every scope outside it," which is a genuinely different function even though most of its body is a one-line call to the one above.
for (Env *scope = env; scope != NULL; scope = scope->parent) { Binding *b = env_find_local(scope, name, length); if (b != NULL) return b; } return NULL;}
env_define does not change at all. It only ever writes into
env->items directly, never env->parent, which is exactly
the behavior you want: a new let or var always lands in the
innermost scope.
Chapter five's exec_statement refuses a second let with the
same name by calling env_find first and checking it comes back
NULL. If that check still calls the new, climbing env_find in
step 12, declaring a local result next to an outer one fails with
'result' is already defined, which is precisely the bug step one exists to
fix, now reintroduced by the very function meant to fix it. The redefinition check has
to ask "is this name already in this scope," which is what
env_find_local is for. General lookups, a name used in an expression or an
assignment target, genuinely do want to climb, since reading or reassigning an outer
variable from inside a block is supposed to work. Two questions, and now, correctly,
two functions.
You are about to see, in step 12, code that does Env local; env_init(&local,
outer); as a plain local variable, no malloc anywhere. That's
deliberate. A block's scope only needs to exist for as long as that block is running,
which is exactly how long a C local variable lives. When the block's { }
finishes, the C function returns or the loop iterates, and the scope's storage is
reclaimed automatically. Chaining Envs with a parent pointer turns
"nested scope" into "nested C stack frame," and you don't pay for a single
free anywhere in this chapter.
Checkpoint
Every existing call to env_init(&env) is now wrong, and the compiler
names every one of them. Fix each by passing the enclosing scope, or NULL
for the one that has no enclosing scope: the global environment in main.
Five keywords, four bracket-and-comma characters, and six comparisons. One of the
comparisons is ==, which chapter four put off on purpose. This is where it
gets paid.
src/lexer.h · inside the typedef enum, directly below TOK_VAR, above TOK_EOF
TOK_VAR, TOK_LBRACE,{. Opens a block.
TOK_RBRACE,}. Closes one.
TOK_COMMA,Separates parameters and call arguments.
TOK_ARROW,->. Introduces a function's return type.
TOK_EQEQ,==. Compares for equality. Not to be confused with the single = you already have, which assigns.
TOK_BANGEQ,!=. The language has no other use for a bare ! yet, so this is the only thing that character can mean right now.
TOK_LESS, TOK_LESSEQ, TOK_GREATER, TOK_GREATEREQ, TOK_IF, TOK_ELSE, TOK_WHILE, TOK_FN, TOK_RETURN, TOK_EOF,src/lexer.c · inside token_type_name, before the TOK_ERROR case
case TOK_LBRACE: return "LBRACE"; case TOK_RBRACE: return "RBRACE"; case TOK_COMMA: return "COMMA"; case TOK_ARROW: return "ARROW"; case TOK_EQEQ: return "EQEQ"; case TOK_BANGEQ: return "BANGEQ"; case TOK_LESS: return "LESS"; case TOK_LESSEQ: return "LESSEQ"; case TOK_GREATER: return "GREATER"; case TOK_GREATEREQ: return "GREATEREQ"; case TOK_IF: return "IF"; case TOK_ELSE: return "ELSE"; case TOK_WHILE: return "WHILE"; case TOK_FN: return "FN"; case TOK_RETURN: return "RETURN";src/lexer.c · inside the keyword chain in lexer_next, directly below the existing print check, above return make_token(lx, TOK_IDENT, start);
if (word_is(lx, start, "if")) return make_token(lx, TOK_IF, start); if (word_is(lx, start, "else")) return make_token(lx, TOK_ELSE, start); if (word_is(lx, start, "while")) return make_token(lx, TOK_WHILE, start); if (word_is(lx, start, "fn")) return make_token(lx, TOK_FN, start); if (word_is(lx, start, "return")) return make_token(lx, TOK_RETURN, start);Same function, same trap as chapter four: word_is checks length before it checks bytes, so returned stays an ordinary name instead of matching here and leaving ed as a stray token.
src/lexer.c · directly above lexer_next, next to word_is
static Token two_char(Lexer *lx, int start, char second, TokenType two, TokenType one) {Four of your six new operators share one shape: read one character, then peek at the next one to decide whether it changes the meaning. Rather than write that peek four times, this function takes both possible token types and picks.
if (peek(lx) == second) { advance(lx); return make_token(lx, two, start); } return make_token(lx, one, start);}src/lexer.c · in the switch (c) at the bottom of lexer_next
case '-': return two_char(lx, start, '>', TOK_ARROW, TOK_MINUS);This line already existed as return make_token(lx, TOK_MINUS, start);. Now - alone is still minus, but -> is the arrow.
case '=': return two_char(lx, start, '=', TOK_EQEQ, TOK_EQUAL);This is the peek chapter four deferred. It said plainly, back in the names chapter, that = would need to look ahead for a second = once comparisons existed. Here it is: one character is assignment, two is equality.
case '<': return two_char(lx, start, '=', TOK_LESSEQ, TOK_LESS); case '>': return two_char(lx, start, '=', TOK_GREATEREQ, TOK_GREATER); case '{': return make_token(lx, TOK_LBRACE, start); case '}': return make_token(lx, TOK_RBRACE, start); case ',': return make_token(lx, TOK_COMMA, start); case '!':! has no one-character meaning yet, so it doesn't get a one-line case. If the next character isn't =, this is simply not a valid token today.
if (peek(lx) == '=') { advance(lx); return make_token(lx, TOK_BANGEQ, start); } return make_token(lx, TOK_ERROR, start);
It's tempting to think maximal munch was a fact about letters. It isn't. It's a fact
about scanning: whenever one valid token's spelling is a prefix of another's, you have
to look past the short one before you commit to it. = is a prefix of
==, and - is a prefix of ->, for the same
reason let is a prefix of letter. two_char is
maximal munch written once and reused four times instead of copied.
Checkpoint
Fifteen new token types, and nothing downstream of the lexer has seen any of them yet. That starts next step.
One new grammar rule, sitting above everything chapters one through six built, and one real edit to a function you already have.
comparison -> expression (("==" | "!=" | "<" | "<=" | ">" | ">=") expression)?At most one comparison. a < b < c is not legal izvor. Chaining comparisons reads as "is a less than b, and is that less than c" in math, but as "compare a and b, then compare the 1-or-0 result to c" in most languages that allow it, which is a trap, not a feature. Leaving it out means there is nothing to get confused about.
expression -> term (("+" | "-") term)*term -> factor (("*" | "/") factor)*factor -> NUMBER | BOOL | IDENT | call | "(" comparison ")" | "-" factor
Comparisons bind the loosest of anything you have. a + b == c * d has to
mean "compare the sum to the product," not something stranger, so comparison
sits above expression, calling it the way expression calls
term. Grouping with parentheses has to reach comparisons too, or
(a == b) would be a syntax error, so the parenthesized branch inside
factor needs to call the new top of the chain, not the old one.
src/parser.c · near the top, directly below the existing forward declaration of parse_expression
static Node *parse_expression(Parser *p);static Node *parse_comparison(Parser *p);Needed for the same reason the expression forward declaration already exists: parse_factor, defined earlier in the file, is about to call this before its real definition appears.
static Node *parse_statement(Parser *p);Also needed now. parse_block, coming in the next step, has to call the statement parser before it's defined too.
src/parser.c · directly below parse_expression's definition
/* comparison -> expression (COMPOP expression)? */static Node *parse_comparison(Parser *p) { Node *left = parse_expression(p); if (left == NULL) return NULL; TokenType op = p->current.type; if (op == TOK_EQEQ || op == TOK_BANGEQ || op == TOK_LESS || op == TOK_LESSEQ || op == TOK_GREATER || op == TOK_GREATEREQ) { long offset = p->current.start - p->lexer.src; parser_advance(p); Node *right = parse_expression(p); if (right == NULL) { ast_free(left); return NULL; } return at(ast_binary(op, left, right), offset);Chapter five's ast_binary takes no offset. Every construction goes through at(), chapter five's own helper, which stamps node->offset and passes the node straight through, tolerating NULL. This is the same wrapper the chapter one ast_binary calls in parse_term and parse_expression already use.
} return left;}
No new node kind here. A comparison is a NODE_BINARY exactly like
+ or *, just with one of the six new token types as its
op. The evaluator decides what each op means; the parser's only job is
building the tree with the right shape.
src/parser.c · inside parse_factor, the parenthesized-grouping branch, unchanged since chapter one until now
if (parser_match(p, TOK_LPAREN)) { Node *inner = parse_comparison(p);Was parse_expression(p). This is the line that makes (x == 5) parse at all: without it, the grouping parentheses would only ever accept the tighter grammar, and a comparison inside parens would be a syntax error.
if (inner == NULL) return NULL;
Three more places call parse_expression for a statement's own expression:
parse_let and parse_var's initializer, and parse_print's
argument. Change each of those three calls to parse_comparison too. Nothing
else about those functions moves.
parse_let: Node *value = parse_comparison(p);parse_var: Node *value = parse_comparison(p);parse_print: Node *value = parse_comparison(p);Checkpoint
print 3 + 4 == 7; now parses. It won't evaluate correctly yet: the evaluator doesn't know what TOK_EQEQ means as an operator until step 12.
A block is a brace, a list of statements you already know how to parse one at a time,
and a closing brace. The interesting part isn't the syntax, it's that NodeList
does the work again without changing shape.
src/ast.h · inside the NodeType enum, directly below NODE_EXPR_STMT
NODE_EXPR_STMT, NODE_BLOCK,src/ast.h · in the union inside struct Node, directly below the expr_stmt member
struct { NodeList stmts; } block;
That's the whole payload: a block is nothing but its statements. Here is NodeList
again, quoted exactly as chapter five defined it, because this chapter leans on it hard:
typedef struct { Node **items; int count; int capacity;} NodeList;src/ast.c · directly above ast_free
Node *ast_block(NodeList stmts) {No offset parameter. Chapter five's constructors never take one, ast_number(long value), ast_ident(const char *name, int length), and so on. Stamping the position is at()'s job, called from the parser, not the constructor's.
Node *node = new_node(NODE_BLOCK); if (node == NULL) return NULL; node->as.block.stmts = stmts; return node;}src/ast.c · in the switch inside ast_free, directly below the NODE_EXPR_STMT case
case NODE_BLOCK: nodelist_free(&node->as.block.stmts);nodelist_free already frees the nodes it holds, per chapter five. One call frees every statement in the block, recursively, all the way down.
break;src/parser.c · directly above parse_statement
/* block -> "{" statement* "}" */static Node *parse_block(Parser *p) { long offset = p->current.start - p->lexer.src; if (!parser_match(p, TOK_LBRACE)) { error_at(p, "expected '{'"); return NULL; } NodeList stmts; nodelist_init(&stmts); while (!parser_check(p, TOK_RBRACE) && !parser_check(p, TOK_EOF)) {The TOK_EOF check matters. Without it, a file that never closes its brace loops forever, since parse_statement would keep being asked to parse a statement starting at end-of-file.
Node *stmt = parse_statement(p); if (stmt == NULL) { nodelist_free(&stmts); return NULL; } if (!nodelist_push(&stmts, stmt)) { ast_free(stmt); nodelist_free(&stmts); return NULL; } } if (!parser_match(p, TOK_RBRACE)) { error_at(p, "expected '}'"); nodelist_free(&stmts); return NULL; } return at(ast_block(stmts), offset);}src/parser.c · inside parse_statement, before the final fallback to parse_expr_stmt
if (parser_check(p, TOK_PRINT)) return parse_print(p); if (parser_check(p, TOK_LBRACE)) return parse_block(p); return parse_expr_stmt(p);
A bare { x = 1; } is now legal on its own, with no if or
while attached, and that's fine. It's a scope with nothing conditional about
it, useful on its own for limiting how long a variable lives.
main.c's print_tree switches over NodeType with
no default, same as everything else. It never learned to print statements, because by
chapter five your programs print their own results with print, which made
S-expression dumping of a whole program pointless and it stopped being called. If you
kept it around for expression debugging, it needs a case for each of the six kinds this
chapter adds, same shape as the ones it already has. Nothing past this note depends on
it either way.
Checkpoint
That error is eval's, in main.c. Expected: eval
doesn't get its NODE_BLOCK case until step 12, because it changes shape for all six new
node kinds at once, together with the scope-stack wiring. Everything you wrote this
step is otherwise correct and stays exactly as it is.
The classic dangling-else problem doesn't get resolved here, it gets made impossible, because of a decision one step back gave you for free.
In C, if (a) if (b) x(); else y(); is ambiguous on paper: does
else belong to the inner if or the outer one? C's real answer
is "the nearest unclaimed if," a rule you have to know, because C lets an
if's body be a single bare statement with no braces required.
izvor never has that problem, because step 5 made a block's braces mandatory for the
body of anything that has one. There is no bare-statement body to attach an
else to ambiguously: every if and every else is
followed by a {, full stop, and that brace marks exactly where it ends.
The one exception is else if, which isn't really an exception: it's
else followed by another whole if-statement, which happens to
end in its own mandatory block. The rule I picked is: no bare statement is ever a
branch body. It costs you two extra characters, {}, around a
one-line branch. It buys you a parser with nothing to disambiguate.
if_stmt -> "if" comparison block ("else" (block | if_stmt))?Unlike C, the condition needs no parentheses. if n < 2 { ... } is unambiguous, because the opening { is the only thing that can end a condition, so nothing is lost by not requiring ( ) around it. Write them anyway if you want; (n < 2) is just an ordinary grouped expression.
src/ast.h · enum, directly below NODE_BLOCK
NODE_IF,src/ast.h · union, directly below the block member
struct { Node *cond; Node *then_branch; Node *else_branch;NULL when there's no else at all.
} if_stmt;src/ast.c · directly above ast_free
Node *ast_if(Node *cond, Node *then_branch, Node *else_branch) { Node *node = new_node(NODE_IF); if (node == NULL) return NULL; node->as.if_stmt.cond = cond; node->as.if_stmt.then_branch = then_branch; node->as.if_stmt.else_branch = else_branch; return node;}src/ast.c · ast_free, directly below the NODE_BLOCK case
case NODE_IF: ast_free(node->as.if_stmt.cond); ast_free(node->as.if_stmt.then_branch); ast_free(node->as.if_stmt.else_branch);ast_free(NULL) is a no-op, its very first line. Calling it on a possibly-absent else_branch needs no extra check here.
break;src/parser.c · directly above parse_statement, below parse_block
/* if_stmt -> "if" comparison block ("else" (block | if_stmt))? */static Node *parse_if(Parser *p) { long offset = p->current.start - p->lexer.src; parser_advance(p); /* 'if' */ Node *cond = parse_comparison(p); if (cond == NULL) return NULL; Node *then_branch = parse_block(p); if (then_branch == NULL) { ast_free(cond); return NULL; } Node *else_branch = NULL; if (parser_match(p, TOK_ELSE)) { else_branch = parser_check(p, TOK_IF) ? parse_if(p) : parse_block(p);This one line is the entire "else if chains" feature. If another if follows else, parse a whole nested if-statement recursively; otherwise it must be a plain block.
if (else_branch == NULL) { ast_free(cond); ast_free(then_branch); return NULL; } } return at(ast_if(cond, then_branch, else_branch), offset);}src/parser.c · inside parse_statement, alongside the block check
if (parser_check(p, TOK_LBRACE)) return parse_block(p); if (parser_check(p, TOK_IF)) return parse_if(p);Checkpoint
Same expected error, from eval, one enumerator later. Still fine.
Everything in this step is a smaller version of the last one. while has a
condition and one body, no branching to think about.
src/ast.h · enum, directly below NODE_IF
NODE_WHILE,src/ast.h · union, directly below the if_stmt member
struct { Node *cond; Node *body; } while_stmt;src/ast.c · directly above ast_free
Node *ast_while(Node *cond, Node *body) { Node *node = new_node(NODE_WHILE); if (node == NULL) return NULL; node->as.while_stmt.cond = cond; node->as.while_stmt.body = body; return node;}src/ast.c · ast_free, directly below the NODE_IF case
case NODE_WHILE: ast_free(node->as.while_stmt.cond); ast_free(node->as.while_stmt.body); break;src/parser.c · directly above parse_statement, below parse_if
/* while_stmt -> "while" comparison block */static Node *parse_while(Parser *p) { long offset = p->current.start - p->lexer.src; parser_advance(p); /* 'while' */ Node *cond = parse_comparison(p); if (cond == NULL) return NULL; Node *body = parse_block(p); if (body == NULL) { ast_free(cond); return NULL; } return at(ast_while(cond, body), offset);}src/parser.c · inside parse_statement
if (parser_check(p, TOK_IF)) return parse_if(p); if (parser_check(p, TOK_WHILE)) return parse_while(p);Checkpoint
Parameters need a name and a type each. The tempting move is a second array just for
the types. Resist it: NodeList already holds a list of nodes, and a
parameter's name is already a node, an identifier. Give that identifier one more field.
fn_decl -> "fn" IDENT "(" (param ("," param)*)? ")" ("->" type)? blockparam -> IDENT ":" typeChapter six's let can infer its type from the value on the right of =. A parameter has no value yet, only a name, so there is nothing to infer from until somebody calls the function, possibly several call sites, possibly disagreeing. The annotation is mandatory here for that reason, not out of extra strictness for its own sake.
type -> IDENTRead by chapter six's parse_type_name(Parser *p), which you already have. It consumes the type name and hands back a TypeKind, reporting an unrecognized name itself. Reuse it here rather than duplicating it.
Chapter five defines ident as exactly two fields, and this is the moment
that stops being enough:
struct { const char *name; int length; } ident;src/ast.h · the ident member
struct { const char *name; int length; TypeKind param_type; } ident;Meaningful only when this identifier is standing in a parameter list, the same way chapter six's has_annotation and annotation are meaningful only when a binding actually has one. An ordinary reference to a variable, NODE_IDENT used as an expression, never reads this field. One tagged reuse of a small struct, not a second kind of node.
src/ast.h · enum, directly below NODE_WHILE
NODE_FN,src/ast.h · union, directly below the while_stmt member
struct { const char *name; int length; NodeList params;A list of NODE_IDENT, one per parameter, each carrying its own param_type. Reused, not a second list type.
TypeKind ret_type;Defaults to TYPE_VOID when there's no ->. A function that returns nothing still has a return type, it's just the one that means nothing.
Node *body; } fn;src/ast.c · directly above ast_free
Node *ast_fn(const char *name, int length, NodeList params, TypeKind ret_type, Node *body) { Node *node = new_node(NODE_FN); if (node == NULL) return NULL; node->as.fn.name = name; node->as.fn.length = length; node->as.fn.params = params; node->as.fn.ret_type = ret_type; node->as.fn.body = body; return node;}src/ast.c · ast_free, directly below the NODE_WHILE case
case NODE_FN: nodelist_free(&node->as.fn.params);Frees every parameter's identifier node. There's no separate array to free alongside it, because there isn't one.
ast_free(node->as.fn.body); break;src/parser.h · inside struct Parser, directly below Token current;
int fn_depth;Zero at the top level, incremented while parsing inside a function body. Step 10 uses it to catch return outside a function, at parse time rather than run time.
src/parser.c · parser_init, directly below parser_advance(p);
p->fn_depth = 0;src/parser.c · directly above parse_statement, below parse_while
/* fn_decl -> "fn" IDENT "(" params? ")" ("->" type)? block
params -> IDENT ":" type ("," IDENT ":" type)* */static Node *parse_fn(Parser *p) { long offset = p->current.start - p->lexer.src; parser_advance(p); /* 'fn' */ if (!parser_check(p, TOK_IDENT)) { error_at(p, "expected function name"); return NULL; } const char *name = p->current.start; int name_length = p->current.length; parser_advance(p); if (!parser_match(p, TOK_LPAREN)) { error_at(p, "expected '('"); return NULL; } NodeList params; nodelist_init(¶ms); if (!parser_check(p, TOK_RPAREN)) { for (;;) { if (!parser_check(p, TOK_IDENT)) { error_at(p, "expected parameter name"); nodelist_free(¶ms); return NULL; } long param_offset = p->current.start - p->lexer.src; const char *param_name = p->current.start; int param_length = p->current.length; parser_advance(p); if (!parser_match(p, TOK_COLON)) { error_at(p, "expected ':' after parameter name"); nodelist_free(¶ms); return NULL; } Node *param = at(ast_ident(param_name, param_length), param_offset); if (param == NULL) { nodelist_free(¶ms); return NULL; } param->as.ident.param_type = parse_type_name(p);Built with the same ast_ident chapter five already gives you for a plain reference, then the one extra field is set directly, right here, because ast_ident itself has no reason to know about parameter types.
if (!nodelist_push(¶ms, param)) { ast_free(param); nodelist_free(¶ms); return NULL; } if (!parser_match(p, TOK_COMMA)) break; } } if (!parser_match(p, TOK_RPAREN)) { error_at(p, "expected ')'"); nodelist_free(¶ms); return NULL; } TypeKind ret_type = TYPE_VOID; if (parser_match(p, TOK_ARROW)) { ret_type = parse_type_name(p); } p->fn_depth++; Node *body = parse_block(p); p->fn_depth--;Decremented immediately after parsing the body, success or failure, so nesting a second fn textually after this one, not inside it, starts from zero again.
if (body == NULL) { nodelist_free(¶ms); return NULL; } return at(ast_fn(name, name_length, params, ret_type, body), offset);}src/parser.c · inside parse_statement
if (parser_check(p, TOK_WHILE)) return parse_while(p); if (parser_check(p, TOK_FN)) return parse_fn(p);
A function can only be declared at the top level or inside a block, same as any other
statement, since parse_statement is the one dispatcher every block goes
through. Nothing in this chapter stops you from writing a fn inside another
fn's body syntactically. Whether that should mean anything, and what a
function nested that way would even close over, is a question chapter seven leaves
alone. It parses; step 12's evaluator gives it one specific, deliberately narrow answer.
Checkpoint
add(1, 2) * 3 has to multiply the call's result by three, not call
add with some multiplied thing as an argument. That means a call binds as
tightly as a bare number does: it belongs inside factor, not above it.
factor -> NUMBER | BOOL | IDENT | call | "(" comparison ")" | "-" factorcall -> IDENT "(" (comparison ("," comparison)*)? ")"src/ast.h · enum, directly below NODE_FN
NODE_CALL,src/ast.h · union, directly below the fn member
struct { const char *name; int length; NodeList args;Call arguments are just expressions. No per-argument type field needed here, unlike a parameter: an argument doesn't declare anything, it evaluates to something.
} call;src/ast.c · directly above ast_free
Node *ast_call(const char *name, int length, NodeList args) { Node *node = new_node(NODE_CALL); if (node == NULL) return NULL; node->as.call.name = name; node->as.call.length = length; node->as.call.args = args; return node;}src/ast.c · ast_free, directly below the NODE_FN case
case NODE_CALL: nodelist_free(&node->as.call.args); break;src/parser.c · near the top, directly below the parse_statement forward declaration added in step 4
static Node *parse_call(Parser *p, const char *name, int length, long offset);parse_factor is defined near the top of the file and is about to call this before its definition, same reasoning as every other forward declaration here.
src/parser.c · parse_factor, the TOK_IDENT branch chapter five gave you
if (parser_check(p, TOK_IDENT)) { long offset = p->current.start - p->lexer.src; const char *name = p->current.start; int length = p->current.length; parser_advance(p); if (parser_check(p, TOK_LPAREN)) { return parse_call(p, name, length, offset); } return at(ast_ident(name, length), offset); }src/parser.c · directly below parse_factor
/* call -> IDENT (already consumed) "(" (comparison ("," comparison)*)? ")" */static Node *parse_call(Parser *p, const char *name, int length, long offset) { parser_advance(p); /* '(' */ NodeList args; nodelist_init(&args); if (!parser_check(p, TOK_RPAREN)) { for (;;) { Node *arg = parse_comparison(p); if (arg == NULL) { nodelist_free(&args); return NULL; } if (!nodelist_push(&args, arg)) { ast_free(arg); nodelist_free(&args); return NULL; } if (!parser_match(p, TOK_COMMA)) break; } } if (!parser_match(p, TOK_RPAREN)) { error_at(p, "expected ')'"); nodelist_free(&args); return NULL; } return at(ast_call(name, length, args), offset);}
Notice what parse_call takes: a name, a length, and an offset, all already
read by the time it's invoked. That's on purpose. By the time parse_factor
knows it's looking at a call rather than a bare identifier, it has already consumed the
identifier token to find out. Threading those three values through as parameters is
simpler than trying to rewind the parser, which chapter one's lexer was never built to
do.
Checkpoint
Parsing return is the easy half. The hard half, getting its value out of
however many blocks and loops are sitting between it and the call that's waiting, is
step 12's problem. This step only builds the node and rejects the one case that can be
caught without running anything: return where there's no function to return
from.
return_stmt -> "return" comparison? ";"A void function returns with a bare return;. Anything else needs a value to hand back.
src/ast.h · enum, directly below NODE_CALL
NODE_RETURN,src/ast.h · union, directly below the call member
struct { Node *value;NULL for a bare return;.
} return_stmt;src/ast.c · directly above ast_free
Node *ast_return(Node *value) { Node *node = new_node(NODE_RETURN); if (node == NULL) return NULL; node->as.return_stmt.value = value; return node;}src/ast.c · ast_free, directly below the NODE_CALL case
case NODE_RETURN: ast_free(node->as.return_stmt.value); break;src/parser.c · directly above parse_statement, below parse_fn
/* return_stmt -> "return" comparison? ";" */static Node *parse_return(Parser *p) { long offset = p->current.start - p->lexer.src; if (p->fn_depth == 0) { error_at(p, "'return' outside a function");Checked before consuming the token, while p->current still points at return itself, so the error points at the right place. This is one of this chapter's three golden error cases, and it's caught here, in the parser, rather than later while running the program.
return NULL; } parser_advance(p); /* 'return' */ Node *value = NULL; if (!parser_check(p, TOK_SEMICOLON)) { value = parse_comparison(p); if (value == NULL) return NULL; } if (!parser_match(p, TOK_SEMICOLON)) { error_at(p, "expected ';'"); ast_free(value); return NULL; } return at(ast_return(value), offset);}src/parser.c · inside parse_statement
if (parser_check(p, TOK_FN)) return parse_fn(p); if (parser_check(p, TOK_RETURN)) return parse_return(p);
A bare return; when value is NULL reaches
ast_free(value) on the error path harmlessly, since ast_free(NULL)
does nothing. No special case needed there.
Checkpoint
All six node kinds now exist and parse. eval is the only thing left that
doesn't know about any of them, and it's the whole of the next step.
No code in this step either. Before I show you a 60-line switch statement, you should know exactly why chapter five's two functions can't stay two, because the reason is not obvious from the diff alone.
Chapter five split evaluation cleanly in two: eval for anything that
produces a value, exec_statement for anything that just happens. Their real
signatures, quoted:
static long eval(const Node *node, Env *env, const char *src, int *ok);static int exec_statement(const Node *node, Env *env, const char *src);
That split held up fine right through chapter six, because nothing an expression did ever
needed to reach back into a statement's control flow, and nothing a statement did ever
needed a second kind of answer from an expression beyond its plain value. The two
functions could talk across the boundary in one direction, a value moving from
eval into exec_statement, and never had to talk back.
return talks back. A return statement lives on the statement
side, it evaluates an expression to get its value, hands that value up, and every block,
if and while standing between it and the function call waiting
for it has to notice "a return is in progress" and stop doing anything else, immediately,
all the way out. That noticing has to happen inside exec_statement's own
loop over a block's statements, which means exec_statement needs to hand
back more than 1 or 0: it needs to say whether it succeeded
and whether a return fired and, when one did, what value it carried.
That third thing, a value, is exactly what eval already returns. Once a
statement's return signal has to carry a value the same way an expression's result does,
keeping them as two different shapes, one a bare long with a side-channel
int *ok, the other a bare int, means writing the exact same
propagation logic once for each and hoping they never drift apart. So they become one
shape, checked into existence by one struct, and the two functions that produced the two
old shapes collapse into the one function that produces it.
src/main.c · near the top of the file, above every function that will use it
typedef struct { long value;A number for anything that has one. Zero and unread for a plain statement like let, the same way chapter five's exec_statement just returned 1 with nothing else to say.
bool ok;Chapter five's int *ok out-parameter, folded into the return value instead of threaded through as a pointer, since every caller gets a struct back anyway now. false means diag_error already ran and everything above this call has to stop, without an exit() in sight.
bool returning;New this chapter. false for everything, until a return fires, and then true for every frame its value climbs back out through on the way to the call that's waiting for it.
} ExecResult;
Every place chapter five wrote if (!ok) return 0; after a recursive call
becomes if (!r.ok || r.returning) return r; in the new function: stop this
block or loop right now and hand the exact same result up, whether the reason is a
runtime error that already happened or a return that's already in flight. Those turn out
to be the same instruction, so one guard covers both, instead of two separate mechanisms
each needing their own copy of it in every block, if, and while
case.
Chapter five's exec_statement ends its switch with a default:
case, and says exactly why: "This switch covers statement kinds, but
NodeType also contains expression kinds that can never appear here, so
listing them all would be noise." That was true. A NODE_NUMBER reaching
exec_statement really was a bug, since nothing ever called it on anything
but the six statement kinds, and refusing to write ten pointless empty cases to guard
against an impossible input was the right call.
It stops being true the moment eval and exec_statement are
the same function over the same switch. NODE_NUMBER reaching this function
is no longer a bug, it's print 5; doing exactly what it should. There is no
longer any NodeType value that can't legitimately show up here, so the
condition that justified the default has evaporated. The default goes. Every one of the
sixteen enumerators this chapter ends with gets a real case, and the no-default rule
that has governed every other switch in this project since chapter four finally applies
to this one too, without an exception carved out for it.
One more consequence, small but worth naming before the big listing: the merged function
drops the int *ok parameter, since ok now lives in the return
value, and it needs one new one. A call has to be able to find the function it's calling
anywhere in the program, not just in whatever Env it's standing in, so the
new signature is static ExecResult eval(const Node *node, Env *env, const char
*src, NodeList *program);. Step 12 is that function, in full.
Checkpoint
No build to run yet; nothing compiles differently until the next step replaces the
function itself. Just be able to say, out loud, why ok and
returning end up needing the identical guard everywhere they're checked.
One function, one switch, every node kind the language has. The hard part isn't any
single case, it's that return has to be able to interrupt a block that's
inside an if that's inside a while, and every layer in between
has to pass that interruption, or a runtime error, along instead of swallowing it.
eval and the function that resolves a call are about to call each other, so
one of them needs a forward declaration first.
src/main.c · near the top of the file, below the ExecResult typedef from step 11
static ExecResult eval_call(const Node *node, Env *env, const char *src, NodeList *program);eval's NODE_CALL case needs to call this before you've written its body. eval_call, once you do write it, needs to call eval, which is defined below it. Standard C fix for two functions that call each other: forward-declare one of them.
src/main.c · #include list at the top of the file
#include <string.h>For memcmp, comparing a call's name against each function declaration's name. Neither is null-terminated, so this is the same trap as chapter four's word_is: compare lengths first.
Here's the merged function, deleting exec_statement entirely and replacing
eval. Every case chapter five and six already wrote keeps its logic
unchanged; only the shape of what it hands back changes, from a bare long or
a bare int to (ExecResult){ value, ok, returning }. I'm showing
the full switch so there's one place that shows every node kind side by side, statement
and expression alike, exactly as the trap in step 11 said there would have to be.
src/main.c · replacing eval and exec_statement together
static ExecResult eval(const Node *node, Env *env, const char *src, NodeList *program) { switch (node->type) { case NODE_NUMBER: return (ExecResult){ node->as.number, true, false }; case NODE_BOOL: return (ExecResult){ node->as.boolean ? 1 : 0, true, false }; case NODE_IDENT: { Binding *b = env_find(env, node->as.ident.name, node->as.ident.length); if (b == NULL) { diag_error(src, node->offset, "undefined name '%.*s'", node->as.ident.length, node->as.ident.name); return (ExecResult){ 0, false, false };Not exit(1). Chapter five's whole point in adding ok was to stop this file from calling exit on a user's mistake, so nothing new in this chapter reintroduces it.
} return (ExecResult){ b->value, true, false }; } case NODE_UNARY: { ExecResult operand = eval(node->as.unary.operand, env, src, program); if (!operand.ok) return operand; return (ExecResult){ -operand.value, true, false }; } case NODE_BINARY: { ExecResult left = eval(node->as.binary.left, env, src, program); if (!left.ok) return left; ExecResult right = eval(node->as.binary.right, env, src, program); if (!right.ok) return right;Both sides get a chance to fail on their own terms before either one's value is used. Checking left the moment it comes back, rather than after evaluating right too, matters once one side can have side effects worth stopping early: an assignment inside left that already failed should not be followed by evaluating right as if nothing happened.
switch (node->as.binary.op) { case TOK_PLUS: return (ExecResult){ left.value + right.value, true, false }; case TOK_MINUS: return (ExecResult){ left.value - right.value, true, false }; case TOK_STAR: return (ExecResult){ left.value * right.value, true, false }; case TOK_SLASH: if (right.value == 0) { diag_error(src, node->offset, "division by zero");Chapter five moved this off fprintf and exit onto the ok-propagation path. It now also goes through diag_error like everything else in the project, rather than printing its own one-off format.
return (ExecResult){ 0, false, false }; } return (ExecResult){ left.value / right.value, true, false }; case TOK_EQEQ: return (ExecResult){ left.value == right.value, true, false }; case TOK_BANGEQ: return (ExecResult){ left.value != right.value, true, false }; case TOK_LESS: return (ExecResult){ left.value < right.value, true, false }; case TOK_LESSEQ: return (ExecResult){ left.value <= right.value, true, false }; case TOK_GREATER: return (ExecResult){ left.value > right.value, true, false }; case TOK_GREATEREQ: return (ExecResult){ left.value >= right.value, true, false };A C comparison already produces 0 or 1, which is exactly how chapter six represents a boolean. No conversion needed, just six new cases in a switch that was already dispatching on the operator token.
default: break; } break; } case NODE_CALL: return eval_call(node, env, src, program);Returned directly, not wrapped. eval_call hands back a real ExecResult now, and a call's own returning is always false here regardless of whether the function it invoked used return internally: that signal gets consumed inside eval_call, before it ever reaches this case.
case NODE_LET: case NODE_VAR: { if (env_find_local(env, node->as.binding.name, node->as.binding.length) != NULL) {env_find_local, not env_find. This is the trap from step 2: the climbing version would refuse a local name that only happens to share a spelling with an outer one, which is exactly the bug this whole chapter exists to fix.
diag_error(src, node->offset, "'%.*s' is already defined", node->as.binding.length, node->as.binding.name); return (ExecResult){ 0, false, false }; } ExecResult value = eval(node->as.binding.value, env, src, program); if (!value.ok) return value; if (!env_define(env, node->as.binding.name, node->as.binding.length, value.value, node->type == NODE_VAR)) { diag_error(src, node->offset, "too many variables, the limit is %d", ENV_MAX); return (ExecResult){ 0, false, false }; } return (ExecResult){ 0, true, false }; } case NODE_ASSIGN: { Binding *b = env_find(env, node->as.binding.name, node->as.binding.length);The climbing env_find, correctly, this time: reassigning a variable from inside a nested block is supposed to reach outward.
if (b == NULL) { diag_error(src, node->offset, "undefined name '%.*s'", node->as.binding.length, node->as.binding.name); return (ExecResult){ 0, false, false }; } if (!b->mutable) { diag_error(src, node->offset, "'%.*s' was declared with let, so it cannot be changed. Use var instead", node->as.binding.length, node->as.binding.name); return (ExecResult){ 0, false, false }; } ExecResult value = eval(node->as.binding.value, env, src, program); if (!value.ok) return value; b->value = value.value; return (ExecResult){ 0, true, false }; } case NODE_PRINT: { ExecResult value = eval(node->as.stmt.value, env, src, program); if (!value.ok) return value; printf("%ld\n", value.value); return (ExecResult){ 0, true, false }; } case NODE_EXPR_STMT: { ExecResult value = eval(node->as.stmt.value, env, src, program);NODE_PRINT and NODE_EXPR_STMT both read node->as.stmt.value. Chapter five gives them one shared union member, stmt, for the same reason NODE_LET, NODE_VAR and NODE_ASSIGN share binding: they're both just a wrapper around one expression.
if (!value.ok) return value; return (ExecResult){ 0, true, false }; } case NODE_BLOCK: { Env local; env_init(&local, env);The block's own scope, parented to whatever scope was active where the block appears. A fresh one every time this case runs, which for a loop body means a fresh one every iteration.
for (int i = 0; i < node->as.block.stmts.count; i++) { ExecResult r = eval(node->as.block.stmts.items[i], &local, src, program); if (!r.ok || r.returning) return r;The shared guard from step 11. An error or a return, either one, stops the rest of this block and hands the same result straight up to whoever called eval on it.
} return (ExecResult){ 0, true, false }; } case NODE_IF: { ExecResult cond = eval(node->as.if_stmt.cond, env, src, program); if (!cond.ok) return cond; if (cond.value != 0) { return eval(node->as.if_stmt.then_branch, env, src, program); } if (node->as.if_stmt.else_branch != NULL) { return eval(node->as.if_stmt.else_branch, env, src, program); } return (ExecResult){ 0, true, false };Whichever branch runs, its ExecResult is returned directly, error and return propagation both included for free. The branch is always a block, and a block already knows how to hand both signals upward.
} case NODE_WHILE: for (;;) {Not while (eval(...).value != 0). The condition itself can now fail, and a plain while expression has nowhere to put that check. An infinite for with an explicit break gives the condition's own ok a line to be checked on.
ExecResult cond = eval(node->as.while_stmt.cond, env, src, program); if (!cond.ok) return cond; if (cond.value == 0) break; ExecResult r = eval(node->as.while_stmt.body, env, src, program); if (!r.ok || r.returning) return r;A return inside a loop has to stop the loop, not just the current iteration's block. Propagating out of the whole for here, instead of just breaking it, is what makes that happen: the caller of eval on this NODE_WHILE sees returning == true too.
} return (ExecResult){ 0, true, false }; case NODE_FN: return (ExecResult){ 0, true, false };Reaching a function declaration while running top to bottom does nothing. It isn't an action, it's a name and a body sitting in program, waiting for eval_call to find it.
case NODE_RETURN: { if (node->as.return_stmt.value == NULL) { return (ExecResult){ 0, true, true }; } ExecResult value = eval(node->as.return_stmt.value, env, src, program); if (!value.ok) return value; return (ExecResult){ value.value, true, true };The only place returning ever starts out true. Everywhere else in this function either passes an existing result upward unchanged or manufactures a fresh false one.
} } fprintf(stderr, "internal error: unknown node type\n");Same fallback chapter one already had below its switch. The switch has no default and covers every enumerator, so this line is unreachable in practice, but it satisfies the compiler and matches how token_type_name handles the identical situation. This is the one exit() left in the file, and it's guarding against a bug in the compiler itself, not a mistake in someone's program, which is a different thing from what chapter five removed.
exit(1);}
Now the two functions that make a call actually happen. global_env walks a
parent chain to its root, and since a function call always parents its frame to that
root (never to whichever scope happened to call it), this is also how a function is kept
from seeing the caller's locals.
src/main.c · directly above eval_call
static Env *global_env(Env *env) { while (env->parent != NULL) env = env->parent; return env;}
It's tempting to write env_init(&frame, env) here, parenting the
call's new frame to whatever scope made the call. Try it on paper: a function
f defined once, called from two different places that each happen to have
a local named n with a different value, would silently read whichever
caller's n was lying around, because env_find climbs. That's
dynamic scoping, and it means a function's behavior would depend on who called it, not
on what it was written next to. Parenting to the global scope instead, always, is what
makes a function's meaning depend only on the source code sitting inside its own
braces.
src/main.c · replacing the forward declaration, directly above eval
static ExecResult eval_call(const Node *node, Env *env, const char *src, NodeList *program) { const Node *fn = NULL; for (int i = 0; i < program->count; i++) { const Node *stmt = program->items[i]; if (stmt->type == NODE_FN && stmt->as.fn.length == node->as.call.length && memcmp(stmt->as.fn.name, node->as.call.name, (size_t)node->as.call.length) == 0) { fn = stmt; break; } }A plain scan of the top-level program, no table built ahead of time. It also means a function can be called before its own fn appears further down the file: the whole program is searched, not just the part above the call.
if (fn == NULL) { diag_error(src, node->offset, "undefined function '%.*s'", node->as.call.length, node->as.call.name); return (ExecResult){ 0, false, false };The first golden error case: calling something that was never declared. Reported, not exited.
} int want = fn->as.fn.params.count; int got = node->as.call.args.count; if (want != got) { diag_error(src, node->offset, "'%.*s' expects %d argument%s, got %d", node->as.call.length, node->as.call.name, want, want == 1 ? "" : "s", got); return (ExecResult){ 0, false, false };The second golden error case: the wrong number of arguments. This checks a count, not the parameters' types; whether the values themselves are the right type is chapter eight's job.
} Env frame; env_init(&frame, global_env(env)); for (int i = 0; i < want; i++) { const Node *param = fn->as.fn.params.items[i]; ExecResult arg = eval(node->as.call.args.items[i], env, src, program);Arguments are evaluated in env, the caller's own scope, since that's where the caller's variables live. Only the new frame the function body runs in is parented to global.
if (!arg.ok) return arg; env_define(&frame, param->as.ident.name, param->as.ident.length, arg.value, false);Parameters bind like let, not var: a function can read n but not reassign it. Declare a fresh var from it inside the body if you need a working copy.
} ExecResult result = eval(fn->as.fn.body, &frame, src, program); if (!result.ok) return result;A runtime error inside the function body has to keep climbing past the call itself, not stop here as if the call had simply returned 0.
return (ExecResult){ result.returning ? result.value : 0, true, false };Falling off the end of a function without hitting return gives back 0. That's fine for a void function, whose result nobody should be reading anyway, and it's the reason a missing return in a non-void function isn't caught here: catching it needs to know the function's declared return type against what actually happened, which is exactly the kind of check chapter eight's sema pass is for. Note returning is hard-coded false in what this function hands back: whatever happened inside the call is over, and a call is just an ordinary expression to whoever asked for its value.
}
Last piece: main() needs a global Env, has to pass the whole
program down into every top-level call to eval so eval_call has
something to search, and now has something worth checking on the way out.
src/main.c · in main, where the driver runs the parsed program
Env global; env_init(&global, NULL); for (int i = 0; i < program.count; i++) { ExecResult r = eval(program.items[i], &global, src, &program); if (!r.ok) { return 1; } }Checkpoint
First clean build since step 4. Every switch over NodeType in the project
now handles all sixteen node kinds, with no default: anywhere among them,
and exec_statement no longer exists.
One program that has to work, and three that have to fail with the right message. The Makefile finds this file by name, same as every test file before it.
tests/test_functions_7.c · new file · copy the shape of tests/test_parser_2.c
fn fib(n: int) -> int { if n < 2 { return n; } var a = 0; var b = 1; var i = 2; while i <= n { var next = a + b; a = b; b = next; i = i + 1; } return b;} print fib(10);
Run it in your head before you run it for real. fib(10) takes the early
return n; only for n under 2, otherwise walks the loop eight
times. Each iteration declares a fresh next, which only proves the block
case makes a new scope every time through if next would otherwise collide
with the next iteration's own next, so this is the same collision from step
one, now shown to be actually fixed. You should see 55.
Then three programs that have to fail, one law each:
print missing(1);Calling a function that was never declared. eval_call's scan of program finds nothing, and you should see undefined function 'missing'.
fn add(a: int, b: int) -> int { return a + b; } print add(1);One argument for a function that wants two. You should see 'add' expects 2 arguments, got 1.
return 5;A return with no enclosing fn at all. This one is caught before the program runs a single statement, by parse_return's fn_depth check from step 10. You should see 'return' outside a function, and the parse fails cleanly rather than the program running partway.
Notice that the third error looks different in kind from the first two. It comes from
parser_parse returning failure, before main ever calls
eval on anything. The first two come from deep inside evaluation, after the
program has already been running for a while. Both are real errors, reported through the
same diag_error, but they're caught by two different parts of the compiler,
for a reason worth writing down in your own test file as a comment: a return outside a
function is a fact about the program's shape, knowable without running it, while an
undefined function or a wrong argument count depends on what actually gets called, which
you can't always know until you try.
Checkpoint
fib(10) prints 55, and all three golden errors name the right problem.