I finished Let Over Lambda recently. I'm not an experienced lisp programmer but it's strangely readable if you allow yourself to not understand every little detail of the Common Lisp code. For example I hadn't read On Lisp. I had read The Little Schemer.
It's a wonderful tour of advanced macro concepts. With several of the macros presented I found myself saying "well that's just x language feature in y" but that's missing the point. The point is that Common Lisp allows you to implement your own language features, with macros of course!
This chapter is a wonderful climax to the whole book. Lisp moving Forth moving Lisp reminded me of the intertwined nature of DNA, RNA and proteins in Biology.
There's great humour throughout the book, I especially enjoyed the little sideways jabs at Scheme programmers!
peheje•Jul 29, 2026
What are some genuinely useful things macros can do?
db48x•Jul 29, 2026
They can define new syntax and special forms.
Akronymus•Jul 29, 2026
AFAICT, actually manipulating the AST, which then in turn allows you to implement your own language features, inserting additional debugging info, memoizing a function by just wrapping it in a macro, or optimizations based on domain knowledge.
arvyy•Jul 29, 2026
all the things, if macro system is sufficiently sophisticated (ie., hygienic with ability to attach metadata to syntax object). Racket is one of languages with such system, and they implemented static typing using them.
Vohlenzer•Jul 29, 2026
Interestingly, Let Over Lambda eschews hygienic macros because they limit the power (and danger) inherent in macro design.
klibertp•Jul 29, 2026
IIRC, the criticism is of syntax-rules and syntax-case kind of macros in Scheme; I don't recall (happy to be proved wrong) LoL directly mentioning syntax-parse that's specific to Racket. syntax-parse is no less powerful than defmacro; it's just (much) more structured and featureful. It's a difference between hand-rolling a recursive-descent parser and using a (really robust) PEG library.
Racket's define-syntax/syntax-parse has escape hatches into procedural generation (through which you can easily break the hygiene), and the equivalent of CL's &environment contains more information and allows more operations in Racket. Racket also supports reader extensions but often doesn't need them because it has a macro-based API for module definitions (a feature completely absent from, or even alien to, Common Lisp; OTOH, Elixir borrows some of Racket's ideas in this space).
For a practical example, Coalton and Typed Racket implementations show that both systems are capable of large-scale language extensions. I never studied their respective codebases, so I can't say anything about relative ease of introducing those extensions, but they are possible in both cases, at least.
arvyy•Jul 29, 2026
what I meant, the macro system should support hygiene, not necessarily that all macros in it must be hygienic. The syntax bits being manipulated come with an extra scope context, which means you get better preserved information about initial structure of your program; the more information macro has, the more it can achieve. And crucially, in a macro system capable of hygiene, you're not implicitly losing unhygienic capabilities. Implementing unhygienic macros in a hygienic system just means deleting all the scope information. Implementing hygienic macros in unhygienic system is a question mark. As (a somewhat imprecise) analogy, consider dynamic vs lexical scoping. If you have just lexical scoping support, you can mimic dynamic scoping with globals (since global scope is what's left when you delete other scopes). If you have just dynamic scoping support, implementing
lexical scope is a question mark.
What Let Over Lambda calls limited power, is probably in reference narrowly to `syntax-rules`. It's not just that it's hygienic, but that it purposefully has no way to opt out of it (unhygienic capabilities in a hygienic system are trivial to provide; but they still need to be provided by language implementation) but what's even more limiting is that it doesn't let you write procedural macros, it's all arcane pattern matching and templates. `syntax-case` (part of r6rs) doesn't have these issues.
Vohlenzer•Jul 29, 2026
One of the cleanest examples in the book is the CL-PPCRE library for regular expressions.
Without macros, regular expressions are strings at compile time and then at runtime objects are created based on the strings to evaluate them.
With macros, regular expressions are expanded at compile time into Common Lisp code that implements the evaluation.
There's a similar idea with an ORM library.
Without macros, the object-relational mapping is defined at compile time and any dynamic SQL is generated at run time.
With macros, the object-relational mapping is used at compile time to expand the query macros into Common Lisp code that includes the actual SQL.
Then also remember that the compiler is available at runtime in a lisp, so the distinction only matters when comparing it to non-lisps.
Take C#, if you want a new language feature or syntax added you have to propose it to the language design team, they have to decide it's worth implementing then you have to wait on it being implemented and released and your tooling updated.
If I want some new syntax in a LISP, I just write a macro.
e.g. the following threading macro in clojure
(->> x
foo
bar)
Takes x and passes it as the last argument to foo, then the result of foo is passed as the last argument to bar. Alternative you have -> for passing it as the first argument. These are from the standard library, but if they weren't they would be trivial to write yourself.
And no strings involved when you write macros in lisp either. It's just lists, your macro is just building a list. Because the syntax is just lists and there's no difference between lists and the syntax.
dreamcompiler•Jul 29, 2026
And that particular threading macro looks a lot like monadic code in Haskell, which looks a bit like Forth, which looks like a shell command with pipes.
Macros let a domain programmer (as opposed to the compiler writer) in Lisp implement almost any other programming paradigm as a native language feature.
throw-qqqqq•Jul 29, 2026
Generally, it enables type-safe, ahead-of-time compiled code to perform work at compile time that would require runtime execution in many other languages
dreamcompiler•Jul 29, 2026
That's true but it's not the most important aspect of macros. Macros let you define a new language that describes your problem solution more directly than you can in raw Lisp. In other words they let you create a Domain-Specific Lamguage (DSL). You can create DSLs in other languages too but it's so difficult that it's uncommon. In Lisp it's so easy that DSLs are a primary programming technique.
klibertp•Jul 29, 2026
I don't think that moving the evaluation to compile time is something you can just ignore when discussing macros. It's an important differentiator between macros and Fexprs[1].
IME, the primary use case for macros in Lisps is language extension, a step before a DSL. For every `loop` there are thousands of `with-x` macro implementations. See the macro-writing macros in Alexandria/Serapeum for the most common patterns.
Also, some dynamic languages (Io, Prolog, Groovy) support DSLs through non-macro mechanisms. DSLs in those languages are as easy (or easier) to implement as in Lisps; however, they tend to be less performant. So, again, compile-time macro execution does matter for the DSL use case, too.
In the case of Prolog, term and goal expansion are arbitrary compile-time execution mechanisms, completely isomorphic with macros, and are referred to as macro mechanisms.
The characterization of it not being "compile-time macro execution" is strange, what do you think these predicates are doing?
klibertp•Jul 29, 2026
Good point. I was thinking about meta-interpreters (and it's still possible I misunderstood or misremembered how they work!), but you're right; I should have at least been more precise.
carlosneves•Jul 29, 2026
Thanks for linking to Fexpr's. I've thinking about "functions with lazily evaluated arguments" recently, and trying to understand how they are different from macros. So I was basically thinking about fexprs.
I have some reading to do, but overall it seems like macros are favored instead of fexprs. Macros completely avoid some environment handling issues.
klibertp•Jul 29, 2026
The only language with Fexprs that I used was Io. In Io, unevaluated code is represented as a tree of Message nodes, and for each call, an activation record is instantiated and provided to the called method body. That reified Call object lets you access the caller's environment and the raw Message chains passed as arguments. You can then decide whether to evaluate those messages, which ones, how many times, and in what context/environment. It's a very niche language, but if you want to see Fexprs "in action", that's the best place to look at (at least to my knowledge) - they are central to Io's design rather than an advanced feature nobody touches. It's even mentioned on the front page[1]:
> Messages as Code — Messages form trees that can be inspected and rewritten at runtime. Argument evaluation can be deferred, so if, while, and for are implementable in Io itself.
I don’t disagree, but I think this is a matter of taste: i.e. which aspect you’d consider more important.
You can create DSLs without macros, for example by parsing and interpreting at runtime, but macros let you move much of that logic to compile time.
For me, that's the killer feature: writing code that looks dynamic or interpreted, but ends up as compiled code.
klibertp•Jul 29, 2026
If your language has macros, you don't need to wait for it to gain async/await (and it was quite a wait even in Python and JS, while Java got its virtual threads very recently); you can just[1] implement a CPS transform as a macro:
[1] Yeah, I know, this word is extremely overworked in that sentence. Sorry, that's an exaggeration for effect. In reality, it's not that simple, and not always possible: it depends on the language and macro system. It's still more likely to be implementable in library code if there is a macro system, though.
dreamcompiler•Jul 29, 2026
> reminded me of the intertwined nature of DNA, RNA and proteins in Biology
My analogy is how a changing electric field generates a changing magnetic field which generates a changing electric field which ... is light.
Forth is the electric field and Lisp is magnetic (or vice versa). They are two sides of the same coin.
Lisp really is the Maxwell's equations of software.
kamaal•Jul 29, 2026
I learned some Forth and played around right before the LLM trend began.
It was a mind bending experience, although I quit after a while as the programs were kind of glass sheet quality brittle as they grew.
Sometimes I feel I need to start again to keep my brain from losing thinking skills over time.
I like the idea of s-expressions over the wire. Have you thought about replacing the JSON with s-expressions for snapshots?
Archit3ch•Jul 29, 2026
I'm not fluent enough in Common Lisp for that book, but the patterns in Let over Lambda translate cleanly to Julia. You can have "Let over Lambda", "Let over Lambda over Let over Lambda", "Anaphoric Macros" (though you have to break hygiene with esc(), and in practice should just use the native julia do block), "Reader Macros" (via String Macros), "Pandoric Macros", and so on.
4 Comments
It's a wonderful tour of advanced macro concepts. With several of the macros presented I found myself saying "well that's just x language feature in y" but that's missing the point. The point is that Common Lisp allows you to implement your own language features, with macros of course!
This chapter is a wonderful climax to the whole book. Lisp moving Forth moving Lisp reminded me of the intertwined nature of DNA, RNA and proteins in Biology.
There's great humour throughout the book, I especially enjoyed the little sideways jabs at Scheme programmers!
Racket's define-syntax/syntax-parse has escape hatches into procedural generation (through which you can easily break the hygiene), and the equivalent of CL's &environment contains more information and allows more operations in Racket. Racket also supports reader extensions but often doesn't need them because it has a macro-based API for module definitions (a feature completely absent from, or even alien to, Common Lisp; OTOH, Elixir borrows some of Racket's ideas in this space).
For a practical example, Coalton and Typed Racket implementations show that both systems are capable of large-scale language extensions. I never studied their respective codebases, so I can't say anything about relative ease of introducing those extensions, but they are possible in both cases, at least.
What Let Over Lambda calls limited power, is probably in reference narrowly to `syntax-rules`. It's not just that it's hygienic, but that it purposefully has no way to opt out of it (unhygienic capabilities in a hygienic system are trivial to provide; but they still need to be provided by language implementation) but what's even more limiting is that it doesn't let you write procedural macros, it's all arcane pattern matching and templates. `syntax-case` (part of r6rs) doesn't have these issues.
Without macros, regular expressions are strings at compile time and then at runtime objects are created based on the strings to evaluate them.
With macros, regular expressions are expanded at compile time into Common Lisp code that implements the evaluation.
There's a similar idea with an ORM library.
Without macros, the object-relational mapping is defined at compile time and any dynamic SQL is generated at run time.
With macros, the object-relational mapping is used at compile time to expand the query macros into Common Lisp code that includes the actual SQL.
Then also remember that the compiler is available at runtime in a lisp, so the distinction only matters when comparing it to non-lisps.
A small tidbit: you don't actually need full syntactic macros to get compile time computation in CL, compiler macros (https://www.lispworks.com/documentation/HyperSpec/Body/03_bb...) suffice.
If I want some new syntax in a LISP, I just write a macro.
e.g. the following threading macro in clojure
(->> x foo bar)
Takes x and passes it as the last argument to foo, then the result of foo is passed as the last argument to bar. Alternative you have -> for passing it as the first argument. These are from the standard library, but if they weren't they would be trivial to write yourself.
And no strings involved when you write macros in lisp either. It's just lists, your macro is just building a list. Because the syntax is just lists and there's no difference between lists and the syntax.
Macros let a domain programmer (as opposed to the compiler writer) in Lisp implement almost any other programming paradigm as a native language feature.
IME, the primary use case for macros in Lisps is language extension, a step before a DSL. For every `loop` there are thousands of `with-x` macro implementations. See the macro-writing macros in Alexandria/Serapeum for the most common patterns.
Also, some dynamic languages (Io, Prolog, Groovy) support DSLs through non-macro mechanisms. DSLs in those languages are as easy (or easier) to implement as in Lisps; however, they tend to be less performant. So, again, compile-time macro execution does matter for the DSL use case, too.
[1] https://en.wikipedia.org/wiki/Fexpr
The characterization of it not being "compile-time macro execution" is strange, what do you think these predicates are doing?
I have some reading to do, but overall it seems like macros are favored instead of fexprs. Macros completely avoid some environment handling issues.
> Messages as Code — Messages form trees that can be inspected and rewritten at runtime. Argument evaluation can be deferred, so if, while, and for are implementable in Io itself.
[1] https://iolanguage.org/
You can create DSLs without macros, for example by parsing and interpreting at runtime, but macros let you move much of that logic to compile time.
For me, that's the killer feature: writing code that looks dynamic or interpreted, but ends up as compiled code.
https://github.com/nim-works/cps/blob/master/docs/cps.svg
[1] Yeah, I know, this word is extremely overworked in that sentence. Sorry, that's an exaggeration for effect. In reality, it's not that simple, and not always possible: it depends on the language and macro system. It's still more likely to be implementable in library code if there is a macro system, though.
My analogy is how a changing electric field generates a changing magnetic field which generates a changing electric field which ... is light.
Forth is the electric field and Lisp is magnetic (or vice versa). They are two sides of the same coin.
Lisp really is the Maxwell's equations of software.
It was a mind bending experience, although I quit after a while as the programs were kind of glass sheet quality brittle as they grew.
Sometimes I feel I need to start again to keep my brain from losing thinking skills over time.