Composing macros inside-out with Julia

Macros vs. functions

Macros in Julia, denoted by the @ prefix, are used to transform code before it is executed. They are often used to reduce boilerplate or implement domain specific languages (DSLs).

In a way, they are just normal functions which take in an abstract syntax tree or AST and return a different one, and their execution happens before that of all the normal code.

While functions run inside-out, macros run outside-in. When you execute outer(inner(x)) then the inner function runs first, and outer takes in what inner outputs. But if you execute @outer(@inner(x)), then @outer runs first, only after which any remaining macros inside the AST it outputs are run. In this example could still be @inner depending on what @outer outputs, but it doesn’t have to be. For example, there could be a @remove_macros macro, which just deletes any macro calls inside its body.

Macros don’t compose

One problematic consequence of this is that macros do not really compose. For example, let’s say you have two macros that operate on function definitions and add some useful things to them.

Let’s make one which wraps the body of a function in a timing operation. Note that I do this with an inner function as a quick-and-dirty way, because otherwise I have to deal with possibly multiple return statements from the function body.

macro functime(expr)
    expr.head == :function || error("Not a function expression.")
    funcname = expr.args[1].args[1]
    :(
        function $(esc(funcname))(args...; kwargs...)
            f = $expr
            println("Started execution at $(time())")
            result = f(args...; kwargs...)
            println("Stopped execution at $(time())")
            return result
        end
    )
end

@functime function func()
    sleep(0.5)
    return "result"
end

func()

Started execution at 1.694535142499498e9
Stopped execution at 1.694535143013996e9

"result"

And here’s one that just logs that the function is being run:

macro funclog(expr)
    expr.head == :function || error("Not a function expression.")
    funcname = expr.args[1].args[1]
    :(
        function $(esc(funcname))(args...; kwargs...)
            f = $expr
            @info("Running function.")
            result = f(args...; kwargs...)
            return result
        end
    )
end

@funclog function func2()
    sleep(0.5)
    return "result"
end

func2()

[ Info: Running function.

"result"

But you cannot use both macros at once on a single function definition, because each macro expects an expression in form of a function definition as its argument. And putting a different macro inside means that the expression is of type :macrocall and not type :function, which our macros don’t know how to deal with.

So this doesn’t work:

@functime @funclog function func3()
    sleep(0.5)
    return "result"
end

Of course we can use higher-order functions for what I’m showing here, but that’s not the point of the exercise, it’s to try and see if we can use macros in a layered / composed way.

The inside-out macro

What I wanted to try here was to make the macros run from inside-out, like functions. For this, I made another small meta-macro which calls macroexpand from the inside out if it encounters multiple macros (with recursive = false because we want to keep any macros inside the main body intact throughout the transformations like usual). That means @insideout @macro1 @macro2 expr first expands @macro2 expr and then @macro1 output_expr.

macro insideout(exp)
    function apply_macro(exp::Expr)
        if exp isa Expr && exp.head == :macrocall
            exp.args[3] = apply_macro(exp.args[3])
            return macroexpand(@__MODULE__, exp, recursive = false)
        else
            return exp
        end
    end
    
    apply_macro(exp)
end

@insideout (macro with 1 method)

This means one can now compose macros:

@insideout @functime @funclog function func3()
    sleep(0.5)
    return "result"
end

func3()

Started execution at 1.694535144477041e9
Stopped execution at 1.694535144978576e9

[ Info: Running function.

"result"

These examples are contrived but I wonder if someone can come up with a more interesting use-case for the technique.

At least it’s fun trying @insideout with some of the usual macros to modify what happens in an interesting way:

For example, using @show on @show:

@show @show 1 + 2

1 + 2 = 3
#= In[6]:1 =# @show(1 + 2) = 3

3

vs. with the inside-out mode, which turns @show into some weird kind of macroexpand-and-run:

@insideout @show @show 1 + 2

1 + 2 = 3
begin
    Base.println("1 + 2 = ", Base.repr(begin
                #= show.jl:1128 =#
                local var"#186#value" = 1 + 2
            end))
    var"#186#value"
end = 3

3