What Do Macros Actually Give You?

Posted: 4 years ago (2007-11-23 19:09:35 UTC ) / Updated: 2 years ago (2009-06-01 22:27:28 UTC )

Imported from WordPress

Originally posted on 2007-11-23 19:09:35

(Disclaimer: I would love to see somebody better-qualified tackle this topic. In the mean time, I think it's something that needs more discussion)

Paul Graham writes a lot of rhapsodies to using LISP for server-side apps. Particularly when it comes to LISP macros and their productivity.

For those who have never heard of LISP macros: LISP is a language with very little syntax. Its code is in the same syntax as its list data, so code consists of nested list data. For instance: "(print (+ 7 (- 6 4)))" is both perfectly good LISP code and, if you treat "print" as a piece of data, a perfectly good LISP list. You could write a little LISP function that takes a list as input and gives a list as output. If you then fed a piece of code through it to get another piece of code, that would be a macro. Neat, no?

Perl can do a similar trick by taking blocks of code as text and outputting more code as text, then passing it to 'eval'. That doesn't work very well because Perl is an incredible pain to parse. Python and Ruby do similar tricks, also with 'eval'. While they're not as awful to parse as Perl, they're still pretty bad. The brilliant stroke that makes LISP better is that it's trivial to parse, and can be passed around pre-parsed. So it's easy to write code that modifies other code.

So what would you do with this code-modifying-other-code? One answer is to fill in the blanks in a code template, rather like big messy C macros do. In languages like C where it's difficult to pass code around, this is a very useful thing. In languages like LISP, Python and Ruby where it's easier, this capability is mostly handled by method objects, closures and other higher-order-function stuff. So in those languages, you don't need macros to do that, and it's often a bad idea to have them do so.

Here's a simple example of fill-in-the-blanks code in Ruby:

  def grouping_iter(tokens, pre = proc{|tok|}, post = proc{|tok|}, &myproc)

    pre.call(tokens)


    newtokens = tokens.collect do |token|

      token.kind_of?(Array) ? grouping_iter(token, &myproc) : token

    end

    post_tok = myproc.call(newtokens)

    post.call(post_tok)

    post_tok

  end

This iterator is a bit like the 'map' statement, which takes a list and a function and returns a new list of results of that function call. The code above is like that, but it also descends into any lists inside the list and calls the function on them. So it's a simple, map-type convenience function. It also lets you pass separate "pre" and "post" functions for setup and teardown. So to call it, you might say:

grouping_iter(filelist, proc { |tok| init_jpeg_lib() }, proc { |tok| shutdown_jpeg_lib() }) { |sublist|

  open_jpeg_files(sublist)

}

This is "fill in the blank" template code because the iterator leaves blanks for

• code to run on your token list for setup


• code to run on each token in your nested list


• code to run on your final list for shutdown

A LISP macro can fill in those blanks in a similar way. In fact, LISP has first-class functions and closures, so common LISP macros like the LOOP statement could have instead been written to pass a function through as a parameter. That's how most Ruby iterators work, as well.

Macros can fill in the blanks in code templates with values rather than code. But that can be done by ordinary function calls -- only performance is potentially different, and it's not different enough for us to worry much about.

LISP macros can also be used for the kind of polymorphism that is handled by method dispatch in OO languages. That's a special kind of 'fill-in-the-blank' code like the above, except it runs different code depending on some other object. We know our other languages can do that, either by standard OO method dispatch, or by wacky things like Ruby's ability to override a method for any specific single object.

Macros can be used to replace particular operators, function calls or data constructs with slightly different ones -- for instance, in order to print debug messages, track allocated memory or otherwise do simple bookkeeping. Languages capable of Aspect-Oriented Programming do this tracking and bookkeeping routinely, but it's neat that LISP (vintage 1960) had a construct that got this right. Still, languages like Ruby that allow lots of rebinding of methods on standard object classes (for instance, overriding 'plus' on integers) can finally match this ability.

The final use that comes to mind is to track what functions or variables are used. On this, AOP and Ruby start to fall down, because they detect usages when these methods are called at runtime. It's much harder for Ruby or Java-plus-AOP to statically analyze code and determine what methods is can call or constants it can use [1]. LISP macros, since they can iterate through the code in full detail, can do this immediately at compile time. It's true that certain static analysis tools can do the same thing for the languages they operate on, but they're not usually callable from the program, and that makes a *lot* of difference. They also can't usually do much with functions that are defined dynamically or at run-time, while LISP macros have no trouble with that.

So a quick analysis suggests that Ruby (and other languages, I'm sure, just none I know well) can match most of the uses of LISP macros for most purposes... But *not* do code analysis the way good macros, applied well, can manage.

(I have written a response to this article's comments as well. If you found this interesting or enlightening, you may wish to read that as well)

[1]Ruby's "eval" on text blocks could be used to similar effect if it was easy to parse the text block for your static analysis. Sadly, there's not a good Ruby parser written in Ruby currently, nor invocable from the Ruby language. Other languages, especially easier languages to parse, may fare better on this point.