Clojure macros continue to surprise me
138 points
1 month ago
| 8 comments
| tonsky.me
| HN
lloydatkinson
1 month ago
[-]
I really enjoyed reading this, and I don't know Clojure at all. This is a problem I've encountered several times while building design systems/component libraries. It's a very universal documentation problem, regardless of what platform it's targeting.

I have done the same with React - but it gets messy. Something nice like in the post would be great for that.

Too often I see very poor Storybook sites in the wild (a web based design system visual explorer, somewhat like the screenshot in the post) where no thought was given to this exact problem.

Instead of something meaningful like in the post, I see:

    () => <TextInputStorybook />
As the only avaliable code. The point of the code blocks is so people can copy paste. When I see that, I know I'm in for a rough ride because the code quality inevitably isn't any good either.

For reference, this is easily the best design system I know of: https://seek-oss.github.io/braid-design-system/components/Te... I use this as a reference when I work on my own.

reply
josefrichter
1 month ago
[-]
In elixir you can run docs as (init) tests too. The doctest macro is generating the tests from the docs within the module. Maybe this might be interesting together with this article.
reply
seancorfield
1 month ago
[-]
The fifth time this has been posted in just over a week... it must be a REALLY good article! :)
reply
seancorfield
1 month ago
[-]
My bad... the sixth time (the fifth time was under a different title).
reply
kazinator
1 month ago
[-]
> What if our macro read the source file?

> Like, actually went to the file system, opened a file, and read its content? We already have the file name conveniently stored in file, and luckily Clojure keeps sources around.

> So this is what I ended up with:

> (defn slurp-source [file key]

Looks like a function to me (which is good).

reply
chii
1 month ago
[-]
it can't be a function because the source is only guaranteed to be available at compile time, rather than at runtime.
reply
kazinator
1 month ago
[-]
It's reading code from a file, which you can make arrangements for to be there at any time you want.

Files are not directly relevant to macros.

The code that a macro invocation has access to by design is the datum which comprises that invocation itself. That may have come from an expression in a file, and may be annotated in some way with the location info. Other than that, macros do not necessarily have access to their textual source code. No such thing necessarily exists because macros can be invoked by code that was generated (possibly by other macros or in other ways) and so never existed as characters in a file.

I don't have a clear picture of the architecture of the system that the article refers to, but the author presents, as his end result, a defn form, which defines a function and not a macro.

If a macro is needed to get that function to be called at compile time, that would be a minor hooking detail, and not an instance of a macro being used to perpetrate a complex code transformation. The function just needs a file name and a string.

(It looks as if the author's system may be interactive; the code files are there all the time and can be changed, and he wants the view of the files to refresh?)

reply
lispm
1 month ago
[-]
It could be a helper-function which gets called by a macro expansion at compile-time. One would define the feature as a function which is available at compile-time and the macro calls it during expansion.
reply
phoe-krk
1 month ago
[-]
A macro is a perfectly normal function. It's simply called at compile time, with its contents as its argument.
reply
chii
1 month ago
[-]
no you're misunderstanding me.

I'm saying that the macro's purpose is to read the source file and produce something at compile time. It's not a "normal" (read: runtime) function, as this feature cannot be implemented using a function that only executes at runtime.

reply
phoe-krk
1 month ago
[-]
Sure, but the macro function is a normal function - it has inputs, outputs, and (preferably) should be pure. There's nothing special about the macro function itself, really.

You can see it explicitly in Common Lisp, where you can easily access and replace a macro-function with a custom one that you DEFUN yourself, like this.

    CL-USER> (defun my-quote-fn (form env)
               (declare (ignore env))
               (destructuring-bind (operator &rest body) form
                 (assert (eq operator 'my-quote))
                 (list* 'quote body)))
    MY-QUOTE-FN
It's a perfectly normal function, I can call it right away:

    CL-USER> (my-quote-fn '(my-quote (+ 2 2)) nil)
    '(+ 2 2)
I can also tell the compiler that this perfectly normal function is a macro function...

    CL-USER> (setf (macro-function 'my-quote) #'my-quote-fn)
    #<FUNCTION MY-QUOTE-FN>
...and the compiler will start using it to expand macros:

    CL-USER> (macroexpand-1 '(my-quote (+ 2 2)))
    '(+ 2 2)
    T
What is special about macros, and what you describe as the "feature that cannot be implemented", is the language's ability to call that function at compilation time and feed its output back into the compiler. This is the link that is missing in most programming languages.
reply
chii
1 month ago
[-]
> call that function at compilation time

that's why it's called a macro and not a function. You're using the word function in the mathematical sense, and not in the lisp sense.

reply
kazinator
1 month ago
[-]
The irrefutable fact is that the author presents the core logic of the file-reading solution as a defn, not a defmacro. It may be driven by a macro, but evidently that part is not noteworthy enough to be presented.
reply
KingMob
1 month ago
[-]
I mean, sorta, but Lisp macro writers learn early on that macros are also functions, it's just they operate on a list of symbols, and are called in a different phase.

So saying "it can't be a function" doesn't ring true, either.

reply
compacct27
1 month ago
[-]
The real trick here is dodging ASTs, which, after trying to use in so many parse-the-code projects, really aren’t needed all the time but are put pretty highly on the pedestal
reply
layer8
1 month ago
[-]
An AST can carry the original source representation on each node as metadata — best of both worlds.
reply
simply-typed
1 month ago
[-]

  sometimes a vector is just a vector, but sometimes it’s a UI component and shows the structure of the UI.
Easily the worst aspect of Clojure. Everything is an untyped map or an untyped vector. Your editor can't tell the difference between a vector of keywords and some DSL for a macro. Things like this make Clojure a nightmare to refactor and scale poorly relative to languages like TypeScript.
reply
geokon
1 month ago
[-]
Thankfully this hiccup style seems less common in recent code. ie. where the index in the vector has a hidden meaning.. Now vectors are typically used for array of the same datatype. I think this makes more sense as basically if you see a vector, you can shove it in to map/filter/etc.

People now prefer using maps - even if it's a bit more verbose. Not sure why records fell out of fashion. You get the terseness of hiccup and the named keys of maps

reply
pjmlp
1 month ago
[-]
Just wait for using full blown CL macros then.
reply
packetlost
1 month ago
[-]
CL macros are more or less the same as Clojure macros and neither hold a candle to modern Scheme or Racket macros :)
reply
BoingBoomTschak
1 month ago
[-]
Eh? Pretty sure they're more powerful. And readable, unlike the Scheme ones that look as elegant and well integrated as a caravan being pulled by a Porsche =).
reply
yladiz
1 month ago
[-]
Used in the way the author of the article presents, or generally the way you could use them in Clojure, Racket and Scheme macros are pretty similar. However at least in Racket you have a lot more control over certain parts of the way compilation affects the macro (look up phases for example). I assume Scheme has similar constructs, as does CL.

Note this is ignoring anything about the reader, for which Racket has substantially more powerful functionality around than Clojure.

reply
bjoli
1 month ago
[-]
Writing defmacro in syntax-case is simple. Writing syntax-case with defmacro is not.

Everybody seems to think scheme macros means syntax-rules. You can get something pretty cl-esque with implicit renaming macros, but with hygiene as opt-out instead of opt-in.

reply
BoingBoomTschak
1 month ago
[-]
Didn't R7RS drop syntax-case?
reply
bjoli
1 month ago
[-]
It just deferred a lower level macro system to r7rs-large.
reply
kazinator
1 month ago
[-]
TXR Lisp:

  $ cat slurp-source.tl
  (defun slurp-source (path key)
    (let* ((lines (file-get-lines path))
           (match (member-if (op contains key) lines))
           (tail (rest match))
           (indent (find-min-key tail : (op match-regex @1 #/\s*/)))
           (dedent (mapcar (op drop indent) tail)))
      `@{dedent "\n"}\n`))
  $ txr -i slurp-source.tl
  1> (put-string (slurp-source "slurp-source.tl" "let*"))
       (match (member-if (op contains key) lines))
       (tail (rest match))
       (indent (find-min-key tail : (op match-regex @1 #/\s*/)))
       (dedent (mapcar (op drop indent) tail)))
  `@{dedent "\n"}\n`))
  t
reply
fithisux
1 month ago
[-]
Wow!
reply