Jank development update – Moving to LLVM IR
183 points
3 days ago
| 12 comments
| jank-lang.org
| HN
masijo
1 day ago
[-]
Incredible work. A native Clojure would be a dream come true!

Wish jank the best of lucks. Hope I can contribute soon.

reply
systems
19 hours ago
[-]
From their website, Jank is a dialect of clojure, and strongly compatible, I am not 100% sure what this means, but to me at least, it means it is not clojure and it is not native coljure

That being said, what would be the benefits of a native clojure, you have common-lips and guile if you want native

But again, Jank is not clojure .. just clojure-like, or so it seem

reply
refset
18 hours ago
[-]
Clojure is language that was designed to be 'hosted' on many different underlying language runtimes, not just the JVM/Java. If Jank can implement clojure.core correctly (pass the tests!) and handle .cljc files then it counts as Clojure in the eyes of the community. See also ClojureScript, ClojureDart, ClojureCLR, SCI, Cherry, Electric, Rama etc.
reply
refset
40 minutes ago
[-]
** Clojure is _a_ language [...]

(apologies!)

reply
Jeaye
15 hours ago
[-]
> But again, Jank is not clojure .. just clojure-like, or so it seem

jank is Clojure. If it works on Clojure JVM and it works on ClojureScript, it should generally work on jank. Clojure doesn't have a language specification, but the community knows what a proper Clojure should feel like. Strong Clojure compatibility means anything outside of interop should just work. Naturally, interop on Clojure is very host-specific; Clojure JVM does one thing, ClojureScript does another, Clojure CLR does a third thing.

> That being said, what would be the benefits of a native clojure, you have common-lips and guile if you want native

At this point, we'd be discussing the benefits of Clojure over CL and Guile, which ends up being a different discussion. jank is for Clojure devs who want a native host and good native interop. It's also for native devs who want to introduce interactive programming and FP into their systems.

reply
dustingetz
14 hours ago
[-]
can you use jank in .cljc files and portable code regions just work? platform specific interop guarded of course?
reply
Jeaye
13 hours ago
[-]
Yes! jank has its own :jank reader conditional which you can use.
reply
eduction
17 hours ago
[-]
refset is right, if this implements clojure.core it will be seen as a "real" clojure in the community, ala ClojureScript.

People use the Clojure that is the right fit for the environment they need to operate in, so arguments about authenticity don't tend to come up. Clojure has always been positioned as a pragmatic Lisp and the situation very much reflects that.

The OG Clojure is the one for the JVM and for some people this will always be the only "real" one in some sense, because JVM interop is a huge deal - it gets you access to tons of high-performance Java libraries. If you're writing server-side Clojure that's hard to beat.

But the other clojures have strengths in their niches. If you're writing for the browser, JVM interop doesn't buy you anything but JS interop does, hence ClojureScript. If you're doing a lot of shell scripting you'll probably want babashka for fast startup times. If you're doing mobile+desktop GUI development with Flutter you'll use ClojureDart (if you use Clojure at all).

Jank is aiming for C++ and I'll be curious if there is an interop story there. But again if you're looking to interop with an existing C++ code base it's sort of academic to ask whether Jank is "real" clojure or not.

reply
RossBencina
1 day ago
[-]
Is LLVM IR a stable target these days? I once heard of a project that got bit pretty hard with LLVM IR interface changes in the (not all that recent) past.
reply
michielderhaeg
1 day ago
[-]
Absolutely not. They do invasive changes all the time. At work we maintain an LLVM compatibility library that provides a stable interface across different LLVM versions. It has grown quite big over the years. The pure C interface is in comparison much more stable, but also quite limited.

Not just the API changes, the LLVM IR syntax can change drastically as well. E.g. not too long ago they switched from types pointer types to opaque pointers. `i32*` just becomes `ptr`. [1]

[1] https://llvm.org/docs/OpaquePointers.html

reply
fooker
1 day ago
[-]
IR can be auto-upgraded between LLVM versions.
reply
almostgotcaught
22 hours ago
[-]
citation? i work on LLVM (like llvm/llvm-project) and i'm not aware of such a tool.
reply
theresistor
21 hours ago
[-]
The textual IR is not backwards compatible, but the bitcode format has been best-effort auto-upgradeable for as long as I've been involved (2006). The policy is documented here: https://llvm.org/docs/DeveloperPolicy.html#ir-backwards-comp...
reply
almostgotcaught
20 hours ago
[-]
bitcode policy i'm aware of - it's the "IR" part that caught my attention.
reply
mannycalavera42
20 hours ago
[-]
a pretty evident case of not almost got caught
reply
fooker
1 day ago
[-]
As long as you use the C++ API, the pace of change is reasonable.

If you try to keep by by generating textual IR, it can be a nightmare.

reply
michielderhaeg
23 hours ago
[-]
I do agree that the pace is reasonable. Both with opaque pointers and the new pass manager API there was a quite a long transition period where both were supported before the old system was deprecated/removed. But with every new non-patch release, some part of the public API is changed and we have to adapt.
reply
almostgotcaught
22 hours ago
[-]
> But with every new non-patch release, some part of the public API is changed and we have to adapt.

bumping only on major release is recipe for pain and misery. everyone project i'm familiar with bumps weekly against HEAD. much more manageable.

reply
michielderhaeg
23 hours ago
[-]
I think the bitcode parsers (binary and textual) are quite backwards compatible. Old bitcode IR will be upgraded transparently. So you might not have to keep up.
reply
mightyham
21 hours ago
[-]
I'm curious what advantages Jank has over GraalVM, since Graal also supports interop with llvm languages.
reply
refset
20 hours ago
[-]
Jank should be able to produce significantly smaller binaries than Graal native images at the very least, but the potential for performance gains looks to be rather large, e.g. per "jank has been consistently beating Clojure in benchmarks" [0]

[0] https://jank-lang.org/blog/2023-07-08-object-model/

reply
mightyham
17 hours ago
[-]
That makes sense that the potential for performance improvements is greater, I'm just a bit skeptical it can actually "consistently beat clojure" in anything except for micro-benchmarks. It's mentioned in that article that jank uses a pretty basic GC while the JVM/Graal contains multiple state-of-the-art GCs tailored to different use cases. It's been shown many times that the JVM or Graal's JIT compiler have better peak performance over a long duration than compiled programs. This simply comes down to the fact that there are a host of specialized optimization strategies only possible when compilation can happen during runtime.
reply
refset
16 hours ago
[-]
> there are a host of specialized optimization strategies only possible when compilation can happen during runtime

That's a big part of why Jank is interesting to me, because it can properly embrace the idea of runtime JIT using LLVM. The JVM may still have an edge with profile-guided optimization for the foreseeable future, but perhaps Jank can help close that gap and support even more advanced possibilities for data-intensive workloads, e.g. along the lines of https://www.lingo-db.com/

reply
Jeaye
15 hours ago
[-]
With Graal native images, JIT compilation is not an option any longer. The runtime is static. With jank, you have a native binary, but you can JIT compile code with it and interop seamlessly with the native world. jank will also have an option for static runtimes, with no JIT compilation support, which can perform direct linking, better inlining, and whole program LTO. These builds will be smaller and lighter than native images.

Already, jank is faster than native images in the benchmarking I've done. The largest benchmark, so far, was a ray tracer.

reply
amelius
20 hours ago
[-]
Maybe there are legal benefits because it's not from Oracle?
reply
anovick
1 day ago
[-]
I love this writing style, very easy to read. Best of luck with the optimizations effort! looking forward to hearing more about this project.
reply
Jeaye
15 hours ago
[-]
Thanks for the feedback on the writing! Stay tuned. :)
reply
fire_lake
1 day ago
[-]
IIUIR, Clojure uses the JVM garbage collector to clean up memory.

How does Jank do this whilst keeping the user code… still Clojure?

reply
refset
1 day ago
[-]
Jank uses Boehm GC (written in C, also used by the likes of Inkscape, Guile and Mono) - it is discussed a little in the previous blog posts, e.g. https://jank-lang.org/blog/2023-07-08-object-model/
reply
k_g_b_
22 hours ago
[-]
Guile seems to prepare to switch to https://github.com/wingo/whippet - a GC C library that can wrap Boehm/BDW but also provides more (and less) advanced GC options. Should be a good option for Jank and similar projects as well - especially if they already use BDW.
reply
gaze
19 hours ago
[-]
Have you considered MPS? https://github.com/Ravenbrook/mps .
reply
Jeaye
15 hours ago
[-]
Yep! MPS was considered, but Boehm was significantly easier to get going. For jank's next GC, the plan is to use MMTK.
reply
refset
18 hours ago
[-]
I'm just an enthusiastic bystander so I don't know the answer. Hopefully Jeaye can chime in.
reply
sorrythanks
1 day ago
[-]
This is a wonderful project, I'm very excited for it. I look forward to reading about the WebAssembly plan
reply
erichocean
19 hours ago
[-]
Maybe this has has already been covered, by I would not target LLVM IR in 2024.

I'd target MLIR (like Mojo does).

1. It's a much easier/better target to work with.

2. It's a strict super-set of LLVM IR.

3. Much better optimizations are possible that are specific to your language.

Separately, I'd love to have a Clojure-friendly interface to MLIR—whether via Jank or something else.

reply
Jeaye
15 hours ago
[-]
This is a great suggestion. I've been doing some research on Mojo, but didn't realize MLIR was the recommended approach going forward. I'll look more into this. Codegen is pretty straightforward, from jank's AST, just using LLVM's IRBuilder, so I'd expect migrating to a superset shouldn't be an issue.
reply
pjmlp
1 day ago
[-]
Great that it is already making use of C++20 modules.
reply
amano-kenji
18 hours ago
[-]
How useable is jank already? Is it production ready?
reply
Jeaye
15 hours ago
[-]
Not production ready. Not very usable right now. I need several more months to improve the tooling, error handling, and general compilation functionality. Stay tuned!
reply
rads
1 day ago
[-]
Awesome work Jeaye!
reply
gdsdfe
17 hours ago
[-]
does this mean there's a path from clojure to wasm ?! and the wasm component model ?
reply
Jeaye
15 hours ago
[-]
Yes! Since jank is using LLVM, AOT compiled jank programs have a path to WASM. This isn't a primary goal for jank, but it's one we can definitely pursue once jank's Clojure parity and general dev UX has been established.
reply
eikenberry
1 day ago
[-]
Nice to see a language take developer UX seriously and focus on good compilation times. I wish this was more common.
reply
refset
23 hours ago
[-]
Fast compilation is undoubtedly more essential in the Lisp paradigm of REPL-driven, interactive programming with S-expressions than in other languages. A pre-requisite perhaps for the world to 'Stop Writing Dead Programs' [0]

[0] https://jackrusher.com/strange-loop-2022/

reply