Differentiable Fortran with LFortran and Enzyme
41 points
6 hours ago
| 2 comments
| docs.pasteurlabs.ai
| HN
new_user55
20 minutes ago
[-]
Really nice work. I love Enzyme, and used it in my project about differentiable atomic descriptors. Idea was that I can quickly gobble up existing C++ and fortran codes alike for atomic descriptors and create a encompassing library what differentiate against hyper-params as well! But at time Enzyme was very early ~0.0.50 version or so. In our observations also Enzyme was fast enough that performance wise it matched the analytical gradients (when embedded inside entire pipeline) ![libdescriptor](https://libdescriptor.readthedocs.io/en/latest/_images/E_F_C...) .
reply
dionhaefner
6 hours ago
[-]
Author here — I work on Tesseract at Pasteur Labs, and I wrote this up because the "what if this was possible" was bugging me for way too long :)

I was surprised by how well this worked, the LFortran + Enzyme stack seems to be a very clean way to get gradients through Fortran code via LLVM IR transformations. Pretty cool to see a 220-line Fortran heat solver turn into ~6,900-line reverse pass automatically if I dare say so.

Would be awesome to see this applied to a real scientific codebase, and I hope that the demo is enough to convince people that it’s worth trying.

reply
Gangway0829
2 hours ago
[-]
Very interesting stuff. How would I get GPU offload working? I have a rather complex scientific code I'm working on with JAX. Most of it can be expressed well with JAX's programming model, but the last 10% really sucks. It's still worth it so I don't have to mess around with offload onto whatever XPU flavor of the week. But going to C++ would really make my life easier, as long as I could use e.g. Kokkos.
reply
dionhaefner
2 hours ago
[-]
No idea lol. I assume it’s possible since both Enzyme and GPU programming are pervasive in Julia. Let us know if you end up trying.
reply
srean
4 hours ago
[-]
Very interesting. Does LFortran have the same internal array layout as the standard C runtime ?

A shared layout and a shared calling convention would be very nice.

Sorry about my naive question. Haven't touched Fortran directly in three decades I think.

EDIT: thanks for your reply. For some reason it has been flagged dead. So am responding here. You can mail dang hn at ycombinator dot co m about the flagging. He is very nice.

reply
kmaitreys
3 hours ago
[-]
I would also like to know this. Fortran itself is column-major, so I would guess the internal layout isn't same for multi-dimensional arrays when compared to row-major C? I'm not sure how LFortran represents arrays internally though.
reply
assemmedhat
2 hours ago
[-]
LFortran internally uses column-major, so interchanging data with C should be done carefully for multi-dimensional arrays. If row-major representation is highly needed feature, We can introduce a flag to do that. I'm not totally sure about that but it's doable under some conditions for sure.
reply
wombatpm
4 hours ago
[-]
Lots of scientific code in Fortran has sparse arrays, so a NxN array that only has values on 5 diagonals will store that as 5xN array to save memory allowing you to run a larger problem.
reply
srean
4 hours ago
[-]
That's a very orthogonal issue.

Sparse arrays are supported on C libraries too. I have done my time with CSC and CSR even inside Python that called out to C libraries.

reply
tanderson92
3 hours ago
[-]
When you say you 'wrote this up', you mean you had an AI write (at least) chunks of it.
reply
dionhaefner
3 hours ago
[-]
Indeed, just like I let my compiler write (at least) chunks of my AD logic. Not great when the tool becomes a leaky abstraction, but overall net positive don't you think?
reply
tanderson92
3 hours ago
[-]
Disagree; it irks me to read AI slop, and writing is meant to be persuasive and engaging to human readers.
reply
dionhaefner
3 hours ago
[-]
Happy to take the blame for the lack of persuasian and engagement then :) Thanks for the feedback, although I’d like to believe there’s a way to have that cake and eat it too.
reply