Clojure Async Flow Guide
202 points
18 hours ago
| 15 comments
| clojure.github.io
| HN
tomconnors
5 hours ago
[-]
I recently used c.a.flow for a program that reads files from s3, massages the data a bit, and writes to a database. The code came out very easy to understand. The challenges were:

- getting back pressure right. You need to tune the buffers for the input chans correctly, which takes some thinking, and in my case, some blowing up in prd.

- flow monitor (the UI for debugging flows) can't handle large process states, which makes it pretty much useless for my program.

- understanding the flow as a whole (rather than specific processes, which are easy to understand in isolation). For example, answering questions like "why hasn't this process received a message for a while?" was tricky.

reply
puredanger
5 hours ago
[-]
Thanks for the feedback! Flow monitor does now support filters on the process state (and more on that it is coming to flow itself soon). If you were able to use monitor, it shows the channel buffer states, I guess that was not sufficient to guess why values weren't flowing?
reply
kaliszad
6 hours ago
[-]
Clojure(Script) is a mature, boring in the good sense language and ecosystem. It opened a whole subculture and dialects/ implementations like ClojureCLR, Babashka, nbb, Jank, Janet, Fennel, Joker, Basilisp, Hy, Clojerl and at least a few others I forgot to mention that can perhaps be found on this list: https://github.com/chr15m/awesome-clojure-likes

We are building apps for our clients in it, we will also have our own product built with Clojure and ClojureScript soon.

reply
robto
17 hours ago
[-]
I've been meaning to try this out, from my read it's a declarative way to get some structured concurrency. I work in a codebase that heavily uses core.async channels to manage concurrency and you really need to pay close attention to error handling. When you're spawning new threads you need to set up your own custom machinery to re-throw errors on a chans, close chans, and it looks like core.async.flow is a way to do all of this declaratively.

Just like `core.async` itself was a frontrunner of Virtual Threads on the JVM, I view `core.async.flow` as the Clojure version of the upcoming [0]Structured Concurrency JEP. I do wonder if it will use that under the hood once it becomes stable, the same way `core.async` is planning to move away from the `go` macro to just have it dispatch a virtual thread.

[0]https://openjdk.org/jeps/453

reply
puredanger
7 hours ago
[-]
I don't think it would be feasible or wise to structure core.async to use Structured Concurrency, although Structured Concurrency is trying to tackle some of the same problems as flow but in a different way (more akin to data flow style concurrency).
reply
xmcqdpt2
7 hours ago
[-]
We've been looking at virtual threads in a project at work and what we found is that it is quite difficult to adapt existing code to run with virtual threads.

For example, class initialization pins a thread so any singleton defined in the standard, recommended Java way (using a static inner instance of an inner class) can hang your program forever if the singleton definition suspends. And because they worked really hard on making async colourless, there is no way to know that a method call will suspend. This is a known issue with a workaround if the suspend is due to a network call,

https://bugs.openjdk.org/browse/JDK-8355036

which is useful for some applications (web servers). Figuring out that this is why my program was hanging required quite a bit of work too. We are still frustratingly far from the ergonomics of Go concurrency (all threads are virtual threads, hangs automatically panic).

reply
puredanger
7 hours ago
[-]
Clojure's focus on immutable data and pure functions side-step a lot of the trickiest issues with virtual threads. It's often not hard to isolate the I/O parts of your program into flow processes at the edges that can be mapped to the :io pool using virtual threads.
reply
gf000
7 hours ago
[-]
> can hang your program forever if the singleton definition suspends

I am no expert on the topic, but this seems like a very edge case scenario, that is not trivial to reproduce on even the linked bug ticket. Do you think it really is that big of an issue?

reply
judge123
16 hours ago
[-]
For me, the real 'click' moment with core.async wasn't just about replacing callbacks. It was realizing I could model entire business processes as a system of communicating channels. It completely changed how I think about system design.
reply
blackbear_
11 hours ago
[-]
Curious if you have found any resources to learn more about this way of thinking?
reply
puredanger
7 hours ago
[-]
Rich Hickey actually did a talk called "Language of the System" https://www.youtube.com/watch?v=ROor6_NGIWU way back in 2013 before core.async was even created that lays out a lot of the ideas. It even has a big section explicitly about "flow" which contains the germs of core.async.flow.
reply
wvdlaan
10 hours ago
[-]
Perhaps this is a good place to start reading: https://en.wikipedia.org/wiki/Communicating_sequential_proce...
reply
akkad33
10 hours ago
[-]
Is it like elixir actors
reply
lgrapenthin
9 hours ago
[-]
No. Those have unbounded message queues.
reply
akkad33
2 hours ago
[-]
Thanks. Is that the only difference
reply
725686
18 hours ago
[-]
Is Clojure still a thing? I sure would hope so, but I haven't seen much of Clojure activity in HN recently.
reply
aeonik
18 hours ago
[-]
The language itself is still getting updates, a new major release was just dropped a month or two ago.

I do find that for about 5 years things seemed to be slowing down. Though I keep seeing it pop up, and new exciting projects seem to pop up from time to time.

Just today I saw an article about Dyna3, a relational programming language for AI and ML that was implemented on top of Clojure.

I miss the Strange Loop conference. I think a lot of Clojure buzz was generated there. Clojure West and a few others so a decent job, but the quality of the talks at Strange Loop were second to none. Not that it was a Clojure specific conference, but it had that focus on elegance that I don't see very often, and the organizer was a something like the Prince of Clojure, if I recall correctly.

I'm still enjoying the language, and all my projects still build and run just fine.

The major frustration I have with the platform is 3D graphics. That's a JVM issue overall though.

reply
raspasov
16 hours ago
[-]
I just saw a small 3D demo running at 120fps+ that some of the newer JVM vector APIs supposedly enable.

Link to demo @ timestamp: https://youtu.be/UVsevEdYSwI?t=653

My experience with 3D graphics is minimal, but I'm curious to know if these newer developments are significant in any way for 3D work.

reply
pjmlp
10 hours ago
[-]
To be fair, that is only so much that a Lisp can have as foundation beyond the core forms and macros, especially when it doesn't control the runtime.

Cursive, Calva and CIDER are already quite good.

After that, it is all about the ecosystem, what libraries people care to build.

reply
whizzter
9 hours ago
[-]
This is for games? Did you try evaluating Jank that seems to be a LLVM based "native" variant?
reply
aeonik
9 hours ago
[-]
Data visualization and simulation, but games are on the table too.

I'm eagerly awaiting Jank to stabilize.

reply
725686
16 hours ago
[-]
I absolutely loved Hickey's talks even when I never used Clojure more than for a few simple examples.
reply
dapperdrake
16 hours ago
[-]
They even invited Guy Lewis Steele, Jr. hos talk is on YouTube and was awesome. His meta-notation is explained more expansively in a paper on his Oracle page.
reply
ethersteeds
17 hours ago
[-]
As others have said, Clojure is still a thing. For anyone catching up with Clojure again after some time: check out Babashka! Think bash scripts, written in Clojure. It's delightful.

https://babashka.org/

reply
precompute
9 hours ago
[-]
Babashka is the best scripting environment out there. You just need that one executable and it runs flawlessly. The bundled libs are also very useful.
reply
tombert
17 hours ago
[-]
I still use it. They finally fixed my biggest complaint about it a year ago, which is that you couldn't use vanilla Clojure lambdas for the Java functional interface, and so you'd have to reify that interface and it was bulky and ugly. Now it works fine so long as the interfaces actually have the @FunctionalInterface attribute.

Not every project uses @FunctionalInterface, but I've been trying to add it to places [1] [2] [3], and now I'm able to use Clojure in a lot more places.

[1] https://github.com/LMAX-Exchange/disruptor/pull/492

[2] https://github.com/apache/kafka/pull/19234

[3] https://github.com/apache/kafka/pull/19366

reply
ndr
10 hours ago
[-]
I sense Clojure attracts a bunch of people that prefers building stuff than talking about building stuff.

It's bad for marketing, but seems promising for the project longevity.

I thought Rich was being a bit pretentious when he compared to violin, but few people ask "are violins still a thing?"

reply
chamomeal
17 hours ago
[-]
I’d say clojure is very alive and happy. I’m a clojure newb and have been having a super fun time getting into it. Lots of very neat tools are in active development (babashka is the best thing that’s happened to my developer life in a while!!)

The small-medium sized community is actually fantastic for learning. The big names in the community are only a slack away, and everybody is so enthusiastic.

reply
casion
17 hours ago
[-]
There's more clojure users than ever before and the team is active and afaik larger than ever before.

Things just mature and hype isn't as cool when you heard it 5 years ago.

reply
chii
13 hours ago
[-]
> Things just mature and hype isn't as cool when you heard it 5 years ago.

which is why now is exactly the right time to start using clojure - after the hype died, but have active community and users.

reply
chr15m
16 hours ago
[-]
Is Make still a thing? I sure would hope so, but I haven't seen much of Make activity in HN recently.
reply
725686
2 hours ago
[-]
I don't know. Is it? I haven't touched one in 20 years.
reply
doubleg
5 hours ago
[-]
Not surprised as Clojure is boring tech: slowly evolving with a big focus on stability (ie backwards compatibility).

Meanwhile: the core team has been extended the last couple of years. Also this summer NuBank (the company behind Clojure) announced the first 'Clojure Developer Advocate'. Their role will be to "focus on ways to support the existing Clojure community and grow the community through outreach and development."^1

[1]: https://building.nubank.com/clojure-developer-advocate-nuban...

EDIT: wording.

reply
xanth
17 hours ago
[-]
I was asking the same question today after investigating XTDB¹ (a Clojure centric bitemporal DB) and went looking for a batteries included WebAssembly framework like Blazor²

1. https://xtdb.com/

2. https://dotnet.microsoft.com/en-us/apps/aspnet/web-apps/blaz...

reply
didibus
16 hours ago
[-]
I'm not sure, but maybe you can use Blazor with ClojureCLR. It's a feature complete Clojure for .net
reply
giancarlostoro
7 hours ago
[-]
A lot of the Clojure editors I liked that were made specifically for Clojure seemed to have died down too. It's a shame, they were cool and unique.

Between Clojure and Racket, those have always been my two favorite Lisp / Scheme languages. I don't do a lot of Lisp but when I do its either in Clojure (thanks to Lein) or in Racket.

reply
raspasov
5 hours ago
[-]
Cursive, Calva, and CIDER are all excellent and very well-supported options. What are you missing specifically?
reply
shaunxcode
18 hours ago
[-]
:yes #{of course it is}
reply
raspasov
16 hours ago
[-]
reply
KingMob
15 hours ago
[-]
It is, but the community has been shrinking in recent years.

FWIW, Google Trends shows the hype peaking in 2016, but I doubt that reflects usage as much as buzz.

Instead, if you look at the annual State of Clojure survey results, which solicits opinions directly from the community, the number of responders peaked in 2020 at ~2500, and is down to ~1500 for the most recent 2024 survey.

- 2020 State of Clojure - https://www.surveymonkey.com/results/SM-CDBF7CYT7/

- 2024 State of Clojure - https://www.surveymonkey.com/results/SM-hht04mGydwZ6Nqr7N8vj...

reply
puredanger
8 hours ago
[-]
The absolute number of survey respondents is not a good proxy for community size - the survey runs at different times of the year, for different lengths of time, and with different amounts of marketing. The only goal with the survey is to get a representative sample size. We have other sources of data, both public and private, that are better indicators and indicate the community size is likely growing at this time.
reply
dustingetz
3 hours ago
[-]
Hi Alex, if you have data that supports a positive Clojure growth narrative please publish it so that I and other consultants/influencers can share the good news in support of our shared mission. The perception of the decline of Clojure is becoming a board-level conversation at unicorn/ish size companies that are or were all-on on Clojure.
reply
adityaathalye
13 hours ago
[-]
A better source: 2024 survey analysis (and results): https://clojure.org/news/2024/12/02/state-of-clojure-2024

  - 2024 Highlights
  - Trends Over Time
  - 2024 New Users
  - Previous Results
Now... If we are pointing out isolated facts to make an argument, I would caution that survey popularity (sensitive to timing, duration, outreach etc.) is less telling---and less statistically significant---than isolated facts like this:

> Clojure versions

> Clojure 1.12.0 was released in September 2024 and the survey showed rapid uptake, with 58% already using it, and 65% developing or deploying with the prior versions 1.11, and a steep drop-off after that. Clojure’s focus on stability and avoiding breaking changes makes upgrades safe and easy.

> Trends (use at work, hobby, and study have all up-trended)

> https://clojure.org/news/2024/12/02/state-of-clojure-2024#tr...

> Because this survey has been running since 2010 (thanks to Chas Emerick originally!), we have lots of great longitudinal data and it’s interesting to compare some of the answers over time.

> Looking at the question of how Clojure developers use Clojure, we can see this has generally trended more towards using it at work. However, this year we saw an uptick of people using it for hobbies or in their studies:

reply
KingMob
10 hours ago
[-]
Everything you quoted is based on percentages of the responders, not absolute numbers. Changing in-group proportions don't say anything about overall usage. E.g., if responder work usage goes up 10%, but 40% fewer people use Clojure, that's still a drop in absolute numbers.

Look for the number of responses, and you can see a decline each year after 2020.

---

It's possible that the survey may not have been advertised as well, but afaik, it's still posted the same way it always was: announcements on Clojurians, Clojureverse, reddit, etc. I haven't heard of any reason that survey numbers would have been artificially depressed for several years running.

reply
adityaathalye
9 hours ago
[-]
Absolute survey responses are a signal, I don't deny that. But they aren't enough to make the generalisation you are making.
reply
KingMob
8 hours ago
[-]
Fair, but I also mentioned Google Trends.

Or, I picked a random, reasonably popular library to check on Clojars: http-kit. The most recent stable release, 2.8.0, which came out last year, has only been downloaded ~600k times. 2.7.0 from 2023 was downloaded ~1.4m times. 2.6.0 from 2022 was dled ~2m times. Ditto for 2.5.3 from 2021.

I would have used Clojure itself, but I can't find maven dl statistics.

https://clojars.org/http-kit/versions/2.8.0 https://clojars.org/http-kit/versions/2.7.0 https://clojars.org/http-kit/versions/2.6.0 https://clojars.org/http-kit/versions/2.5.3

---

The thing is, I've been seeing little pieces of evidence all over that Clojure is waning, and not much that it's genuinely increasing in popularity. Any individual example doesn't weigh that much, true, but everything seems in the same direction.

If people want Clojure to grow, whether because they need job opportunities, a big employee pool, whatever, it starts with a clear assessment of where it's at.

reply
raspasov
5 hours ago
[-]
All of those other things you listed, while important, are second and third-order side effects that are harder to control directly.

I need a tool that helps with problem-solving and product development, and works reliably and effectively across a wide range of use cases, from basic mobile apps to high-performance computing.

Clojure delivers that better than any other language or ecosystem that I know of in a uniform, well-designed package, all the way from the core internals of the language to deps.edn.

reply
puredanger
7 hours ago
[-]
Note that newer things are always downloaded less because they have been around less time (lots of people continue using old versions).

Maven stats are available to artifact deployers, but they are useless for estimating users or community size as downloads are largely from CI servers constantly downloading artifacts for testing. Download numbers are large and seesaw erratically. Unique IP counts are a little more stable but also inflated beyond relevance by CI.

reply
user3939382
17 hours ago
[-]
I think LISP is cool and want to use it more but I have 0 appetite to learn the toolchain and debug etc for JVM. You have Racket but Clojure ecosystem is already tiny.
reply
raspasov
17 hours ago
[-]
That’s a common misconception. JVM toolchain is way better than the hellscape that are most other language ecosystems. Maven, for example, works reliably and is rock-solid. The only unsolved problem is if you get two libraries/frameworks requesting another dependency, but with different incompatible versions. But I don’t think most other ecosystems solve that painlessly either.

Debug specifically is state-of-the-art. Look at YourKit, or any debugger included with common IDEs.

None of those tools has a shiny Visual 2025 aesthetic, but again, they work reliably and are going to work the same way a year from now.

reply
pjmlp
10 hours ago
[-]
Indeed, JVM and .NET are the only two major ecosystems out the whole Xerox PARC ideas[0], and the reason that to this day they are my two main workhorses, plus JS runtimes, because Web.

[0] - Technincally Objective-C and Swift could also be considered, but they lack the industry wide adoption, as many cool tools only exist in Apple land.

reply
raspasov
5 hours ago
[-]
I agree.

Swift is quite good. I assume the tooling is catching up to Objective-C. It has been a couple of years since I had to solve something with Swift.

Any reason, apart from outside requirements, to pick the .NET CLR over the JVM as a runtime in your experience? Outside of library/framework support, which I've heard is good with the CLR.

How's the garbage collector with the CLR?

reply
jiehong
5 hours ago
[-]
Today the CLR has better SIMD and struct alignment than the JVM.
reply
dapperdrake
16 hours ago
[-]
Clojure taught me lisp where CL failed. Turns out that Scheme and Clojure as a lisp-1 are great for learning.

Switched to SBCL for the faster star-up times. Now lisp-2 also feels more comfortable.

reply
kasajian
13 hours ago
[-]
Since you mentioned SBCL, I thought this announcement about ECL and WebAssembly would be worth mentioning for those who are curious. It was announced just last month: https://ecl.common-lisp.dev/posts/Web-ECL-Grant-Announcement... (2025-07-29)
reply
tombert
17 hours ago
[-]
Leiningen and deps.edn shield you a bit from the awfulness of Java project management. They feel a lot more like something you'd see in Node.js or something, but it still gets dependencies from Maven Central.

Debugging and profiling is still somewhat Java based, and yeah that's can be irritating, but you get used to it.

Personally I do think that it's worth it; Clojure is a very pleasant language that has no business being as fast as it is, and core.async is an absolutely lovely concurrency framework if you can convert your logic into messaging, and you have Haskell-style transactional memory for stuff that can't be. So many problems become less irritating in Clojure.

reply
dapperdrake
16 hours ago
[-]
Is clj-boot still a thing or was it ever a thing?

It or or was a build tool like Leiningen.

reply
Volundr
15 hours ago
[-]
Boot seems to have pretty much stalled out. I think the builtin Clojure CLI/deps.edn killed off what momentum it did have.
reply
tombert
16 hours ago
[-]
I've never used clj-boot. I've historically mostly used Leiningen but for the last year or so I finally migrated over to deps.edn.
reply
nromiun
16 hours ago
[-]
This is cool. So basically you create a group of threads and you can treat them as one unit. Trio does something similar (structured concurrency) with async functions for Python. Are these OS threads or green threads?
reply
didibus
16 hours ago
[-]
They can be OS threads or green threads, you can choose.
reply
nromiun
15 hours ago
[-]
Are OS threads still the default and you have to pass a virtual thread executor to use virtual threads?
reply
puredanger
7 hours ago
[-]
When you deploy a flow, you choose the workload type (:compute, :io, or :mixed) and the process will be deployed with the appropriate thread type.
reply
stbev
13 hours ago
[-]
This is really interesting, although I still can't get my head around the fact that core.async.flow topologies are immutable. I feel like most problems can't be solved with fixed topologies.

I guess one could in theory swap flows the same way values are swapped, but I wonder if this is the way this library is supposed to be used. I also wonder what happens to non-empty channel buffers in this case.

I am curious to hear other opinions.

reply
puredanger
7 hours ago
[-]
Flow is intended for processes with long-running stable topologies. Rich has been thinking about options to "patch" the running topology but it is quite tricky due to the concurrency issues and I'm not sure that will ever be added.

Even though the flow topology is fixed, it's perfectly acceptable for a flow component to use other variable resources and act merely as a coordinator. So you could for example have a process that send data out to an external dynamic thread pool and gets callbacks via a channel.

reply
pgorczak
7 hours ago
[-]
This reminded me of Elixir’s GenStage at first glance. Now I wonder how the underlying libs OTP and core.async relate conceptually and implementation wise.
reply
kasajian
16 hours ago
[-]
Funny. I was just thinking about dismissing Clojure for a project I'm going to work on because I was concerned about it's lack of ability to work with async calls. I'm too used to how async in JavaScript and C#, and I'm not sure I'd want to work in an environment that doesn't have a simple way to structure async calls. It doesn't necessarily have to be async / await. Just some attention to the issue rather than completely ignoring it.
reply
mechanicum
8 hours ago
[-]
I’m curious what led to that conclusion. As far as I remember, making concurrency easier to manage was always presented as one of Clojure’s primary objectives. It’s fundamental to the design e.g. a major motivation for all core data structures being immutable.

STM, atoms and agents were there from the beginning. I think futures and promises were added in 1.1. core.async is from 2013. Even popular third-party libraries like promesa and manifold are around 10 years old at this point.

I think flow promises to make it easier to orchestrate core.async, specifically, in complex applications, but the essential primitives are far from new and I don’t consider them any harder to use than JavaScript.

reply
raspasov
15 hours ago
[-]
That has always been one of Clojure's main strengths (async & concurrency). With the new JVM VirtualThreads, things are looking better than ever.

The transition of core.async specifically to VirtualThreads is still WIP as far as I understand, but with minimal tweaks, 90% of the benefits are already there, even with the current latest version.

reply
puredanger
7 hours ago
[-]
Virtual thread support in core.async is imminent, should land any day now.
reply
raspasov
6 hours ago
[-]
woot
reply
didibus
16 hours ago
[-]
reply
beders
15 hours ago
[-]
You can get basically all variations of async coding with Clojure a la carte.
reply
vim-guru
12 hours ago
[-]
So, kind of the opposite of https://github.com/lovrosdu/klor but with the same goal
reply
raspasov
10 hours ago
[-]
I am not sure if this is “opposite” but on the surface looks very interesting!
reply
yayitswei
16 hours ago
[-]
See also for related ideas: missionary/electric for frontend and rama for backend. I wish for a unified interface combining the best of all three!
reply
jibal
14 hours ago
[-]
typos (from a quick surface scan; I'm not familiar with Clojure or this package):

"The description arity takes the current state ..." should be "The transition arity takes the current state ..."

"excepiton" should be "exception"

"Exceptions thrown from the transition or transform arities, the exception ..." should be "If exceptions are thrown from the transition or transform arities, they ..."

"provded" should be "provided"

reply
puredanger
7 hours ago
[-]
Fixed, thx.
reply
Abdii430
17 hours ago
[-]
reply
whacked_new
13 hours ago
[-]
so is this something like core's interpretation of things like ztellman's manifold (https://github.com/clj-commons/manifold)?

EDIT:

gpt says,

Choose core.async.flow when you want a declarative, monitorable process graph over channels (pause/resume, error channel, flow monitor).

Choose Manifold when you need deferreds/streams as general primitives, precise per-op backpressure, error-aware chaining, or you’re in the Aleph/Netty stack.

very abstract but based on this, the answer is no.

reply