Lies I was told about collaborative editing, Part 2: Why we don't use Yjs
40 points
3 days ago
| 8 comments
| moment.dev
| HN
GermanJablo
21 hours ago
[-]
I remember reading Part 1 back in the day, and this is also an excellent article.

I’ve spent 3+ years fighting the same problems while building DocNode and DocSync, two libraries that do exactly what you describe.

DocSync is a client-server library that synchronizes documents of any type (Yjs, Loro, Automerge, DocNode) while guaranteeing that all clients apply operations in the same order. It’s a lot more than 40 lines because it handles many things beyond what’s described here. For example:

It’s local-first, which means you have to handle race conditions.

Multi-tab synchronization works via BroadcastChannel even offline, which is another source of race conditions that needs to be controlled.

DocNode is an alternative to Yjs, but with all the simplicity that comes from assuming a central server. No tombstones, no metadata, no vector clock diffing, supports move operations, etc.

I think you might find them interesting. Take a look at https://docukit.dev and let me know what you think.

reply
freekh
32 minutes ago
[-]
Cool! We also build client-server sync for our local-first CMS: https://github.com/valbuild/val Just as your docsync, it has to both guarantee order and sync to multiple types of servers (your own computer for local dev, cloud service in prod). Base format is rfc 6902 json patches. Read the spec sheet and it is very similar :)
reply
samlinnfer
1 hour ago
[-]
Just use OT like normal people, it’s been proven to work. No tombstones, no infinite storage requirements or forced “compaction”, fairly easy to debug, algorithm is moderate to complex but there are reference open source implementations to cross check against. You need a server for OT but you’re always going to have a server anyway, one extra websocket won’t hurt you. We regularly have 30-50k websockets connected at a time. CRDTs are a meme and are not for serious applications.
reply
antics
6 minutes ago
[-]
Author here, I did not specifically mention OT in the article, since our main focus was to help people understand the downsides of the currently-most-popular system, which is built on CRDTs.

BUT, since you mention it, I'll say a bit here. It sounds like you have your own experience, and we'd love to hear about that. But OUR experience was: (1) we found (contrary to popular belief) that OT actually does not require a centralized server, (2) we found it to be harder to implement OT exactly right vs CRDTs, and (3) we found many (though not all) of the problems that CRDTs have, are also problems in practice for OT—although in fairness to OT, we think the problems CRDTs have in general are vastly worse to the end-user experience.

If there's interest I'm happy to write a similar article entirely dedicated to OT. But, for (3), as intuition, we found a lot of the problems that both CRDTs and OT have seem to arise from a fundamental impedance mismatch between the in-memory representation of the state of a modern editor, and the representation that is actually synchronized. That is, when you apply an op (CRDT) or a transform (OT), you have to transform the change into a (to use ProseMirror as an example) valid `Transaction` on an `EditorState`. This is not always easy in either case, and to do it right you might have to think very hard about things like "how to preserve position mappings," and other parts of editor state that are crucial to (say) plugins that manage locations of comment marks or presence cursors.

With all of that said, OT is definitely much closer to what modern editors need, in my opinion at least. The less-well-known algorithm we ended up recommending here (which I will call "Marjin Collab", after its author) is essentially a very lightweight OT, without the "transformation" step.

reply
chrisweekly
49 minutes ago
[-]
"CRDTs are a meme and are not for serious applications."

That is one hot take!

reply
kaiwenwang
56 minutes ago
[-]
It appears Moment is producing "high-performance, collaborative, truly-offline-capable, fully-programmable document editor" - https://www.moment.dev/blog

There seems to be a conflict of interest with describing Yjs's performance, which basically does the same thing along with Automerge.

reply
antics
51 minutes ago
[-]
Author here. To be clear, we do not in ANY WAY compete with Yjs! We are a potential customer of Yjs. This article explains why we chose not to be a customer of Yjs, and why we don't think most people building real-time collaborative text editors should be, either.
reply
samwillis
22 minutes ago
[-]
It's disingenuous to suggest that "Yjs deletes and recreates the whole document on each keypress" and that this is "by design" of Yjs. This is a design limitation of the official y-Prosemirror bindings that are integrating two distinct (and complex) projects. The post is implying that this is a flaw in the core Yjs library and an issue with CRDTs as a whole. This is not the case.

It is very true that there are nuances you have to deal with when using CRDT toolkits like Yjs and Automerge - the merged state is "correct" as a structure, but may not match your scheme. You have to deal with that into your application (Prosemirror does this for you, if you want it, and can live with the invalid nodes being removed)

You can't have your cake and eat it with CRDTs, just as you can't with OT. Both come with compromises and complexities. Your job as a developer is to weigh them for the use case you are designing for.

One area in particular that I feel CRDTs may really shine is in agentic systems. The ability to fork+merge at will is incredibly important for async long running tasks. You can validate the state after an agent has worked, and then decide to merge to main or not. Long running forks are more complex to achieve with OT.

There is some good content in this post, but it's leaning a little too far towards drama creation for my tast.

reply
antics
48 minutes ago
[-]
Hi folks, author here. I thought this was dead! I'm here to answer questions if you have them.
reply
drpotato
36 minutes ago
[-]
Just wanted to say thanks! This is a great write up and resonates with issues I encountered when trying to productionise a yjs backed feature.
reply
bawolff
1 hour ago
[-]
Reminds me a bit of google-mobwrite. I wonder why that fell out of favour.
reply
presspot
2 days ago
[-]
Replacing CRDT with 40 lines of code. Amazing.
reply
truetraveller
1 hour ago
[-]
Very likely AI slop, very hard to read. Too many indications. HN should have another rule: explicitly mention if article was written (primarily) by AI.
reply
antics
55 minutes ago
[-]
I'm the author. Literally 0% of this was written with AI. Not an outline, not the arguments, not a single word in any paragraph. We agonized over every aspect of this article: the wording, the structure, and in particular, about whether we were being fair to Yjs. We moved the second and third section around constantly. About a dozen people reviewed it and gave feedback.

EDIT: I will say I'm not against AI writing tools or anything like that. But, for better or worse, that's just not what happened here.

reply
comex
48 minutes ago
[-]
It doesn’t strike me as AI. The writing is reasonably information-dense and specific, logically coherent, a bit emotional. Rarely overconfident or vague. If it is AI then there was a lot more human effort put into refining it than most AI writing I’ve read.
reply
utopiah
1 hour ago
[-]
Funnily enough I had 2 HN tabs open, this one and https://news.ycombinator.com/item?id=47394004
reply