I've stated before that I think the main thing holding back collaborative text / sequence CRDTs is integration with a production database.
Eg-walker looks interesting because it might lend itself to be integrated into a database because the operations are immutable and only appended. However, to demonstrate the effectiveness of these algorithms library authors (see Yjs, DiamondTypes, etc) build stand-alone data structures (usually specialized search trees) that most databases already provide.
Personally, I've been trying to adapt a Piece Table[1] to be collaborative and stored in Triplit[2] which runs on both client and server and already implements logical clocks but I might see how well I can adapt this algorithm instead!
1. https://en.wikipedia.org/wiki/Piece_table 2. https://github.com/aspen-cloud/triplit
I can see this completely reshaping the landscape of what's possible with collaborative documents!
Egwalker has one other advantage here: the data format will be stable and consistent. With CRDTs, every different crdt algorithm (Yjs, automerge/rga, fugue, etc) actually stores different fields on disk. So if someone figure out a new way to make text editing work better, we need to rip up our file formats and network protocols.
Egwalker just stores the editing events in their original form. (Eg insert “a” at position 100). It uses a crdt implementation in memory to merge concurrent changes (and everyone needs to use the same crdt algorithm for convergence). But the network protocol and file format is stable no matter what algorithm you use.
I use ShareDB every day, which originated from Seph's excellent work on OT algorithms. Good stuff!
It's great work, combining the best of OT and CRDTs.
It’s a crdt in that all peers share & replicate the set of all editing events. (A grow-only set crdt if we’re being precise). Peers can use those editing events to generate the document state at any point in time, merge changes and so on.
But the editing events themselves are stored and expressed in their “original” form (unlike existing CRDTs, which need a prepare function). That means lower memory usage during use.
The replying / merging process itself is kind of a batch operational transform algorithm. It works by building a normal crdt state object in memory in order to transform the events so they can be replayed. In that sense, it’s an OT system. (But one which transforms by using a crdt, like Yjs, internally within each peer).
I don’t know if that clarifies things. Feel free to ask more questions!
Figma for example https://www.figma.com/blog/how-figmas-multiplayer-technology...
I've seen CF Durable Objects used quite a lot.
There are other emerging patterns too: https://www.instantdb.com/
Text is hard because it’s a list of characters, and when items are inserted and deleted the operations change the index of all subsequent elements.
Usually, editing a digital whiteboard is much simpler.
> While the downside of OT is p2p, the one up side is that you get GIT like history that is super valuable for us especially if we want to build a CDC system.
How trivial would it be, to implement a CDC system from a CRDT. Does anyone know any github repos or any documentation I could refer to? Thanks
Is there a practical implementation yet that supports not just strings, but also lists and maps?
Would be great to see it integrated into yjs / y-crdt.
