HN Theater @HNTheaterMonth

The best talks and videos of Hacker News.

Hacker News Comments on
Differential Synchronization

Google TechTalks · Youtube · 7 HN points · 5 HN comments
HN Theater has aggregated all Hacker News stories and comments that mention Google TechTalks's video "Differential Synchronization".
Youtube Summary
Google Tech Talks
January 8, 2009

ABSTRACT

Keeping two or more copies of the same document synchronized with each other in real-time is a complex challenge. This talk describes an algorithm which is robust, convergent, and efficient. Differential synchronization is the algorithm powering MobWrite.


Speaker: Neil Fraser
Neil is the developer of MobWrite, a real-time collaborative web-based editor.
HN Theater Rankings

Hacker News Stories and Comments

All the comments and stories posted to Hacker News that reference this video.
Sep 28, 2020 · mhale on CRDTs are the future
I'm working on a project with some offline data synchronization needs, but haven't started implementation yet. I've been following CRDTs with interest. I also saw many of the same downsides mentioned in the OP, e.g. bloat (which apparently are being addressed remarkably well). Beyond OT, another approach I've run across that looks very promising is Differential Synchronization[1] by Neil Fraser. While it also relies on a centralized server, it allows for servers to be chained in such a way that seems to address many of the downsides of OT. I wonder why I rarely ever see Differential Synchronization mentioned here on HN? Is it due to lack of awareness or because of use-case fit issues or some fatal flaw I haven't seen? Or something else?

[1] https://www.youtube.com/watch?v=S2Hp_1jqpY8

archagon
I devoted some time to Differential Synchronization in my article on CRDTs, if you're interested: http://archagon.net/blog/2018/03/24/data-laced-with-history/...

My sense is that it's not a particularly flexible approach, and falters under adverse network conditions, even though it feels very tempting.

Differential synchronization does not require a fixed polling interval. You can have real time updates as each party types, in fact the algorithm prefers this setup as it is easier to process a lot of small changes rapidly. Neil goes into further detail in this google talk about why this is the case: https://youtu.be/S2Hp_1jqpY8?t=2815. The trick is sync after someone changes the document while including a short debounce so that you aren't overloading the system with superfluous edits.
toomim
Thanks! Good point!
I've not tried it, but you might also be interested in an approach called differential synchronization[1]. Basically, it's just a specific way of passing diffs around to guarantee convergence, though (as far as I can tell) it doesn't guarantee much else (as in do you lose data if a patch can't be applied?). The really nice bits, though, are that it is a simple algorithm and that it can be applied to anything that you have diff/patch algorithms for.

[1] https://www.youtube.com/watch?v=S2Hp_1jqpY8

aidos
I already watched that video the other day! Didn't quite get to the end of it but I thought the general idea was interesting.

My use case is a little different, and one that OT seems to suit.

I have multiple users working on the same documents but they don't want/need to see each other's changes straight away. They perform a number of operations and then eventually save. When they save I'd like to propagate that that history to the other user(s) that haven't saved yet. It feels like OT works there - you have two threads of history and you need to rebase the uncommitted one on top of committed one.

Maybe I'm coming at the problem wrong, but it feels like something that could work quite well as a pattern in general.

One real-world use case is differential synchronization. Here's a Google Tech Talk on the subject:

https://www.youtube.com/watch?v=S2Hp_1jqpY8

This is for real-time collaborative editing or replication of structured data (unlike Operational Transformation, which is optimized for text). Sending patches back and forth is at the core of the protocol.

Jan 12, 2009 · 7 points, 1 comments · submitted by babyshake
RiderOfGiraffes
Algorithm actually explained here:

    http://neil.fraser.name/writing/sync/
More about the theory can be found here:

    http://code.google.com/p/google-mobwrite/wiki/Theory
The talk is also here:

    http://neil.fraser.name/news/2009/01/11/
but has links to other stuff by the author. Recommended over the youtube link.

It would be interesting to run a real comparison between this, etherpad (http://www.etherpad.com) and CollabEdit (http://collabedit.com). I suspect that this is a problem everyone is trying to solve. Etherpad's extremely good, I've used it "in anger", but no doubt Google will win because it's Google.

I'm not yet sure whether the data in MobWrite is "public", but it is in EtherPad (if you can guess the URL).

HN Theater is an independent project and is not operated by Y Combinator or any of the video hosting platforms linked to on this site.
~ yaj@
;laksdfhjdhksalkfj more things
yahnd.com ~ Privacy Policy ~
Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.