HN Theater @HNTheaterMonth

The best talks and videos of Hacker News.

Hacker News Comments on
"Transactions: myths, surprises and opportunities" by Martin Kleppmann

Strange Loop · Youtube · 33 HN points · 3 HN comments
HN Theater has aggregated all Hacker News stories and comments that mention Strange Loop's video ""Transactions: myths, surprises and opportunities" by Martin Kleppmann".
Youtube Summary
Back in the 1970s, the earliest databases had transactions. Then NoSQL abolished them. And now, perhaps, they are making a comeback... but reinvented.

The purpose of transactions is to make application code simpler, by reducing the amount of failure handling you need to do yourself. However, they have also gained a reputation for being slow and unscalable. With the traditional implementation of serializability (2-phase locking), that reputation was somewhat deserved.

In the last few years, there has been a resurgence of interest in transaction algorithms that perform well and scale well. This talk answers some of the biggest questions about the bright new landscape of transactions:

What does ACID actually mean? What race conditions can you get with weak isolation (such as "read committed" and "repeatable read"), and how does this affect your application?
What are the strongest guarantees we can achieve, while maintaining high availability and high performance at scale?
How do the new generation of algorithms for distributed, highly-available transactions work?
Linearizability, session guarantees, "consistency" and the much-misunderstood CAP theorem -- what's really going on here?
When you move beyond a single database, e.g. doing stream processing, what are your options for maintaining transactional guarantees?

Martin Kleppmann
@martinkl

Martin Kleppmann is a software engineer and entrepreneur, and author of the O'Reilly book Designing Data-Intensive Applications (http://dataintensive.net), which analyses the data infrastructure and architecture used by internet companies. He previously co-founded a startup, Rapportive, which was acquired by LinkedIn in 2012. He is a committer on Apache Samza, and his technical blog is at http://martin.kleppmann.com.
HN Theater Rankings

Hacker News Stories and Comments

All the comments and stories posted to Hacker News that reference this video.
Dec 08, 2020 · vvern on Kafka Is Not a Database
I mean, I hear you that people do that and it does allow you to avoid needing distributed transactions. When you're stuck with an underlying NoSQL database without transactions, this is the best thing you can do. This is the whole "invert the database" idea that ends up with [1].

I'm arguing that that usage pattern was painful and that if you can have horizontally scalable transaction processing and a stream of events due to committed transactions downstream, you can use that to power correct and easy to reason about materializations as well as asynchronous task processing.

Transact to change state. React to state changes. Prosper.

[1] https://youtu.be/5ZjhNTM8XU8?t=2075

It's great. Incredibly dense with useful information and it just blows my mind how much knowledge Martin has about the topic. I recommend watching this talk from him to give a little glimpse of the book: https://www.youtube.com/watch?v=5ZjhNTM8XU8 This is just about a little part of one of the chapters.
mirekrusin
Oh man, this is good, thank you.
Rich Hickey - "simple made easy" https://youtu.be/rI8tNMsozo0

Rob Pike - "concurrency is not parallelism" https://youtu.be/cN_DpYBzKso

Uncle Bob Martin - "future of programming" https://youtu.be/ecIWPzGEbFc

Martin Kleppmann - "transactions: myths, surprises and opportunities" https://youtu.be/5ZjhNTM8XU8

Simon Brown - "software architecture vs code" https://youtu.be/GAFZcYlO5S0

sparq_beam
That transaction talk is really good, thank you. I can now finally name the effect that I had noticed but had trouble explaining and referring to, write skews.

It's interesting that he doesn't mention phantom reads as the difference between repeatable read/snapshot isolation, and serializable, which is what other sources tend to do.

Snapshot isolation always seemed to me like cheating the intended meaning of repeatable read, insofar as some databases refer to their snapshot isolation level as repeatable read.

That is, in the strictest sense, if you read a row twice, you get the same value with snapshot isolation, but you don't actually know that the value will be the same when you commit, which as I understand is a case of a write skew.

In fact, if one thinks of the definition of these levels in terms of locking semantics, one would expect a repeatable read to have the same meaning as obtaining a read lock on the row you read, which I understand would prevent at least some types of write skew, since no modification would be possible on that row, because it would need a write lock. There could still be hazards related to phantom reads (and possibly other effects), such as making a decision based on a computed aggregate that can change if new rows are inserted. Still, this meaning of repeatable reads would already provide a useful isolation level for various cases, except that it doesn't work with snapshot isolation.

I have a suspicion that applications out there made incorrect assumptions as to the actual isolation provided by the DB they use.

Sep 28, 2015 · 33 points, 1 comments · submitted by adamnemecek
jzelinskie
This talk is super relevant to anybody writing a traditional monolithic web app. Eventually bugs creep in and you learn that your initial assumptions about transactions are basically all wrong. I really enjoyed that the talk extended to micro-services -- it gave a lot of insight into what you take for granted in your monolithic architecture when you start breaking things down.

One tiny improvement I think could benefit the talk would be if the speaker commented about the usage of "SELECT FOR UPDATE" and maybe gap-locking.

HN Theater is an independent project and is not operated by Y Combinator or any of the video hosting platforms linked to on this site.
~ yaj@
;laksdfhjdhksalkfj more things
yahnd.com ~ Privacy Policy ~
Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.