HN Theater @HNTheaterMonth

This book is really good, really dense but the ideas have inspired a generation.

In this video, Sean Parent, at the time working at Adobe on Photoshop, estimated that if PS codebase followed ideas from this book, the codebase could be reduced from 3,000,000 LOC to 30,000 LOC.

https://www.youtube.com/watch?v=4moyKUHApq4&t=39m30s

⬐ agumonkey

so if you put Sean Parent and Alan Kay in a room, they'd reduce the sum of all programs running in the world into a single floppy

They make code reuse possible. Watch this Sean Parent video https://www.youtube.com/watch?v=4moyKUHApq4 where he estimates that if Photoshop was rewritten using generics, the code size would go from 3,000,000 LOC to 30,000 LOC. You are right, during compilation, generics are specialized so you end up with code, however going all in on generics removes a lot of accidental complexity.

⬐ prewett

Interesting talk. If he's right, then a Photoshop clone ought to be doable. 30k LOC is doable for a single person or a small team. I'm guessing that even with generics, it wouldn't help that much. Also, I think there are probably a lot more algorithm LOC in Photoshop than he thinks...

⬐ chc

I think you've misunderstood my point. What code reuse is possible with generics that is not not possible in Objective-C without generics? I don't think there is very much, because Objective-C fundamentally doesn't care the types of objects.

⬐ adamnemecek

The full answer would be kinda long but this is a rehash of the debate that has been had many times over.

You cannot just compare obj-c vs generics, you need to also account for the fact that when you are obj-c development, you might need some fast code and as a result drop to c. And code reuse is hard to impossible in C.

The fact that code reuse can be done without a performance hit is enough of a difference difference on it's own as it pushes down the level at which code reuse can be done. E.g. you can use generics for Graphics or Audio or general low level stuff.

⬐ millstone

Err, why not just drop to C++? Objective-C++ is real and works great. That's how Mac Chrome does things for example.

There's this book that I've been mentioning around here called Elements of Programming https://www.amazon.com/Elements-Programming-Alexander-Stepan... that makes exactly this claim, that we are writing too much code.

It proposes how to write C++-ish (it's an extremely minimal subset of C++ proper) code in a mathematical way that makes all your code terse. In this talk, Sean Parent, at that time working on Adobe Photoshop, estimated that the PS codebase could be reduced from 3,000,000 LOC to 30,000 LOC (=100x!!) if they followed ideas from the book https://www.youtube.com/watch?v=4moyKUHApq4&t=39m30s Another point of his is that the explosion of written code we are seeing isn't sustainable and that so much of this code is algorithms or data structures with overlapping functionalities. As the codebases grow, and these functionalities diverge even further, pulling the reigns in on the chaos becomes gradually impossible. Bjarne Stroustrup (aka the C++ OG) gave this book five stars on Amazon (in what is his one and only Amazon product review lol). https://smile.amazon.com/review/R1MG7U1LR7FK6/

This style might become dominant because it's only really possible in modern successors of C++ such as Swift or Rust that have both "direct" access to memory and type classes/traits/protocols, not so much in C++ itself (unless debugging C++ template errors is your thing).

⬐ None

None

⬐ tnecniv

I recently picked up this book. Seems quite good, but I'm also mathematically inclined (there's a lot of abstract algebra in there).

⬐ GregBuchholz

Have you looked in the STEPS program by Alan Kay? Trying to recreate modern computing setup from the OS up in 20k lines of code...

http://www.vpri.org/pdf/tr2012001_steps.pdf

"If computing is important -- for daily life, learning, business, national defense, jobs, and more -- then qualitatively advancing computing is extremely important. Fro example, many software systems today are made from millions to hundreds of millions of lines of program code that is too large, complex and fragile to be improved, fixed, or integrated. (One hundred million lines of code at 50 lines per page is 5000 books of 400 pages each! This is beyond humane scale.)

What if this could be made literally 1000 times smaller -- or more? And made more powerful, clear, simple, and robust? This would bring one of the most important technologies of our time from a state that is almost out of human reach -- and dangerously close to being out of control -- back into human scale."

...and of course if you haven't seen it, you'll want to check out the Forth guys who want to do everything with 1000 times less code:

http://www.ultratechnology.com/forth.htm

⬐ adamnemecek

I'm aware of this but Alan Kay's work and this seem to be orthogonal. Alan Kay talks about reducing real systems that have compilers, inputs etc whereas Elements talks about like the day to day ways of writing code. Alan Kay might come up with a new keyword whose semantics magically lets you cut out 30% but Elements shows you that if you make your types behave certain way, generics will let you cut out a lot of code.

⬐ buzzybee

I would counter that this appears to be a repeatedly emerging consensus, including Stepanov, Kay, Simonyi, and a number of other "greats", that an approach that involves some degree of metaprogramming, guided by domain problem, is the way forward. They differ on terms - cooperating systems, model-driven, intentional, generic - and focus - whether to create new syntax, or to guide the creation of specific algorithms or data structures - but they aren't debating the power of the approach.

I'll give you a couple. Note that some of these are rehashes of my earlier comments.

# Elements of Programming

https://www.amazon.com/Elements-Programming-Alexander-Stepan...

This book proposes how to write C++-ish code in a mathematical way that makes all your code terse. In this talk, Sean Parent, at that time working on Adobe Photoshop, estimated that the PS codebase could be reduced from 3,000,000 LOC to 30,000 LOC (=100x!!) if they followed ideas from the book https://www.youtube.com/watch?v=4moyKUHApq4&t=39m30s

Another point of his is that the explosion of written code we are seeing isn't sustainable and that so much of this code is algorithms or data structures with overlapping functionalities. As the codebases grow, and these functionalities diverge even further, pulling the reigns in on the chaos becomes gradually impossible.

Bjarne Stroustrup (aka the C++ OG) gave this book five stars on Amazon (in what is his one and only Amazon product review lol).

This style might become dominant because it's only really possible in modern successors of C++ such as Swift or Rust, not so much in C++ itself.

https://smile.amazon.com/review/R1MG7U1LR7FK6/

# Grammar of graphics

https://www.amazon.com/Grammar-Graphics-Statistics-Computing...

This book changed my perception of creativity, aesthetics and mathematics and their relationships. Fundamentally, the book provides all the diverse tools to give you confidence that your graphics are mathematically sound and visually pleasing. After reading this, Tufte just doesn't cut it anymore. It's such a weird book because it talks about topics as disparate Bayesian rule, OOP, color theory, SQL, chaotic models of time (lolwut), style-sheet language design and a bjillion other topics but always somehow all of these are very relevant. It's like if Bret Victor was a book, a tour de force of polymathical insanity.

The book is in full color and it has some of the nicest looking and most instructive graphics I've ever seen even for things that I understand, such as Central Limit Theorem. It makes sense the the best graphics would be in the book written by the guy who wrote a book on how to do visualizations mathematically. The book is also interesting if you are doing any sort of UI interfaces, because UI interfaces are definitely just a subset of graphical visualizations.

# Scala for Machine Learning

https://www.amazon.com/Scala-Machine-Learning-Patrick-Nicola...

This book almost never gets mentioned but it's a superb intro to machine learning if you dig types, scalable back-ends or JVM.

It’s the only ML book that I’ve seen that contains the word monad so if you sometimes get a hankering for some monading (esp. in the context of ML pipelines), look no further.

Discusses setup of actual large scale ML pipelines using modern concurrency primitives such as actors using the Akka framework.

# Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques for Building Intelligent Systems

https://www.amazon.com/Hands-Machine-Learning-Scikit-Learn-T...

Not released yet but I've been reading the drafts and it's a nice intro to machine learning using modern ML frameworks, TensorFlow and Scikit-Learn.

# Basic Category Theory for Computer Scientists

https://www.amazon.com/gp/product/0262660717/ref=as_li_ss_tl...

Not done with the book but despite it's age, hands down best intro to category theory if you care about it only for CS purposes as it tries to show how to apply the concepts. Very concise (~70 pages).

# Markov Logic: An Interface Layer for Artificial Intelligence

https://www.amazon.com/Markov-Logic-Interface-Artificial-Int...

Have you ever wondered what's the relationship between machine learning and logic? If so look no further.

# Machine Learning: A Probabilistic Perspective (Adaptive Computation and Machine Learning series)

https://www.amazon.com/gp/product/0262018020/ref=as_li_ss_tl...

Exhaustive overview of the entire field of machine learning. It's engaging and full of graphics.

# Deep Learning

https://www.amazon.com/gp/product/0262035618/ref=as_li_ss_tl...

http://www.deeplearningbook.org/

You probably have heard about this whole "deep learning" meme. This book is a pretty self-contained intro into the state of the art of deep learning.

# Designing for Scalability with Erlang/OTP: Implement Robust, Fault-Tolerant Systems

https://www.amazon.com/Designing-Scalability-Erlang-OTP-Faul...

Even though this is an Erlang book (I don't really know Erlang), 1/3 of the book is devoted to designing scalable and robust distributed systems in a general setting which I found the book worth it on it's own.

# Practical Foundations for Programming Languages

https://www.amazon.com/gp/product/1107150302/ref=as_li_ss_tl...

Not much to say, probably THE book on programming language theory.

# A First Course in Network Theory

https://www.amazon.com/First-Course-Network-Theory/dp/019872...

Up until recently I didn't know the difference between graphs and networks. But look at me now, I still don't but at least I have a book on it.

⬐ bad_user

Amazon links with your affiliate tag, seriously?

⬐ None

None

⬐ adamnemecek

what about them?

⬐ kranner

I see nothing wrong with GP providing their affiliate tag.

They are referring customers to Amazon, and customers don't pay extra.

⬐ bad_user

As an ex-Amazon Affiliate myself, I disagree because the incentive to post those links is not aligned with the reader's expectations.

Do you enjoy viewing commercials and product placements without the proper disclaimer? Because this is exactly what this is. I surely don't appreciate hidden advertising, not because of the quality of the advertised products, but because I cannot trust such recommendations, as a salesman can say anything in order to sell his shit.

Notice how this is the biggest list of recommendations in this thread. Do you think that's because the author is very knowledgeable or is it because he has an incentive to post links?

⬐ adamnemecek

> As an ex-Amazon Affiliate myself, I disagree because the incentive to post those links is not aligned with the reader's expectations.

Please don't project your behavior onto others. I take book recommendations seriously. I actually really enjoy it, people have told me IRL that my recommendations helped them a lot.

> Notice how this is the biggest list of recommendations in this thread.

They are all books that I've read in the last ~4 monthish (not all in entirety). Just FYI I'm not sure how much money you think I'm making off this but for me it's mostly about the stats, I'm curious what people are interested in.

> Do you think that's because the author is very knowledgeable

I'm more than willing to discuss my knowledgeability.

> or is it because he has an incentive to post links?

It's the biggest list because due to circumstances I have the luxury of being able to read a ton. I own all the books on the list, I've read all of them and I stand by all of them and some of these are really hidden gems that more people need to know about. I've written some of the reviews before. Just FYI I've posted extensive non-affiliate amazon links before and I started doing affiliate only very recently.

Furthermore, HN repeatedly upvotes blog posts that contain affiliate links. Why is that any different?

⬐ hackermailman

Practical Foundations for Programming Languages by Bob Harper is really good, plus there's a free draft of the second version on the author's site http://www.cs.cmu.edu/~rwh/pfpl.html

I always go to the book author's page first not only to get the errata but also discover things such as free lectures as in the case with Skeina's Algorithm Design Book

If you are curious how does Abstract Algebra relate to software engineering, this book might be of your interest

https://smile.amazon.com/Elements-Programming-Alexander-Step...

The main selling point of this approach is that it can cut out a lot of code

In this talk, Sean Parent, at that time working on Adobe Photoshop, estimated that the PS codebase could be reduced from 3,000,000 LOC to 30,000 LOC (=100x!!) if they followed ideas from the book https://www.youtube.com/watch?v=4moyKUHApq4&t=10s

Another point of his is that the explosion of written code we are seeing isn't sustainable and that so much of this code is algorithms or data structures with overlapping functionalities. As the codebases grow, and these functionalities diverge even further, pulling the reigns in on the chaos becomes gradually impossible.

Bjarne Stroustrup (aka the C++ OG) gave this book five stars on Amazon (in what is his one and only Amazon product review lol).

https://smile.amazon.com/review/R1MG7U1LR7FK6/

⬐ contravariant

The book 'The science of programming' of David Gries is also a nice introduction to writing programs using mathematical logic and reasoning. The book is inspired by the techniques of Dijkstra and Hoare, and written in a way that's easy to follow.

⬐ miloshadzic

Jeez, was about to buy it but it's about 90. What's the deal with this pricing?

⬐ KurtMueller

I did a google search of the book and found a pdf as the first result:

http://blg89.net/blog/wp-content/uploads/2013/11/The-Science...

⬐ happynewyear

I guess this is a whimsical aside/

I looked that up and got this sample of that book: http://www.cs.cornell.edu/courses/CS5860/2011fa/documents/Gr...

Looking up Stepanov's book, landed on his slide deck about the book.

Interestingly enough, both are about the remainder algorithm, and Gries' mentions a "large program" of "3000 lines", which reminded me of the other comment and the 3000000 to 30000 figure.

[Stepanov's deck: http://stepanovpapers.com/EoP-StanfordEE380.pdf]

⬐ RossBencina

I have not yet read "Elements of Programming," but there is another book "From Mathematics to Generic Programming," also co-authored by Alexander Stepanov, which links Abstract Algebra to program design. I found the latter very approachable:

https://smile.amazon.com/Mathematics-Generic-Programming-Ale...

⬐ charlieflowers

Question -- clearly, when you look at Haskell, you see abstract algebra. Were Stepanov's ideas big influencers of Haskell, or vice versa, or neither?

⬐ RossBencina

Stepanov and Musser were working on generic programming in the 80s[0][1], which predates Haskell. I suspect parametric polymorphism was common in the FP world by that time, but I don't know whether it was used as an organising principle to the extent that Stepanov promotes.

[0] https://en.wikipedia.org/wiki/Alexander_Stepanov

[1] https://en.wikipedia.org/wiki/David_Musser

⬐ stillworks

I always had a strong belief that principles of Group Theory must correlate into writing type systems.

Just looked at the ToC of this book and there is an entire chapter on Groups in that book ! Not only Groups but also goes into Rings as well...

Must read this book.

⬐ tel

Groupoids, a generalization of groups, factor heavily into type theory!

⬐ empath75

I think what you want is Category Theory.

⬐ RossBencina

In a Group, every element has an inverse element. If you drop this property, you get a Monoid.[0] Functions under function composition form a Monoid (or Monoidal Category). Brian Beckman did a nice intro video to this at MS Channel 9 a few years back.[1]

[0] https://en.wikipedia.org/wiki/Monoid

[1] "Brian Beckman: Don't fear the Monad" https://www.youtube.com/watch?v=ZhuHCtR3xq8

⬐ mpweiher

Note: apparently both Sean Parent and Bjarne Stroustrup contributed to the book, so their evaluation may not be entirely unbiased.

Still sounds interesting.

⬐ nerdponx

Sorry for a fluff comment, but +1 for using Amazon Smile links.

⬐ stiff

estimated that the PS codebase could be reduced from 3,000,000 LOC to 30,000 LOC (=100x!!)

I challenge anyone making such claims to construct an even _moderately_ realistic example of where such reduction would be possible.

Here is a very simple and already very optimistic model that is still way below a 100x reduction: say my program consists nearly exclusively of sorting of various arrays and I write out bubble sort in full each time using for loops etc. rather than call a sort() procedure. Bubble sort takes roughly 20 lines of code to implement, so I would at best get a reduction of close to a 20x, rather than a 100x one.

⬐ Someone

I doubt the 100x range, too, but Photoshop is decades old and, likely, has lots of its filters written in assembly for speed, with zillions of variants for all kinds of sometimes long gone CPUs and GPUs. It also still may have its own memory manager.

It that is true, it should be possible to get large wins in LOC if one either is willing to give up orders of magnitude in speed and memory usage, or is equipped with a sufficiently advanced compiler.

There likely also is some room for improvement if one is willing to change the file format (.psd is not known for its consistency), but that would likely be a rounding error.

⬐ seanmcdirmid

Reducing a single bubble sort in code size doesn't do much, because it is used many times and its code size cost is well amortized. However, if you had many bubble sorts coded in your program with special cases, then consolidating those into one very abstract/parametric version could be a big win.

The reduction comes from making the code way more abstract and much more difficult to reason about in concrete contexts. So ya, it is probably possible, but no, our heads might not be very capable in writing/maintaining/debugging such code even if the computer could execute it. As an extreme, consider "very small code" contests in the demo scene, where various crazy tricks (self modifying code) are used to drastically reduce code sizes, none of which would ever be used in practice and have little SE value.

⬐ catnaroek

I'd expect more opportunities for reduction in code size to arise when you use fancy algorithms (not sorting) on generic algebraic structures that have many models in your program. For example, when I was a student, I implemented a library of numerical methods for my own use in class. It benefited a lot from being implemented as generically as possible.

That being said, a 100x reduction in code size seems like an exaggeration.

⬐ jules

I don't doubt that your conclusion is correct, but what if the sort is used in 10 other functions, each of which is used 10 times? If you inlined everything you would end up with 100 copies of bubble sort. The code blowup is exponential in the level of nesting.

⬐ adamnemecek

Indeed. Another venue for why it cuts out code is that it makes code more reusable. As a result, you are more likely to integrate nicely with already written libraries than write your own code. Technically, this is a way of reducing your codebase as well lol.

Here's one :

A Possible Future of Software Development : www.youtube.com/watch?v=4moyKUHApq4‎

"Ask any C++ guru and they will tell you: avoid mutation, avoid side effects, don’t use loops, avoid class hierarchies and inheritance. But you will need strict discipline and total control over your collaborators to pull that off because C++ is so permissive.

"Haskell is not permissive, it won’t let you — or your coworkers — write unsafe code. Yes, initially you’ll be scratching your head trying to implement something in Haskell that you could hack in C++ in 10 minutes. If you’re lucky, and you work for Sean Parent or other exceptional programmer, he will code review your hacks and show you how not to program in C++."

For those who aren't familiar with Parent, watch this talk:

https://www.youtube.com/watch?v=4moyKUHApq4

which introduced me to Stepanov's "Elements of Programming" and the idea that I could be working with better/more reliable/provable abstractions in C++.

That said... after a few weeks of shallowly digesting the material, I started thinking pretty much like Milewski says above: it's great if I can learn to write better C++, but if what I really want is to escape from its specific dangers and general common programming pitfalls, something like Haskell seems like a better bet.

Related: http://www.youtube.com/watch?v=4moyKUHApq4

An interesting approach to declarative GUI programming from Adobe.

Turns out I greatly exagerated my memories (sic), he was mentionning equivalence relation using c++ concepts then went into constraint/dependant interactive input.

http://stlab.adobe.com/wiki/images/2/20/2008_07_25_google.pd...

page 18, 38

this was hosted at googletechtalks http://www.youtube.com/watch?v=4moyKUHApq4 (240p)