HN Books @HNBooksMonth

Ian Goodfellow’s Deep Learning book pretty much useless. I own it and have read through most parts of it. I couldn’t explain it better than top Amazon reviews:

https://www.amazon.com/Deep-Learning-Adaptive-Computation-Ma...

And I’m surprised to not find Aurelion Geron’s absolute masterpiece listed below. I believe it is the best machine learning book ever, although Statistical Learning mentioned in the article is really good as well :

https://www.amazon.com/Hands-Machine-Learning-Scikit-Learn-T...

⬐ whymauri

This is pretty harsh. I use Goodfellow as a reference text and then supplement the mathematics behind it with more comprehensive texts like Hastie or Wasserman. Maybe if you sit down and read it cover-to-cover, it will seem disjointed. But I usually read chapters independently - I recently read the convolutional neural network chapter in preparation for an interview and I thought it was fine.

⬐ spectramax

I much prefer reading Karpathy's notes and watching Stanford CS230 than to delve into Goodfellow. It sits on my shelf collecting dust.

⬐ yasp

What are Wasserman and Hastie?

⬐ esfandia

- Wasserman has a book called "All of statistics" that gives a lot of the background required to understand modern machine learning

- Hastie is a co-author of two machine learning books, one is "Elements of Statistical Learning" which is very comprehensive, and "Introduction to Statistical Learning", which is more approachable by people without too much background in stats.

⬐ platz

So it's a reference, not a pedagogical tool then?

A reference implies you already know the topic and just want an index to jog your memory for things you can't hold all in your head at once.

That is different than a pedagogical tool. If so, you shouldn't recommend it to those want to learn the topic.

⬐ whymauri

Phrased like this, I agree with you. I didn't think about it like that.

I agree with the poster below. Outside of classes, lecture notes, the books I listed, and Sutton/Barto (Intro. to Reinforcement Learning) have taught me the material. I use Goodfellow to brush up before interviews or jog my memory about topic I don't work with very often (like computer vision).

⬐ a_bonobo

FYI, there's a new edition of Geron's book coming out in August which will include Keras: https://www.goodreads.com/book/show/40363665-hands-on-machin...

⬐ hooloovoo_zoo

I think the Amazon review is rather dramatic, and probably not in a position to comment on style. I thought both Goodfellow and Geron were good. Goodfellow is deep learning for academics coming from a different field; Geron is deep learning for software engineers.

⬐ spectramax

That's the thing, even as an academic book it falls short. It feels disjointed, unorganized and poorly written - and most frustratingly, incomplete.

The reason for Goodfellow's popularity is that it was publish in 2014 right at the turn of exponential interest in Deep Learning after AlexNet. It took off and became popular, but readers now feel it is stale for the aforementioned reasons.

⬐ hooloovoo_zoo

I certainly agree it could be better, but I also think "disjointed, unorganized and poorly written - and most frustratingly, incomplete" more or less sums up the field of deep learning :).

⬐ Tarq0n

I can second the recommendation for Geron's book, it's absolutely stellar.

⬐ hsikka

Do you think there is a need for a better written DL textbook? I definitely agreed with the review you linked.

I've always thought that Hands on ML by Geron was great implementation wise, but lacking in the mathematical rigor and depth. While I would have a general sense of what is going on after reading it, and I'd certainly be able to structure and implement a model, I don't know if I would have any deep intuitions.

⬐ spectramax

Geron's book is more of a tutorial/cookbook coalesced with important insights into the practice of machine learning. So, I recommend reading Introduction to Statistical Learning (and Elements of Statistical Learning for theoretical background) before jumping into Geron's book. As engineers, I agree we need to have some theoretical background but at the same time, we are applying this knowledge to real world problems. Geron's book is invaluable and I hope publishes more, it is a gem.

⬐ sharcerer

2nd edition coming in August. Preorders opened a few days ago. He posted on twitter. Some preview chapters available on O'Reilly's site.

⬐ ausbah

I've heard poor things about Goodwell's Deep Learning book as well, what is a good alternative?

⬐ spectramax

Take CS230, I believe it used to be taught by Fei Fei followed by Karpathy and I don't know who teaches it now.

⬐ glial

For what it's worth, I read Goodwell's book cover to cover and loved it. It answers "why" questions rather than "how" questions, but those are the questions I had, and you can find "how" questions answered for your framework of choice on the internet.

⬐ DanielleMolloy

Gérons book is both entertaining and educational, I really enjoy it so far.

⬐ nilkn

For what it's worth, I disagree quite strongly with that review. The book is aimed at those with a pretty mature appetite for abstract mathematical reasoning, but not much specific knowledge in the areas of statistics, machine learning, and neural networks. It's an actual graduate-level book, and one must approach it with the appropriate background and education.

The Goodfellow book is not complete as an academic intro, but no one book can be. It's not very useful as a practical tutorial, but no book seeking this could cover the mathematical arguments that Goodfellow's book does. I found Goodfellow's book extremely useful for consolidating a lot of handwaving that I'd seen elsewhere and putting it in a slightly more rigorous framework that I could make sense of and immediately work with as a (former) mathematician.

Goodfellow's treatment is especially useful for mathematicians and mathematically-trained practitioners who nevertheless lack a background in advanced statistics. The Elements of Statistical Learning, for instance, is extremely heavy on statistics-specific jargon, and I personally found it far more difficult to extract useful insights from that book than I did from Goodfellow's.

⬐ freyr

The best textbooks, the ones considered classic gems in their field, are careful about what they say and what they leave out. You get the sense that every word is in its right place.

By comparison, Goodfellow and his co-authors seemed to just dump everything they know onto the page. It's fragmented, bloated, and it meanders all over the place. Goodfellow was on a recent podcast where he seems to acknowledge that the book straddles an awkward place between tutorial and reference.

I don't mean to sound too harsh. I appreciate its scope, and I've certainly read much worse textbooks.

⬐ nilkn

Considering how much is not in the book -- and that the book is not even that large as far as textbooks go -- part of me feels this criticism is somewhat disingenuous. I'd agree that the direction of the final part on research topics feels fragmented, but the rest of the book certainly doesn't. It's very clearly focused on developing deep neural networks and doesn't actually meander at all.

If the worst you can say is it's not a classic text, that's really not saying much at all. I feel weird defending the book so much when to me it's just a book I found useful and I don't even feel that strongly towards it. But the strength of some of the criticism here doesn't seem motivated by the book itself.

⬐ freyr

I have to disagree. There are several cases in a chapter’s introductory section, for example, he veers off onto an unnecessarily detailed tangent.

⬐ spectramax

The problem with Goodfellow's book is that it is half-baked. I don't know why he has to introduce Linear Algebra section for a 3rd of the book (which ends abruptly) but then moves on to the ANNs. If Goodfellow intended this book for mathematicians, that whole section about LA can be omitted with literally no loss in the book's value. The whole book arguably feels rushed.

So, no amount of praise and mathematician's justification makes sense. I agree that it is inclined for mathematicians, but this book is overrated and it is terribly due for a rewrite, update and frankly in my personal view - the writing style.

I am curious of specific parts of the book you found valuable.

⬐ jhanschoo

Perhaps restructuring the LA part into an appendix would be preferable. For mature readers, it nevertheless serves as a way to focus and agree on notation that is used for the rest of the book.

⬐ nilkn

The section on linear algebra moves quickly. I don't see how a book including extra prerequisite material is an example of it being half-baked or rushed. Surely that would actually be an example of the opposite? That section is a nice reference to have immediately available for things like SVD and PCA.

I think this is the same author that published "Hands-On Machine Learning with Scikit-Learn and TensorFlow...". The quality of the book (thus far) is so high that I immediately started Googling about the author to try and learn more (and did not learn much), assuming he must be well known. I did not learn much, but can at least say the book is fantastic.

[1]: https://www.amazon.com/Hands-Machine-Learning-Scikit-Learn-T...

⬐ j_s

Too bad O'Reilly no longer sells books -- Cyber Monday 50% off is no more.

⬐ chillee

He used to be the PM of youtube video classification too.

⬐ colmvp

I second that book. Well worth the small price and very accessible to read

⬐ isp

Yes, same author. He has a picture of his book at the end of the video (21:40). With these recommendations, and after that video, I am going to buy & read his book.

If TensorFlow is what you're interested in I personally found "Hands-on Machine Learning with SciKit-Learn and TensorFlow by Aurélien Géron" to be the best introduction after introducing myself to the subject with Siraj's YouTube videos

https://www.amazon.com/Hands-Machine-Learning-Scikit-Learn-T...

What? No. Why in the world do people even ask this kind of question. To a first approximation, the answer to "is it too late to get started with ..." question is always "no".

If no, what are the great resources for starters?

The videos / slides / assignments from here:

http://ai.berkeley.edu/home.html

This class:

https://www.coursera.org/learn/machine-learning

This class:

https://www.udacity.com/course/intro-to-machine-learning--ud...

This book:

https://www.amazon.com/Artificial-Intelligence-Modern-Approa...

This book:

https://www.amazon.com/Hands-Machine-Learning-Scikit-Learn-T...

This book:

https://www.amazon.com/Introduction-Machine-Learning-Python-...

These books:

http://greenteapress.com/thinkstats/thinkstats.pdf

http://www.greenteapress.com/thinkbayes/thinkbayes.pdf

This book:

https://www.amazon.com/Machine-Learning-Hackers-Studies-Algo...

This book:

https://www.amazon.com/Thoughtful-Machine-Learning-Test-Driv...

These subreddits:

http://artificial.reddit.com

http://machinelearning.reddit.com

http://semanticweb.reddit.com

These journals:

http://www.jmlr.org

http://www.jair.org

This site:

http://arxiv.org/corr/home/

Any tips before I get this journey going?

Depending on your maths background, you may need to refresh some math skills, or learn some new ones. The basic maths you need includes calculus (including multi-variable calc / partial derivatives), probability / statistics, and linear algebra. For a much deeper discussion of this topic, see this recent HN thread:

https://news.ycombinator.com/item?id=15116379

Luckily there are tons of free resources available online for learning various maths topics. Khan Academy isn't a bad place to start if you need that. There are also tons of good videos on Youtube from Gilbert Strang, Professor Leonard, 3blue1brown, etc.

Also, check out Kaggle.com. Doing Kaggle contests can be a good way to get your feet wet.

And the various Wikipedia pages on AI/ML topics can be pretty useful as well.

I'll give you a couple. Note that some of these are rehashes of my earlier comments.

# Elements of Programming

https://www.amazon.com/Elements-Programming-Alexander-Stepan...

This book proposes how to write C++-ish code in a mathematical way that makes all your code terse. In this talk, Sean Parent, at that time working on Adobe Photoshop, estimated that the PS codebase could be reduced from 3,000,000 LOC to 30,000 LOC (=100x!!) if they followed ideas from the book https://www.youtube.com/watch?v=4moyKUHApq4&t=39m30s

Another point of his is that the explosion of written code we are seeing isn't sustainable and that so much of this code is algorithms or data structures with overlapping functionalities. As the codebases grow, and these functionalities diverge even further, pulling the reigns in on the chaos becomes gradually impossible.

Bjarne Stroustrup (aka the C++ OG) gave this book five stars on Amazon (in what is his one and only Amazon product review lol).

This style might become dominant because it's only really possible in modern successors of C++ such as Swift or Rust, not so much in C++ itself.

https://smile.amazon.com/review/R1MG7U1LR7FK6/

# Grammar of graphics

https://www.amazon.com/Grammar-Graphics-Statistics-Computing...

This book changed my perception of creativity, aesthetics and mathematics and their relationships. Fundamentally, the book provides all the diverse tools to give you confidence that your graphics are mathematically sound and visually pleasing. After reading this, Tufte just doesn't cut it anymore. It's such a weird book because it talks about topics as disparate Bayesian rule, OOP, color theory, SQL, chaotic models of time (lolwut), style-sheet language design and a bjillion other topics but always somehow all of these are very relevant. It's like if Bret Victor was a book, a tour de force of polymathical insanity.

The book is in full color and it has some of the nicest looking and most instructive graphics I've ever seen even for things that I understand, such as Central Limit Theorem. It makes sense the the best graphics would be in the book written by the guy who wrote a book on how to do visualizations mathematically. The book is also interesting if you are doing any sort of UI interfaces, because UI interfaces are definitely just a subset of graphical visualizations.

# Scala for Machine Learning

https://www.amazon.com/Scala-Machine-Learning-Patrick-Nicola...

This book almost never gets mentioned but it's a superb intro to machine learning if you dig types, scalable back-ends or JVM.

It’s the only ML book that I’ve seen that contains the word monad so if you sometimes get a hankering for some monading (esp. in the context of ML pipelines), look no further.

Discusses setup of actual large scale ML pipelines using modern concurrency primitives such as actors using the Akka framework.

# Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques for Building Intelligent Systems

https://www.amazon.com/Hands-Machine-Learning-Scikit-Learn-T...

Not released yet but I've been reading the drafts and it's a nice intro to machine learning using modern ML frameworks, TensorFlow and Scikit-Learn.

# Basic Category Theory for Computer Scientists

https://www.amazon.com/gp/product/0262660717/ref=as_li_ss_tl...

Not done with the book but despite it's age, hands down best intro to category theory if you care about it only for CS purposes as it tries to show how to apply the concepts. Very concise (~70 pages).

# Markov Logic: An Interface Layer for Artificial Intelligence

https://www.amazon.com/Markov-Logic-Interface-Artificial-Int...

Have you ever wondered what's the relationship between machine learning and logic? If so look no further.

# Machine Learning: A Probabilistic Perspective (Adaptive Computation and Machine Learning series)

https://www.amazon.com/gp/product/0262018020/ref=as_li_ss_tl...

Exhaustive overview of the entire field of machine learning. It's engaging and full of graphics.

# Deep Learning

https://www.amazon.com/gp/product/0262035618/ref=as_li_ss_tl...

http://www.deeplearningbook.org/

You probably have heard about this whole "deep learning" meme. This book is a pretty self-contained intro into the state of the art of deep learning.

# Designing for Scalability with Erlang/OTP: Implement Robust, Fault-Tolerant Systems

https://www.amazon.com/Designing-Scalability-Erlang-OTP-Faul...

Even though this is an Erlang book (I don't really know Erlang), 1/3 of the book is devoted to designing scalable and robust distributed systems in a general setting which I found the book worth it on it's own.

# Practical Foundations for Programming Languages

https://www.amazon.com/gp/product/1107150302/ref=as_li_ss_tl...

Not much to say, probably THE book on programming language theory.

# A First Course in Network Theory

https://www.amazon.com/First-Course-Network-Theory/dp/019872...

Up until recently I didn't know the difference between graphs and networks. But look at me now, I still don't but at least I have a book on it.

⬐ bad_user

Amazon links with your affiliate tag, seriously?

⬐ None

None

⬐ adamnemecek

what about them?

⬐ kranner

I see nothing wrong with GP providing their affiliate tag.

They are referring customers to Amazon, and customers don't pay extra.

⬐ bad_user

As an ex-Amazon Affiliate myself, I disagree because the incentive to post those links is not aligned with the reader's expectations.

Do you enjoy viewing commercials and product placements without the proper disclaimer? Because this is exactly what this is. I surely don't appreciate hidden advertising, not because of the quality of the advertised products, but because I cannot trust such recommendations, as a salesman can say anything in order to sell his shit.

Notice how this is the biggest list of recommendations in this thread. Do you think that's because the author is very knowledgeable or is it because he has an incentive to post links?

⬐ adamnemecek

> As an ex-Amazon Affiliate myself, I disagree because the incentive to post those links is not aligned with the reader's expectations.

Please don't project your behavior onto others. I take book recommendations seriously. I actually really enjoy it, people have told me IRL that my recommendations helped them a lot.

> Notice how this is the biggest list of recommendations in this thread.

They are all books that I've read in the last ~4 monthish (not all in entirety). Just FYI I'm not sure how much money you think I'm making off this but for me it's mostly about the stats, I'm curious what people are interested in.

> Do you think that's because the author is very knowledgeable

I'm more than willing to discuss my knowledgeability.

> or is it because he has an incentive to post links?

It's the biggest list because due to circumstances I have the luxury of being able to read a ton. I own all the books on the list, I've read all of them and I stand by all of them and some of these are really hidden gems that more people need to know about. I've written some of the reviews before. Just FYI I've posted extensive non-affiliate amazon links before and I started doing affiliate only very recently.

Furthermore, HN repeatedly upvotes blog posts that contain affiliate links. Why is that any different?

⬐ hackermailman

Practical Foundations for Programming Languages by Bob Harper is really good, plus there's a free draft of the second version on the author's site http://www.cs.cmu.edu/~rwh/pfpl.html

I always go to the book author's page first not only to get the errata but also discover things such as free lectures as in the case with Skeina's Algorithm Design Book

Probably TensorFlow the hype is too strong. This book looks solid https://www.amazon.com/Hands-Machine-Learning-Scikit-Learn-T...

⬐ inlineint

I'm reading an early release of this book in O'Reilly Safari and confirm that it's worth reading.

However, if you don't get used to deep learning previously it might be worth to combine it with something more foundational about deep learning. I like the lectures by Nando de Freitas (https://www.cs.ox.ac.uk/people/nando.defreitas/machinelearni...) and a book by Ian Godfellow (http://www.deeplearningbook.org/).