HN Books @HNBooksMonth

Some of the best textbooks in statistics and machine learning:

Applied -------

Hosmer et al., Applied Logistic Regression. An exhaustive guide to the perils and pitfalls of logistic regression. Logistic regression is the power tool of interpretable statistical models, but if you don't understand it, it will take your foot off (concretely, your inferences will be wrong and your peers will laugh at you.) This book is essential. Graduate level, or perhaps advanced undergraduate, intended for STEM and social science grad students.

https://www.amazon.com/Applied-Logistic-Regression-Probabili...

Peter Christen's Data Matching. Record Linkage is a relatively niche concept, so Christen's book has no right to be as good as it is. But it covers every relevant topic in a clear, even-handed way. If you are working on a record linkage system, then there's nothing in this book you can afford not to know. Undergraduate level, but intended for industry practitioners.

https://www.amazon.com/Data-Matching-Techniques-Data-Centric...

Max Kuhn's Applied Predictive Modeling. Even if you don't use R, this is an incredibly good introduction to how predictive modeling is done in practice. Early undergraduate level.

http://appliedpredictivemodeling.com/

Theoretical -----------

The Elements of Statistical Learning. Probably the single most respected book in machine learning. Exhaustive and essential. Advanced undergraduate level.

https://www.amazon.com/Elements-Statistical-Learning-Predict...

Kevin Murphy's Machine Learning: A Probabilistic Perspective. Covers lots of the same ground as Elements but is a little easier. Undergraduate level.

https://www.amazon.com/Machine-Learning-Probabilistic-Perspe...

Taboga's Lectures on Probability Theory and Mathematical Statistics. Has the distinction of being available for free in web-friendly format at https://www.statlect.com/ . Undergraduate level for math majors.

https://www.amazon.com/Lectures-Probability-Theory-Mathemati...

I'll give you a couple. Note that some of these are rehashes of my earlier comments.

# Elements of Programming

https://www.amazon.com/Elements-Programming-Alexander-Stepan...

This book proposes how to write C++-ish code in a mathematical way that makes all your code terse. In this talk, Sean Parent, at that time working on Adobe Photoshop, estimated that the PS codebase could be reduced from 3,000,000 LOC to 30,000 LOC (=100x!!) if they followed ideas from the book https://www.youtube.com/watch?v=4moyKUHApq4&t=39m30s

Another point of his is that the explosion of written code we are seeing isn't sustainable and that so much of this code is algorithms or data structures with overlapping functionalities. As the codebases grow, and these functionalities diverge even further, pulling the reigns in on the chaos becomes gradually impossible.

Bjarne Stroustrup (aka the C++ OG) gave this book five stars on Amazon (in what is his one and only Amazon product review lol).

This style might become dominant because it's only really possible in modern successors of C++ such as Swift or Rust, not so much in C++ itself.

https://smile.amazon.com/review/R1MG7U1LR7FK6/

# Grammar of graphics

https://www.amazon.com/Grammar-Graphics-Statistics-Computing...

This book changed my perception of creativity, aesthetics and mathematics and their relationships. Fundamentally, the book provides all the diverse tools to give you confidence that your graphics are mathematically sound and visually pleasing. After reading this, Tufte just doesn't cut it anymore. It's such a weird book because it talks about topics as disparate Bayesian rule, OOP, color theory, SQL, chaotic models of time (lolwut), style-sheet language design and a bjillion other topics but always somehow all of these are very relevant. It's like if Bret Victor was a book, a tour de force of polymathical insanity.

The book is in full color and it has some of the nicest looking and most instructive graphics I've ever seen even for things that I understand, such as Central Limit Theorem. It makes sense the the best graphics would be in the book written by the guy who wrote a book on how to do visualizations mathematically. The book is also interesting if you are doing any sort of UI interfaces, because UI interfaces are definitely just a subset of graphical visualizations.

# Scala for Machine Learning

https://www.amazon.com/Scala-Machine-Learning-Patrick-Nicola...

This book almost never gets mentioned but it's a superb intro to machine learning if you dig types, scalable back-ends or JVM.

It’s the only ML book that I’ve seen that contains the word monad so if you sometimes get a hankering for some monading (esp. in the context of ML pipelines), look no further.

Discusses setup of actual large scale ML pipelines using modern concurrency primitives such as actors using the Akka framework.

# Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques for Building Intelligent Systems

https://www.amazon.com/Hands-Machine-Learning-Scikit-Learn-T...

Not released yet but I've been reading the drafts and it's a nice intro to machine learning using modern ML frameworks, TensorFlow and Scikit-Learn.

# Basic Category Theory for Computer Scientists

https://www.amazon.com/gp/product/0262660717/ref=as_li_ss_tl...

Not done with the book but despite it's age, hands down best intro to category theory if you care about it only for CS purposes as it tries to show how to apply the concepts. Very concise (~70 pages).

# Markov Logic: An Interface Layer for Artificial Intelligence

https://www.amazon.com/Markov-Logic-Interface-Artificial-Int...

Have you ever wondered what's the relationship between machine learning and logic? If so look no further.

# Machine Learning: A Probabilistic Perspective (Adaptive Computation and Machine Learning series)

https://www.amazon.com/gp/product/0262018020/ref=as_li_ss_tl...

Exhaustive overview of the entire field of machine learning. It's engaging and full of graphics.

# Deep Learning

https://www.amazon.com/gp/product/0262035618/ref=as_li_ss_tl...

http://www.deeplearningbook.org/

You probably have heard about this whole "deep learning" meme. This book is a pretty self-contained intro into the state of the art of deep learning.

# Designing for Scalability with Erlang/OTP: Implement Robust, Fault-Tolerant Systems

https://www.amazon.com/Designing-Scalability-Erlang-OTP-Faul...

Even though this is an Erlang book (I don't really know Erlang), 1/3 of the book is devoted to designing scalable and robust distributed systems in a general setting which I found the book worth it on it's own.

# Practical Foundations for Programming Languages

https://www.amazon.com/gp/product/1107150302/ref=as_li_ss_tl...

Not much to say, probably THE book on programming language theory.

# A First Course in Network Theory

https://www.amazon.com/First-Course-Network-Theory/dp/019872...

Up until recently I didn't know the difference between graphs and networks. But look at me now, I still don't but at least I have a book on it.

⬐ bad_user

Amazon links with your affiliate tag, seriously?

⬐ None

None

⬐ adamnemecek

what about them?

⬐ kranner

I see nothing wrong with GP providing their affiliate tag.

They are referring customers to Amazon, and customers don't pay extra.

⬐ bad_user

As an ex-Amazon Affiliate myself, I disagree because the incentive to post those links is not aligned with the reader's expectations.

Do you enjoy viewing commercials and product placements without the proper disclaimer? Because this is exactly what this is. I surely don't appreciate hidden advertising, not because of the quality of the advertised products, but because I cannot trust such recommendations, as a salesman can say anything in order to sell his shit.

Notice how this is the biggest list of recommendations in this thread. Do you think that's because the author is very knowledgeable or is it because he has an incentive to post links?

⬐ adamnemecek

> As an ex-Amazon Affiliate myself, I disagree because the incentive to post those links is not aligned with the reader's expectations.

Please don't project your behavior onto others. I take book recommendations seriously. I actually really enjoy it, people have told me IRL that my recommendations helped them a lot.

> Notice how this is the biggest list of recommendations in this thread.

They are all books that I've read in the last ~4 monthish (not all in entirety). Just FYI I'm not sure how much money you think I'm making off this but for me it's mostly about the stats, I'm curious what people are interested in.

> Do you think that's because the author is very knowledgeable

I'm more than willing to discuss my knowledgeability.

> or is it because he has an incentive to post links?

It's the biggest list because due to circumstances I have the luxury of being able to read a ton. I own all the books on the list, I've read all of them and I stand by all of them and some of these are really hidden gems that more people need to know about. I've written some of the reviews before. Just FYI I've posted extensive non-affiliate amazon links before and I started doing affiliate only very recently.

Furthermore, HN repeatedly upvotes blog posts that contain affiliate links. Why is that any different?

⬐ hackermailman

Practical Foundations for Programming Languages by Bob Harper is really good, plus there's a free draft of the second version on the author's site http://www.cs.cmu.edu/~rwh/pfpl.html

I always go to the book author's page first not only to get the errata but also discover things such as free lectures as in the case with Skeina's Algorithm Design Book

As others have said, game AI doesn't draw from "real" or academic AI or Machine Learning.

That said, the references for AI and machine learning are quite old. Particularly the machine learning parts. The only ML texts on the list are Mitchell and Duda and Hart. The former is extremely outdated at this point. That's not Mitchell's fault -- it was a nice book for learning the basics of machine learning in 1997 when it was published, but all the developments that have made ML a hot subject have occurred since then and in areas that the book simply didn't predict coming. Duda and Hart, similarly, was the bible for certain subfields of ML for a long time, but it won't tell you what everyone's been doing in the past 15 years when ML exploded onto the wider scene.

If I were to add one book, it would be Kevin Murphy's excellent text (https://www.amazon.com/Machine-Learning-Probabilistic-Perspe...). There's no one book that will give you a complete picture of the field, but his is I think the closest available and does a solid job of preparing you with enough fundamentals that you can extend your knowledge from there on your own.

In retrospect, my other comment was stupidly obtuse. Both too technical (in the sense of specificity) and too unstructured (in the sense of presentation order). A more appropriate path from CS might be analogous (well, inverse if anything) to the path Robert Goldblatt has taken. It dips into nonstandard analysis, but not totally without reason. Some subset of the following, with nLab and Wikipedia supplementing as necessary:

0. Milewski's "Category Theory for Programmers"[0]

1. Goldblatt's "Topoi"[1]

2. McLarty's "The Uses and Abuses of the History of Topos Theory"[2] (this does not require [1], it just undoes some historical assumptions made in [1] and, like everything else by McLarty, is extraordinarily well-written)

3. Goldblatt's "Lectures on the Hyperreals"[3]

4. Nelson's "Radically Elementary Probability Theory"[4]

5. Tao's "Ultraproducts as a Bridge Between Discrete and Continuous Analysis"[5]

6. Some canonical machine learning text, like Murphy[6] or Bishop[7]

7. Koller/Friedman's "Probabilistic Graphical Models"[8]

8. Lawvere's "Taking Categories Seriously"[9]

From there you should see a variety of paths for mapping (things:Uncertainty) <-> (things:Structure). The Giry monad is just one of them, and would probably be understandable after reading Barr/Wells' "Toposes, Triples and Theories"[10].

The above list also assumes some comfort with integration. Particularly good books in line with this pedagogical path might be:

9. Any and all canonical intros to real analysis

10. Malliavin's "Integration and Probability"[11]

11. Segal/Kunze's "Integrals and Operators"[12]

Similarly, some normative focus on probability would be useful:

12. Jaynes' "Probability Theory"[13]

13. Pearl's "Causality"[14]

---

[0] https://bartoszmilewski.com/2014/10/28/category-theory-for-p...

[1] https://www.amazon.com/Topoi-Categorial-Analysis-Logic-Mathe...

[2] http://www.cwru.edu/artsci/phil/UsesandAbuses%20HistoryTopos...

[3] https://www.amazon.com/Lectures-Hyperreals-Introduction-Nons...

[4] https://web.math.princeton.edu/%7Enelson/books/rept.pdf

[5] https://www.youtube.com/watch?v=IS9fsr3yGLE

[6] https://www.amazon.com/Machine-Learning-Probabilistic-Perspe...

[7] https://www.amazon.com/Pattern-Recognition-Learning-Informat...

[8] https://www.amazon.com/Probabilistic-Graphical-Models-Princi...

[9] http://www.emis.de/journals/TAC/reprints/articles/8/tr8.pdf

[10] http://www.tac.mta.ca/tac/reprints/articles/12/tr12.pdf

[11] https://www.springer.com/us/book/9780387944098

[12] https://www.amazon.com/Integrals-Operators-Grundlehren-mathe...

[13] http://www.med.mcgill.ca/epidemiology/hanley/bios601/Gaussia...

[14] https://www.amazon.com/Causality-Reasoning-Inference-Judea-P...

Some good books on Machine Learning:

Machine Learning: The Art and Science of Algorithms that Make Sense of Data (Flach): http://www.amazon.com/Machine-Learning-Science-Algorithms-Se...

Machine Learning: A Probabilistic Perspective (Murphy): http://www.amazon.com/Machine-Learning-Probabilistic-Perspec...

Pattern Recognition and Machine Learning (Bishop): http://www.amazon.com/Pattern-Recognition-Learning-Informati...

There are some great resources/books for Bayesian statistics and graphical models. I've listed them in (approximate) order of increasing difficulty/mathematical complexity:

Think Bayes (Downey): http://www.amazon.com/Think-Bayes-Allen-B-Downey/dp/14493707...

Bayesian Methods for Hackers (Davidson-Pilon et al): https://github.com/CamDavidsonPilon/Probabilistic-Programmin...

Doing Bayesian Data Analysis (Kruschke), aka "the puppy book": http://www.amazon.com/Doing-Bayesian-Data-Analysis-Second/dp...

Bayesian Data Analysis (Gellman): http://www.amazon.com/Bayesian-Analysis-Chapman-Statistical-...

Bayesian Reasoning and Machine Learning (Barber): http://www.amazon.com/Bayesian-Reasoning-Machine-Learning-Ba...

Probabilistic Graphical Models (Koller et al): https://www.coursera.org/course/pgm http://www.amazon.com/Probabilistic-Graphical-Models-Princip...

If you want a more mathematical/statistical take on Machine Learning, then the two books by Hastie/Tibshirani et al are definitely worth a read (plus, they're free to download from the authors' websites!):

Introduction to Statistical Learning: http://www-bcf.usc.edu/~gareth/ISL/

The Elements of Statistical Learning: http://statweb.stanford.edu/~tibs/ElemStatLearn/

Obviously there is the whole field of "deep learning" as well! A good place to start is with: http://deeplearning.net/

⬐ yedhukrishnan

Those are really useful. Thank you. Books are pricey though!

⬐ shogunmike

I know...some of them are indeed expensive!

At least the latter two ("ISL" and "ESL") are free to download though.

⬐ alexcasalboni

Those are great resources!

In case you are interested in MLaaS (Machine Learning as a Service), you can check these as well:

Amazon Machine Learning: http://aws.amazon.com/machine-learning/ (my review here: http://cloudacademy.com/blog/aws-machine-learning/)

Azure Machine Learning: http://azure.microsoft.com/en-us/services/machine-learning/ (my review here: http://cloudacademy.com/blog/azure-machine-learning/)

Google Prediction API: https://cloud.google.com/prediction/

BigML: https://bigml.com/

Prediction.io: https://prediction.io/

OpenML: http://openml.org/

⬐ yedhukrishnan

I went through the links and your review. They are really good. Thanks!

While not free, 'Machine Learning: A Probabilistic Perspective' (http://www.amazon.co.uk/gp/aw/d/0262018020) is the best book I have found so far. I also second the recommendations for Tibshirani's and MacKay's books; the former for mathematical foundations, the latter for the intuition.

As a reference, you'll want one or two of the Big 6 texts, by Murphy, Koller/Friedman, Bishop, MacKay, and Hastie et al ESL. The first review is good http://www.amazon.com/product-reviews/0262018020/ref=dp_top_...

Also, there are many freely available texts on ML, data mining, stats/prob distributions, linear algebra, optimization etc, incl Barber, Mackay and ESL. See http://www.reddit.com/r/MachineLearning/comments/1jeawf/mach...