HN Books @HNBooksMonth

I didn’t understand probability, until I read E T Jaynes Probability Theory: Logic of Science

It defines the base blocks of probability very, very slow. And never hand-waves anything. But it’s the “bayesian” view of probability; but it’s honestly the easier one to understand.

https://www.amazon.com/Probability-Theory-Science-T-Jaynes/d...

⬐ curious16

What is the prerequisite to read this book?

⬐ shp0ngle

It is quite math heavy, but I guess you can skip the proofs.

You can see the book for free, I think it’s scanned online, it’s from mid-20th century I think

⬐ AlanYx

There are no real prerequisites. In fact, in the introduction, Jaynes suggests that readers might be better off not having any previous introduction to statistics.

That being said, there is an assumed level of general mathematical sophistication to the presentation.

Ack. I can't believe I messed that up. He wrote an awesome book: "Probability Theory: The Logic of Science".

https://www.amazon.com/Probability-Theory-Science-T-Jaynes/d...

And there is a website with more information and a collection of his papers:

https://bayes.wustl.edu/

https://bayes.wustl.edu/etj/node1.html

⬐ kgwgk

You may find Caticha's Entropic Dynamics interesting:

https://arxiv.org/abs/1005.2357

http://dl.icdst.org/pdfs/files1/77964f05542451c01e8e420e975d...

⬐ flubert

That does look interesting. Thanks.

I'm currently reading up on formal logic in order to fully understand Probability Theory by E. T. Jaynes [1] and I've found it difficult to find a good logic book. There's a thousand on Amazon and it's hard to tell which ones are written a century ago w/ re-released dates with old notation or missing some new ideas. An "original publishing" data feature on Amazon would be a godsend.

I've seen a few well reviewed Symbolic logic books but not sure if that fits the criteria. Mostly digging into boolean algebra atm.

[1] https://www.amazon.com/Probability-Theory-Science-T-Jaynes/d...

⬐ 6AA4FD

I've used forall x (calgary remix), both the winter 2018 and the spring 2020 editions, and I found it quite good. It teaches truth functional and first order logic, and it's probably accessible to most high school graduates. I've seen it taught with good results to students of different academic backgrounds as a 'math' course for general education. I've linked it below and it's free (as in beer and as in freedom) so it's definitely worth glancing at, along with the rest of the open logic project.

http://forallx.openlogicproject.org/

⬐ nmadden

https://www.logicmatters.net/tyl/

⬐ bubblyworld

As a mathematician, I find this surprising - formal logic and probability theory don't usually have much to do with each other. Is the issue that you have trouble with formal mathematical notation?

⬐ BaronSamedi

This reminds me of my wish to see a unified theory (and notation) that encompasses formal logic and probability. On their own, each is lacking. A unified system, for example, in which we have all the tools of first order logic and Bayesian probability would be very powerful.

⬐ dmix

If you look at the probability book I mentioned it uses an algebraic boolean logic notation and basic set theory to build a formal theory of inference. The first chapter is an condensed introduction to the formal logic and notation used - which is what I was struggling with following some of the more complex equations.

Having read half a book on logic and subsequently learned basic set theory has already helped read the first portion of the probability book. But then I also got really into formal logic, I found it really fascinating as a programmer and I think every person should learn it (with plenty of applications to regular life), so I decided to take a deep dive into it. The venn diagrams visualizations are what helped me the most.

I think one of things that held me back initially was my background as a programmer, it made reading the logic set notation challenging, ie the plus signs meaning disjunctions and primes negation conjunctions.

⬐ bubblyworld

Ah, I see. That's interesting, I'll see if I can find a bootleg copy and have a read =). Logic is a beautiful, deep subject - all the best with your studies!

⬐ dmix

Here's the first 3x chapters, which I presume was for some course:

https://bayes.wustl.edu/etj/prob/book.pdf

I've heard amazing things about it and it hasn't yet disappointed (the little I've read). Worth the $60 I spent on Abebooks for it (used) but the full copy is also on ThePirateBay if you want to see a longer preview.

⬐ auggierose

Try this one:

https://www.cambridge.org/core/books/handbook-of-practical-l...

⬐ fmoralesc

I second the recommendation of checking the Teach Yourself Logic guides by Peter Smith, but I will give some more specific recommendations:

- Greg Restall - Logic (and he also gives some recommendations of his own that are worth checking out). If you use this, check the errata in Restall's website.

- Richard Jeffrey - Formal Logic: Its Scope and Limits (Smith has a textbook that is modelled largely after this, but Jeffrey's book is more to the point and fun to follow along)

- Daniel Velleman - How to Prove It, which has tons of good exercises for practicing symbolig logic in the context of the construction of proofs, and introduces some mathematics along the way.

⬐ dmix

Thanks, I wasn't familiar with the "Teach Yourself" series, that's going to help me on my path learning math/stats/probability theory.

I took a course on Applied Bayesian Statistics taught by David Draper in grad school and we covered Bayesian Data Analysis (Gelman et Al.) http://www.stat.columbia.edu/~gelman/book/ and Probability Theory and tbe Logic of Science by Ed Jaynes: https://www.amazon.com/dp/0521592712/ref=cm_sw_r_em_apa_i_v3...

The former is a much recommended book since it's very comprehensive and builds everything from the ground up and was the basis for the entire course. The latter is a beast of it's own and we simply covered what was effectively the first chapter as part of the course.

https://www.amazon.com/Probability-Theory-Science-T-Jaynes/d... is pretty decent.

⬐ elcapitan

Thanks!

I think you might enjoy E.T. Jaynes

http://bayes.wustl.edu/

https://www.amazon.com/Probability-Theory-E-T-Jaynes/dp/0521...

The style reminds me of E.T. Jaynes' Probability Theory: The Logic of Science[0]. If you haven't read it, I highly recommend it.

[0] http://www.amazon.com/Probability-Theory-The-Logic-Science/d...

Probability Theory: The Logic of Science

http://www.amazon.com/Probability-Theory-The-Logic-Science/d...

I might add that what made me understand these concepts is the writings of physicist and probability theorist E.T. Jaynes, especially his unpublished manuscripts: http://bayes.wustl.edu/etj/node2.html

I think if he would have been alive at the right time these would have been blog posts. Before reading them, I had taken an intro class in thermodynamics which at left me completely confused.

Read THE EVOLUTION OF CARNOT'S PRINCIPLE ( http://bayes.wustl.edu/etj/articles/ccarnot.pdf ) for incredible insights on how Carnot pioneered thermodynamics by trying to optimize steam engines.

Also if you think you dislike statistics and probabilities but you like math in general his book might change your mind: Probability Theory: The Logic of Science. Free draft: http://omega.albany.edu:8008/JaynesBook.html

Amazon: http://www.amazon.com/Probability-Theory-Science-T-Jaynes/dp...

In fact understanding his stance on probabilities, the mind projection fallacy in particular might be prerequisite to understand thermodynamics, the fundamental point being that entropy is not really directly a property of matter but more of a meta property that is about knowledge or information which is taken to mean correlations across aggregate matter.

Going to Amazon right now...

* edit: Doh, no Kindle version. I don't mind paying $90+ for a good book though, just like it to be electronic: http://www.amazon.com/Probability-Theory-The-Logic-Science/d...

⬐ incision

Well, it's available on Google Books [1], but I don't know about $63 for what appears to be a skewed scan of the print book.

Personally, I searched out a PDF and based on what I've read so far, I'm itching to pull the trigger on Amazon as I'm simply loving what I'm reading.

1: http://goo.gl/UHMBi

⬐ Datonomics

http://www.naturalthinker.net/trl/texts/Science/Jaynes,%20E....

⬐ dan_yall

Free pdf is available here:

http://bayes.wustl.edu/etj/prob/book.pdf

It's always nice to see good things come out of Wash U. (Alum here.)

⬐ Fixnum

Unfortunately, it's only the first 95 pages.

⬐ gwern

There must be a fuller version floating around, though; my PDF version has 548 pages and ends with Appendix E, 'Multivariate Gaussian Integrals'.

EDIT: In case anyone wants to make me feel bad about pirating, Jaynes is dead, and besides that, I bought a hardcopy as backup.

⬐ Wilduck

I found the full text here:

http://www.naturalthinker.net/trl/texts/Science/Jaynes,%20E....

The first couple pages are a bit funny looking, but after that, there are all 500+ pages. It was the fourth result on Google for me.

I can't find the book with that title, do you have a link or ISBN? http://www.amazon.com/Probability-Theory-Science-T-Jaynes/dp... is the closest match but it does not seem to match your description.

⬐ revorad

Not the full versions but still good:

http://bayes.wustl.edu/etj/prob/book.pdf

http://www-biba.inrialpes.fr/Jaynes/prob.html

⬐ T_S_

Thanks and sorry--I misrecollected. The correct title is Probability Theory: The Logic of Science.

⬐ jcarden

I've got a digital copy around here somewhere. Send me a message if you want.

⬐ sidman

Hi jcarden, i wouldnt mind a copy, where can i message you to.

Thanks !

⬐ woodson

I think it's not exactly the same as the book version, but the author's original is available at http://www-biba.inrialpes.fr/Jaynes/prob.html

⬐ sidman

that's great, thank you, this will suffice for now :)

⬐ ajays

Does anyone have a combined PDF? While I appreciate the author putting out the PDF, separating the files out into individual chapters and figures makes it harder to read in a sitting.

Probability Theory: The Logic of Science by E. T. Jaynes

http://www.amazon.com/Probability-Theory-Science-T-Jaynes/dp...

http://bayes.wustl.edu/etj/prob/book.pdf

http://omega.albany.edu:8008/JaynesBook.html

⬐ pm90

really nice! Thanks!

This is terrible advice. There are almost always unexpected unknowns.

There's a human (irrational) bias to be risk averse, but that doesn't mean it's always safer than we think. We also have selection bias as a counterpoint. There are plenty of similar posts with the opposing view based on the later bias, too.

Instead, let's try to be more rational and understand probability:

Probability Theory: The Logic of Science http://bayes.wustl.edu/etj/prob/book.pdf (draft)

http://www.amazon.com/Probability-Theory-Logic-Science-Vol/d...

Also, Jaynes had a very interesting take on how to address these issues.

Probability Theory: The Logic of Science http://bayes.wustl.edu/etj/prob/book.pdf (draft)

http://www.amazon.com/Probability-Theory-Logic-Science-Vol/d...

See also: Probability, The Logic Of Science, by E.T. Jaynes.

http://www.amazon.com/Probability-Theory-Logic-Science-Vol/d...

Available here: http://bayes.wustl.edu/etj/prob/book.pdf

The introduction/first chapter has a nice example about a policeman concluding a crime is being committed that's very relevant here.

⬐ jdale27

That PDF is only the first three chapters.

⬐ khafra

http://www-biba.inrialpes.fr/Jaynes/prob.html <-- the rest.

the products of a thing often supersede it.

in this case, the development of inference is certainly a consequence of the development of the scientific method. but the validity of the latter is a mathematical consequence of the validity of the former. bayesian inference is more fundamental.

another good book that informs my personal views on this matter: http://www.amazon.com/Probability-Theory-Logic-Science-Vol/d...