HN Books @HNBooksMonth

The best books of Hacker News.

Hacker News Comments on
Thinking, Fast and Slow

Daniel Kahneman · 23 HN comments
HN Books has aggregated all Hacker News stories and comments that mention "Thinking, Fast and Slow" by Daniel Kahneman.
View on Amazon [↗]
HN Books may receive an affiliate commission when you make purchases on sites after clicking through links on this page.
Amazon Summary
Major New York Times bestseller Over two million copies sold Selected by the New York Times Book Review as one of the ten best books of 2011 Selected by The Wall Street Journal as one of the best nonfiction books of 2011 2013 Presidential Medal of Freedom Recipient Daniel Kahneman's work with Amos Tversky is the subject of Michael Lewis's best-selling The Undoing Project: A Friendship That Changed Our Minds In his mega bestseller, Thinking, Fast and Slow, Daniel Kahneman, world-famous psychologist and winner of the Nobel Prize in Economics, takes us on a groundbreaking tour of the mind and explains the two systems that drive the way we think. System 1 is fast, intuitive, and emotional; System 2 is slower, more deliberative, and more logical. The impact of overconfidence on corporate strategies, the difficulties of predicting what will make us happy in the future, the profound effect of cognitive biases on everything from playing the stock market to planning our next vacation―each of these can be understood only by knowing how the two systems shape our judgments and decisions. Engaging the reader in a lively conversation about how we think, Kahneman reveals where we can and cannot trust our intuitions and how we can tap into the benefits of slow thinking. He offers practical and enlightening insights into how choices are made in both our business and our personal lives―and how we can use different techniques to guard against the mental glitches that often get us into trouble. Topping bestseller lists for almost ten years, Thinking, Fast and Slow is a contemporary classic, an essential book that has changed the lives of millions of readers.
HN Books Rankings
  • Ranked #30 all time · view

Hacker News Stories and Comments

All the comments and stories posted to Hacker News that reference this book.
I share with you a bit about what I have learned. I've struggled a lot. Everything is like broken. I'm still struggling right now. However, I'm still working on something to make our situation better. I do several research and experiments on Happiness, psychology, neuroscience and here are something I'm want to share.

+ Hedonic adaption: Hedonic adaption is special psychological effects that explains about how we perceive about happiness. Even after a big happy moment, our level of happiness do down quickly. We adapt our perception to our current situations. So it's like nothing will last forever. Hedonic adaption is both good and bad. It makes us adapt quickly with any situations. It keeps us safe. So we should appreciate it and learn how to make use of this effect rather than blaming it. Learns to attend with everything you do even it's bad, explore something news. It will help you deal with bad effects of hedonic adaptation.

+ Mindfulness: Do some mindfulness exercise. We feel stress because our mind think we're having problems. Our mind made up our feelings to keep us safe [7]. It's good for us. Mindfulness help us understand more about feeling and more enjoy the moment.

+ Mind body connection: Your health affects your mental, and your mental will affect your health. To me, it's not because some spiritual belief, but it's how systems work [3] [4]. Our body, our mind are systems. They are part of bigger system. They connect each others and interact with each other, sending some feedback. So try to improve both your health and your mental. Try to improve your health diet, do exercises and taking care of our thoughts and feelings.

+ We aren't rational. Our thinking system is optimal but it has limitations [3]. It has a lot of problems (cognitive biases). Learn to appreciate and find a way to make it better. For example, we can adapt. We update our belief overtime. Try to make new better habits[5]. Make small steps.

+ There isn't perfect things. Every systems aren't perfect. Our immune system, our cognitive system, organizations, data structures, design patterns,... Appreciate what works, what not and improve it.

Some interesting books, articles you might interest:








This sounds a lot like System 1 and System 2 as described by Thinking, Fast and Slow

Veritasium did a great job summarizing the idea:

Is this just not classic recency bias? Many TV shows feature a gay couple/character, and gay marriage was in the national news for years. People are shown examples through media constantly, and therefore think it's more prevalent than it really is.

This applies to almost all issues too, plane accidents being one of the more obvious ones (plane travel is many times safer than car travel, and yet many people don't see it that way).

Thinking Fast and Slow is a great book that covers at length recency bias and its affects [1]. Quite eye opening to me was one study where people were asked to spin a wheel with 1-100, and then asked how many African nations are in the UN. The number on the wheel had a profound affect on the number people picked [2], despite the fact that the number on the wheel should clearly has no meaning.



I keep ~10 books at my desk. 9 of them are related to Javascript / Python / Probability etc [1]., There is one book though, that I really love to see everyday. Arabian Nights. That was the first book that was gifted to me when I was 11. I always had it with me. It reminds me of my childhood when things get too stressed and I read excerpts out of this book.

[1] [2] [3] [4]

Which translation of Arabian Nights do you have? I've been wanting to read it for a few years now but haven't got around to it. I think this will the next book I read. Any suggestions are appreciated!
For others who might be wondering, it is also known under the title One thousand and one nights (Les mille et une nuits)


I'm not sure where you can get it in the states.

Among the books, you included Thinking, Fast and Slow which really stands out, and I was wondering what you gained from it, and how you'd summarize its relevance/value?

I've been meaning to read it, and I think it's really interesting that it provides enough value to be among the others.

This book has helped me recognize various cognitive biases and heuristics that I or the people around me are demonstrating. Sometimes people say things that just "feel" wrong and this book has helped me identify and name why it feels wrong. It can get a little dry at times, but for the most part the research and examples are memorable. I also find the framework of the "two systems" to be a simple reminder to slow down and think about things that surprise/frustrate me before arriving at conclusions (or responding to that frustrating coworker :P).
I would also recommend You are not so Smart and You Are Now Less Dumb if you like the cognitive bais theme
This is a fantastic book. It's essentially a summary of the author (Daniel Kahneman's) academic career, worth reading because he's one of the founders of "behavioral economics" - the idea that economic-decision making should be studied using real people and experiments--how they do it in psychology--rather than a bunch of mathematical models on a blackboard which may or may not accurately capture human behavior (despite being mathematically usable/tractable).

If you read this book, you'll learn how absurdly influential Kahneman has been: he did the original research on the endowment effect, anchoring, loss aversion, and tons of other stuff you'll see quoted around here all the time. He's also heavily cited by Taleb.

I wish more academics would write like this. It's a hard book to summarize because it's long and completely free of bullshit. It's more or less 400 pages of "here's the question, here's what we did, here were the results, we were surprised because" 20-30 pages at a time. It's an outstanding book by an outstanding professor.

Although the book still has tremendous value (FWIW, I've read it too), I hope more people also read the above blog (& the comment on it by Kahneman himself), to keep a balance of perspectives and the current "replication crisis" in psychology studies.

Kahneman writes:

[quote] What the blog gets absolutely right is that I placed too much faith in underpowered studies. As pointed out in the blog, and earlier by Andrew Gelman, there is a special irony in my mistake because the first paper that Amos Tversky and I published was about the belief in the “law of small numbers,” which allows researchers to trust the results of underpowered studies with unreasonably small samples. [/quote]

Previous discussion:



High Output Management

The Master Switch

Thinking Fast and Slow

Upvote for The Master Switch. It's one of the few books that manages to brilliantly cover a large territory within a small number of pages (<300).
Feb 04, 2017 · mherrmann on A Bitter Ending
Kahneman summarises his and Amos's findings in the book "Thinking fast and slow" [1]. It's a fascinating tour of their discoveries. I'm not normally interested in psychology. But this book is awesome.


The advantage of paper is that you can be more creative and use slow thinking[1]. Just take a look at Newton's[2] or DaVinci's[3] notebooks, how would they think so freely on a computer? Sure for a work task list, a computer is fine, but I find a computer too limiting on my creativity for myself. Added bonus: no ads, bugs, or distractions.

People that create apps create them to make money, not because they make you more productive or help you be more creative.




I have dysgraphia, which is a disability which cognitively affects my ability to write, much like dyslexia affects one's ability to read. Writing is extremely taxing for me, to the point where I will be physically exhausted if I hand write a page of text. I've tried every technology I can think of in an attempt to accommodate my disability. I've been using a computer to take notes for 20 years. I've tried every voice recognition technology you can think of. I've tried every note-taking app I can get my hands on. I've owned more portable devices than I can remember.

In my experience, nothing beats a pen and paper.

I doubt there is anyone on the planet who would be happier to ditch paper than me. While the issues I have with writing are primarily cognitive, writing is far more taxing than typing for me. While writing, Typing avoids any of the cognitive issues I have with physically forming letters -- the computer takes care of that for me -- and spelling and grammar check generally prevents me from leaving words out or jumbling up my word order. However, using a computer FORCES you to write. On a piece of paper, I can draw a diagram (my drawing ability is unaffected) and label key parts. I can make a flow chart. I can draw arrows all over the place. I can literally do millions of things other than writing words on a piece of paper.

Doing these things on a computer is a nightmare. Even if an application has the ability to do any one of these things, it generally pales in comparison to the versatility of writing. Sure, you might be able to move things around on a computer, something that is far more difficult on paper, however I've found it's generally faster to re-draw a flow chart than it is to fix the formatting on a computer if you need to move more than a couple of items around.

Beyond that, your work area with a computer is extremely limited. I have to concentrate so hard when writing that I often forget what I was writing about. I've literally cut up papers I've written in to their individual sentences, spread them on the floor, and rearranged them so that they make sense. I can make a flow chart with thousands of elements, and place it in a place where I can see all of it at once, but also make it big enough to read all of it at once.

Paper isn't all great though. I can't stand writing on paper. It takes me forever. Plus organizing paper is a nightmare. Need to find all of the references you've made to a certain person in the past six months? Prepare to spend a few days combing through your stuff. Backing up paper is time consuming as well, and searching paper back-ups is a huge pain, especially if your handwriting sucks too much for OCR.

Have you tried a pen enabled laptop? Software like OneNote let's you doodle and diagram on a computer, and switch over to typing when needed.
I actually bought a surface pro the first week they were out. I love OneNote, and I though the surface would be everything I dreamed of. I covered some of my issues in a bit more depth in a sibling comment to yours, but my issues with the surface come down to a few things:

1) The pens don't register in the same way real pens do. So mechanisms I've formed for creating legible writing over the past couple of decades of writing are completely useless.

2) I break my surface pens all of the time. The tip splits in half. Plus the point the tip registers at is about 2 mm off from the actual tip of the pen.

3) I require a lot of space to write. The surface pro 1 is a bit too small to write on.

4) I really have to concentrate when writing, and actions like formatting, choosing a pen, moving around the page, etc, take way too much concentration, and cause me to loose my concentration on what I'm writing. This isn't entirely a OneNote problem, it's kind of inevitable for me, since switching from typing to grabbing a pen and writing would be enough to cause me to forget what I was writing.

Honestly, I suspect the assistive technology that will really help would be some sort of AI that can summarize my thoughts for me, so that I could talk to it and have it organize it in a way that I could clean it up later. I've tried Dragon NS and other voice recognition in the past, and it doesn't work at all.

Have you heard of the Livescribe pens? They unfortunately use special paper, but digitize everything you write.

I tried one about ten years ago. It didn't work well for me.

The legibility of my writing is really sensitive to the interaction between the paper and writing utensil I'm writing. I rarely pick the pen up off the paper -- instead I use the fact that certain pens can 'skate' across certain types of paper to write in a block print that is written kind of like cursive. It's completely legible to me, and mostly legible to other people, without being too fatiguing to write.

The 'skating' effect is created by abusing the shoulder that holds the ball in a ball-point pen in place. If you look at my hand writing, there are actually depressions from the pens between the letters without ink in them, because I dragged the edge of the pen between the end of one letter and the beginning of the next.

My writing is really sensitive to both the paper I'm writing on and the pen I'm using as a result. If I write on smooth paper, like a glossy card stock, I absolutely have to write in draftsmans letters or my writing skitters all over the place. Certain pens have the shoulder in a different place, causing the connections between letters to have ink, or making some letters not appear.

Most digital writing implements don't work at all. I hold my pen at a relatively severe acute angle to the page, because I'm dragging the pen really close to the shoulder that captures the ball in a ball point pen. A lot of digital writing tools require the tool to be used more upright.

Oftentimes the issue is caused by a button' that needs to be pushed by the tip to activate the detection, and I'm binding the mechanism up because the force I'm applying to the tip is extremely off-axis. In my surface pen, in addition to this issue, the sensor is about 1.5 mm from the actual tip of the stylus, causing everything I write to be shifted rather far from where I'm intending to write.

Beyond that, I press very hard and tend to break even well-made pens. With real pens, I generally crack the tip of the pen, causing the ball to either come loose or bind inside of it. When this happens I throw out my pen and get a new one. Most styluses are much more expensive, and for whatever reason tend to be made out of much less sturdy stuff. I have broken at least 5 surface pens, which are not cheap.

The actual shape of the pen makes a difference as well, and when I tried livescribe I found the pens to be nearly impossible to use because of the shape. The occupational therapist I used to work with thinks this is probably due to a missing tendon in my right thumb, rather than dysgraphia, but it's an issue I've faced in the past.

I worked with that occupational therapist for a long time on my writing, and we set a goal of making my letters and words legible to me, and my numbers legible to everyone. I've basically achieved that goal when using pen and paper, but I have yet to find an assistive technology that doesn't make my handwriting look like a giant scribbly blob.

Have you tried a Wacom tablet? I think you can hold it and write even in the air.

I have bought 3 (upgraded to better models) in this past decade, couldn't be happier.

Ha :) I don't think it gets better then what you just wrote.

I think it is mostly due to tools on computer being bad. When I use writing app like IA Writer, or simple outliner like Outlinely or Vim plugin... I get a lot from that, mostly because I can type fast.

And I love everything paper and pens/pencils.

Money is a decoupling mechanism. An interface.
True, but it's also an incentive. If there were a way to compensate app developers for actual productivity gains then the incentives would be more aligned. This may actually be possible for bigger ticket, B2B apps. Maybe some sort of share-the-risk arrangement tied to a consulting engagement.
I think what they're describing is that there can be a problem in that the pursuit of compensation causes the disalignment of incentives. "Pay me to solve an artificial problem."
For me, the benefit comes from two limitations:

- The slow and physical aspect allows you to think about and consider the idea, as you're writing it.

- The negative reinforcement, for that slow consideration, caused by from the permanence of your mistakes or a tired hand.

But, I don't think paper has to be involved. I see it as an indication of a lack of good stylus input in most devices.

I now use a large iPad Pro, with the stylus...err...Apple Pencil, and have no desire to go back to paper. Having the pages backed up to the cloud, being able to insert links and media when necessary, and being able to quickly switch colors, is all too valuable.

If good stylus input gets cheaper, I don't, personally, see a justification for paper.

I, too, possess an iPad Pro with Apple Pencil, but I opted for the smaller 9.7" one. It was a mistake, the bigger one would have been better, but I love it anyway. for note-taking. It's a godsend to be able to shuffle text around, resize it, undo my errors etc. I wrote something on paper lately and automatically searched for the undo button when I made a mistake and was frustrated when I realized I have to use my eraser.

Anyway, which app are you using for your notes? I'm using OneNote ATM. I can't describe what's missing, I'm just not feeling 100% satisfied.

That's interesting. I've got the larger iPad Pro that I take to meetings for notes. My coworker has the 9.7" and his seems more manageable / not as cumbersome. Perhaps it's due to our meetings being in large community rooms without tables, but there's something that's not sitting right with me. The grass is always greener, I guess.

As far as apps, GoodNotes is where I landed. It's handwriting recognition / search is pretty good, and I like the way it handles importing PDFs and Word documents.

I also use GoodNotes. I chose it because it's relatively low input latency and has handwriting recognition.
I tried both the 9.7" and 12" before I bought my 9.7". For me, while the 12" is fantastic for the available space (and the side by side tile feature is also awesome on the bigger screen), I found the size and weight meant I wouldn't want to carry it around with me wherever I go and is harder for me to hold eg on a bus. The 9.7" is small and light enough to be portable, which was important to me.
Notability works well for me.
I'm on a surface book, but have a similar issue. OneNote is the best I've found, but my issues are a few things:

- Something about how changing color works doesn't quite feel right.

- I can't shuffle pages around, overlap them, etc.

- I want either infinite page size, or fixed, not the weird 'expand when you write near the edge' thing I have now.

I think I almost want a 'virtual desk' kind of thing that I can shuffle paper around on, make things overlap, etc. All the stuff you can do with paper, with the added benefit of being able to save stuff. Organization could be a serious issue though.

Once upon a time, before even the first iPad was released, the Microsoft Courier promised this:
One thing that will be a factor for a long time is that a piece of paper doesn't require integration to share it with other people, or a battery to be charged etc. It's just a real object representing your ideas.
> But, I don't think paper has to be involved. I see it > as an indication of a lack of good stylus input in > most devices.

That was going to be my reply to your first comment. I'm still using a Note 3 with its Wacom stylus, and I cannot ever imagine switching to a phone that requires a capacitive stylus.

Pro tip: The absolute best stylus that I've found for capacitive screens is a fresh, stiff cucumber. The water in the cucumber is detected by the screen just like your finger. Carrots work well too, but they are heavier for any given size.

I'll also add that having tried lots of alternatives, the iPad Pro with pencil is the only thing that comes close to real pen and paper (but still falls a bit short)
I agree completely. I bought an iPad Pro (although the smaller one for convenience of carrying it with me wherever I go) and an Apple Pencil and love it. I use it for sketching ideas, diagramming, note taking and brainstorming. And if I want to write a lot of text, I can type with a physical keyboard too (I'm a faster typist than I can hand-write).
I also have switched entirely to a big iPad Pro + pencil. Works great. Notability also lets you record sound and can play it back in sync with your notes.
I've tried iPad Pro; but went back to using frixion (erasable) pens:

my favourite being the 4-color (w/ variety of colors to choose from) with very fine line 0.5 mm:

p.s. could be combined with even finer line 0.38 mm from this pen costing ~ $1.5 :

For more official occasions these ones are nice:

Why specifically this brand? Is it special, and is the difference between an avg random pen big?
You'd be surprised. Pilot G2 is a great writing pen, and I'm personally a fan of Uniballs.
Frixion pens are erasable.

The ink turns invisible when heated. The pen has a rubbery tip on top which acts as an eraser when rubbed on the ink.

0.5/0.38 a very fine line? I'm using Staedtler pigment liners which are 0.05. The downside is that my handwriting has become positively tiny...
0.5 is pushing it, but for normal size writing 0.38 is a sweet spot of fine for most people. I use a 0.3, and I immediately notice bad paper because it becomes somewhat scratchy. This is in gel pens.

Looking at jet pens, yours seems to be a marker pen, which generally needs to be thinner so it won't bleed. Similar to how ball points need to be fat (0.7mm is called fine) to write smoothly and make a dark enough line.

If you are interested in learning how your brain makes decisions, where biases and error come from, and a whole lot more, I highly recommend this book by Nobel Prize winner Daniel Kahneman:

Interestingly, he won his nobel prize in the field of economics, but he's a psychologist, not an economist. His research was so influential that it changed business strategies (esp. around how meetings are held) forever.

I can't say enough good things about Thinking Fast and Slow. Go read it.

I posted this article in reply to another comment in this thread, but I think many will find it interesting and useful. It's a good jumping off point into his research and why it's important.

Do you have any examples around how meetings have changed? I'll need to relay that information promptly...

"The principle of independent judgments (and decorrelated errors) has immediate applications for the conduct of meetings, an activity in which executives in organizations spend a great deal of their working days. A simple rule can help: before an issue is discussed, all members of the committee should be asked to write a very brief summary of their position. This procedure makes good use of the value of the diversity of knowledge and opinion in the group. The standard practice of open discussion gives too much weight to the opinions of those who speak early and assertively, causing others to line up behind them."

(Following this advice is rare; I think it's bold to say that Kahneman's research has already changed meetings.)

> I tend to not be convinced by self-improvement recommendations that aren't explicitly tailored to specific people and their personalities. There are so many variables in play!

I tend to agree with this sentiment especially considering I have read and tried to apply quite a few in my own life.

However, I am currently working my way through Thinking, Fast and Slow[0] and I can't recommend it enough. It's not so much a self-improvement/help book as it is a way to define the language we use in speaking about the different systems of the brain (think intuitive vs effortful). Worth the read.


Kahneman's book, Thinking Fast and Slow[0] is a great read if you like to think about memory and how thinking functions in practice

[0] -

Thinking, Fast and Slow by Daniel Kahneman:

Great book about the way humans think.

The phenomenon is called anchoring[1], and it works like this: the first price we see anchors our thinking about additional information. This is why furniture stores have a perpetual sale.

"Oh look, this $2500 sofa is only $1899.99 right now. What a great deal!"

One experiment had respondents use the last two digits of their social security number as the initial price for a bottle of wine or other good. This completely-arbitrary price had a strong correlation with the price they were willing to pay for the item.

The effect is described in greater detail in Thinking, Fast and Slow[3]




I wonder how well the `social security number` anchoring study replicates?
Predictably irrational by Dan Ariely has a good number of case studies that I think illuminate the topic particularly well:
Unsurprisingly there's some sort of combo deal to get Thinking Fast and Slow with it.
My link [2] actually has a video of Ariely talking about this stuff.
If you remove the spaces before that link it will be clickable.
Ugh, I always forget that I shouldn't do that here (I move URLs out of the way like this for email) and its been too long to edit.
Oct 21, 2015 · striking on Sam Altman's Twitter AMA
For the question "What are some of the best books to learn from that you recommend for a young startup founder?", I decided to transcribe the answers.


"Zero to One: Notes on Startups, or How to Build the Future" -

"Republic" - (classic, feel free to grab a PDF)

"The Principia : Mathematical Principles of Natural Philosophy" - (classic, feel free to grab a PDF)

"Thinking, Fast and Slow" -

"Molecular Biology of the Cell" - (different edition, forgive me; free through NCBI, thanks jkimmel!)

"Dealers of Lightning: Xerox PARC and the Dawn of the Computer Age" -

"The Supermen: The Story of Seymour Cray and the Technical Wizards Behind the Supercomputer" - (note: "that one's particularly good")

"Hateship, Friendship, Courtship, Loveship, Marriage: Stories" -

"The Score Takes Care of Itself: My Philosophy of Leadership" -

"The Beak of the Finch: A Story of Evolution in Our Time" -

"The Constitutional Convention: A Narrative History from the Notes of James Madison" -

"The Art Of War for Lovers" - (fixed! sorry about that...)

"Hold 'em Poker: For Advanced Players" -

"Solution Selling: Creating Buyers in Difficult Selling Markets" -

"The Endurance: Shackleton's Legendary Antarctic Expedition" -

"Winning" -

I wish he had answered in text. That would have made things easier :) However, I'm still very happy to have some new additions to my reading list!

>"The Art Of War"

It actually is "The art of war for lovers" by Connell Cowan

Thanks for correcting me. Those blurry frames can be tough.
Yeah, I thought it was the Sun Tzu first too, which could as well be on his list. Thank you for compiling them.
Molecular Biology of the Cell (Alberts) is free through NCBI! Many investigators jokingly refer to it as 'the bible'.

Ooh, much appreciated! Especially since a new hardcover is ~$150. Edited to note this.
do you see any reason he put this book in the list?
Unfortunately, it doesn't look like you can read it cover to cover or download it.

  By agreement with the publisher, this book is accessible by 
  the search feature, but cannot be browsed.
This is true. However, you can get to any topic you want easily using the search feature. A Table of Contents is provided, making front-to-back reading by topic pretty trivial.

I've taken many university courses using this book and managed to read all the required material on NCBI without much effort.

This is great! Do you mind if we add it to the HashFav Page? We will credit you.
You might want to make josu's correction above ^
Thanks, we just did.
I don't mind at all. Glad to be of service.
im surprised to see Republic in here
The Art of War one looks more like "The Art of War In The Middle Ages" by C.W.C.Oman

This is correct--art of war in the Middle Ages.
I felt that much of your narrative rings true with me as well. I found this book to be quite amazing and it gave me an interesting perspective on my thoughts:
Favorite quote from the above paper: Albert Einstein once said, “We can't solve problems by using the same kind of thinking we used when we created them.” However, it is possible that the unconscious mental system can, in fact, do just that.

Related reading : Thinking fast & slow [1]


Having a 30 minute subway commute I'm looking forward to seeing what suggestions come out of this thread. My two suggestions:

1. Thinking fast and slow - Understanding how we actually think as opposed to how we think we think is a critical skill, especially in a startup. Having a Nobel prize winner explain how the two systems of your brain work together (and can sabotage you) was enlightening and enjoyable. This book helped me understand many aspects of design and sales that had been black boxes for me

2. Art and Fear - This is a book nominally about the relationship between artists and how they go about making art but it is useful for anyone creative. It's about how to go about making when you have errands to run, a deadline, or just don't feel like it. As a dev I found it inspiring

"Thinking Fast and Slow", was definitely the most impacting book for me last year. Highly recommended.
Has anyone read this book and for those who have is there a correspondence here?

I don't think it has a direct correspondence in terms as you might be implying i.e. the fast in this context is mostly still the slow Type II reflective thinking DK refers to. The "fast" coders are making small iterative changes but that doesn't mean the thinking is automatic, just localised and frequent.

The deeper analytical and design oriented thinking where someone takes extended time to identify goals, prototype and review different implementations is an important variation of Type II but not really addressed in TFAS. There might be plenty of Type I micro-decisions involved in that process too.

I'm pretty interested in how cognitive biases do rear their heads in design problems. I'm scared silly by framing effects where given options A and B, an individual chooses A but adding a third option C and now they would choose B. There are so many decisions that crop up during design/development that are vulnerable to those issues.

Oct 28, 2014 · jacobn on We Are All Confident Idiots
And this is why software schedules are always off. Any schedule for that matter.

If you're into this type of psychology, check out "Thinking Fast and Slow" by D. Kahnemann.

"How to spot the fake answers put there to fool you" == "how to see when an answer isn't even in the ballpark". That's a useful skill. That "context clues" thing suggests teaching students how to solve the problem in front of them, not the easy problem their mind wants to substitute for it [1].

From what this guy describes, "test prep" sounds like "educating students".

I agree that grading of essays is a disaster. It's not specific to standardized tests, however - that's how all my essays were graded from grade 1 all the way to college.

Another way to game the test is on the test giver's side - they get to define what the pass/fail thresholds are.

This is why tests are standardized, not left up to the schools or teachers.

[1] A hard problem: "What is the optimal incarceration time to dissuade people from pedophilia." An easy problem: "How angry do pedophiles make me feel?" When most people hear the first question, which is hard, their mind substitutes the second much easier question for it. See Kahneman's book "Thinking Fast and Slow" for more on this.

Ironically, standardised testing is a prime example of a system choosing to solve an easy problem instead of the problem it has.

Hard question: are our schools meeting the educational needs of their pupils and society?

Easy question: did enough pupils fill in enough of the right circles on this test for us all to not get fired next year?

If you exchanged test scores for production figures and teachers for farmers, this article could be a story directly out of Soviet agriculture in the collectivisation era. I'm finding it highly entertaining to watch you, of all people, defending this system so vehemently.

If you don't have standardized testing, how do you know if the students are being educated? More specifically, how do you identify schools that need help and focus resources effectively?
Nobody said anything about not having standardised testing. The problem here is high-stakes standardised testing.

You can know stuff about a system that you can't measure completely (i.e. any system) by sampling it - you'll get a lot of noise and even some systematic biases, but as long as you maintain an awareness of that you will know something.

The problem comes when you want to control things. If you create a feedback loop by attaching strong incentives to the measures you are using to acquire knowledge, then you end up with neither control nor knowledge. You're no longer taking a representative sample, just measuring the gain of your feedback loop.

And if you apply the incentives at a granularity of measurement such that noise overwhelms the signal (by a factor somewhere between 7 and 100, according to the article)...

The problem is that standardized testing doesn't tell you if students are being educated, it tells you if students are passing standardized tests, which is a different problem that, at best, only partially overlaps with the actual question. If you over-optimize for standardized testing, you end up under-optimizing for actual education.

I had to take the WASL as part of my graduation from high school, back in 2003. Forcing this requirement on schools did not improve our education, it degraded it. Instead of actually reading two or three good books in our english class, we studied cliff notes of 15 or so works, so we could write shallow summaries of "key themes" for the test, for whichever books actually appeared on it. Instead of going forward with trigonometry in math, we went over estimation and hammered on very basic geometry problems. My government class stopped covering its subject matter entirely, and we studied analogies and reading comprehension, because government wasn't on the WASL (at the time; it might be now). The net effect was that as a class we did really well on that test, but our actual education suffered. That's what standardized testing gets you.

The problem is that standardized testing doesn't tell you if students are being educated

But what will?

I program for a living, but I've not taken any programming tests since I was in school. How do people know that I'm doing a good job without my taking a standardized test?

The answer is one we've had for a long time - talk with the teachers. Teachers are professionals, paid to educated children and evaluate what needs to be improved. We developed normal schools to train people how to be teachers. These became known as teachers' colleges, and then became education programs in a university.

In addition to continuing education programs and peer development, we also have oversight programs in place, including the department head, principal, and local school board. Among other things, these are supposed to help identify teaching problems and remedy them.

Unfortunately, management is both support and punishment, which can lead to power imbalance where a school board member says "My nephew must be on the football team or else you won't get a raise next year!" One way to limit this power imbalance is to set up a teacher union or tenure system. Another is for additional community oversight, which may include the parent-teacher organizations like the PTA.

Therefore, your question sounds like you trust the authors of standardized tests (who are often in for-profit companies that sell the tests, sell standards, and sell text books which match the standards) more than you trust teachers or the professional education system.

Why is that, do you think?

Honestly, I have no idea if you're doing a good job. At least (as far as I know) you're not asking for federal funding for your job.

Well, I don't have to trust the authors since I can see and evaluate the test for myself. I can't really evaluate every teacher and analyze the pressures on them.

Education is primarily state funded, not federal. I believe federal funding is only 10% of the local school budget, and includes meal assistance and other things which aren't directly tied to a teaching position.

In any case, your original question is also valid for private schools - how do the parents of private school students know if their children are being educated or if the schools need help? How does the bishop overseeing several Catholic schools do the same?

Therefore, why is "federal funding" relevant to the topic?

As I understand it, you don't have access to the questions and answers for the high stakes tests, so you can't evaluate them. I can be proven wrong. Can you show me the complete set of questions for a state test from last spring? I looked for Florida, and only found FCAT tests from 2005/2006 at . I could not find FCAT 2 questions from 2013 or 2014, though I did find the scores from .

This is what I expected, because some of the questions are potential questions for future tests, and exist to calibrate the tests. If the questions and answers are published, then they can't be used that way.

Which suggests that you don't know what you're talking about, as regards high stakes testing, or that there are some states where all of the tests are published, so that people like you can review them. Which tests are you thinking of?

If I understand you correctly, you are satisfied if you can "see and evaluate the test." Wouldn't you be similarly satisfied if you could "see and evaluate" all of the tests from each teacher at every school? Since that seems a lot cheaper and easier to do than set up high-stakes testing across the country.

You're right about most of the funding but NCLB has provisions to redistribute federal money to specific schools. And while you don't get specific questions, you can see example tests to see what subjects are covered, how much is multiple choice vs essay, etc.
Yes, the feds contribute some of the money in exchange for a lot of the rules. That doesn't change anything of what I said - your original question is independent of federal involvement and could equally apply to privately owned Catholic schools.

Have you changed your viewpoint? You previously said "I don't have to trust the authors since I can see and evaluate the test for myself." Now you're okay with seeing only a synopsis of what's in the tests? Why do you still trust the authors if you can't see the actual test?

If you could get the same synopsis of the questions that the teachers ask, then wouldn't you also be satisfied? Why not?

I'm actually kind of disappointed that there's not more information available. It still seems better than what we got before, which was even less informative. And it would be niceto have that synopsis, but it's much more useful to have a standard so we can compare across schools.
I'm completely bewildered. I said:

> your question sounds like you trust the authors of standardized tests (who are often in for-profit companies that sell the tests, sell standards, and sell text books which match the standards) more than you trust teachers or the professional education system.

> Why is that, do you think?

You answered that it's because you could see the test questions, and evaluate them for yourself. Then you said it's because you could see samples of the questions. Now you say it's because you can compare scores?

Curriculum standards have been around since the 1800s, so is "have a standard" short for "have standardized tests"? Actually, we've had those for decades - my birth state of Florida started them in the 1970s, so I assume you mean "have high stakes standardized tests"? Actually, Florida also introduced the nation's first required high school graduation test in 1977", so you must mean "frequent high stakes standardized tests", yes?

How is this more useful than earlier assessment tests, as well as GPA, SAT scores, ACT scores, graduation percentages, number of students going on to the Ivy League/Big 10/whatever, number of National Merit (semi)finalists, number of available AP/IB courses, average AP score results for a given field, lists of extracurricular activities, football team scores, and a lot of other cross-school comparison metrics?

Again I ask, what is the basis of your trust of the authors of a standardized test over the teachers and the professional education system?

How do people know that I'm doing a good job without my taking a standardized test?

Measure your outputs, ideally with unit tests, manual testing and the like. Does your code do what it's supposed to?

Standardized tests are basically unit tests for teachers - they measure whether the teacher's outputs are capable of reading and writing. with the teachers.

Note that the test manufacturer has no such problem.

There are a couple of other branches in this thread waiting for your followup for the last few hours, and you pick this one? I still want to know if you think that "proficiency in the subject matter" is defined as "ability to pass a standardized test."

You think neither I nor the entire education system over the last 150 years have ever considered the effect of the principal agent problem? I even said "we also have oversight programs in place, including the department head, principal, and local school board." I elsewhere also pointed out to you how test manufacturers stand to make a profit if they can convince people to buy their tests, curriculum, and text books. They most certainly have a bias.

Your comparison to unit tests is telling, in ways you didn't mean it to be. Every project I've worked on has a very different set of unit tests, with essentially nothing shared between the different test cases outside some common test infrastructure.

Even multiple people on the same project end up writing different sorts of unit tests for the same code base. I do more functional and coverage driven tests, a co-worker is a red-green-refactor TDD developer. This diversity of tests is probably better for the overall code base than if we all did the same thing.

You do realize that teachers almost certainly have studied assessment design as part of their coursework, while most developers have almost no formal training in test engineering or experience in, say, coverage analysis?

If the goal is to test the students, then the teacher can - like the developer with good test engineering skills - develop the appropriate tests for the given set of students and expected knowledge. Except the teacher's tests must also be engaging and authentic, while the computer doesn't care what it runs.

And yet you think that one single set of unit tests for, say, all 8th grade English teachers can be useful enough to judge a specific student's progress, or a specific teacher's skills? Where does that optimism of yours come from?

>"How to spot the fake answers put there to fool you" == "how to see when an answer isn't even in the ballpark". That's a useful skill. That "context clues" thing suggests teaching students how to solve the problem in front of them, not the easy problem their mind wants to substitute for it[1]. From what this guy describes, "test prep" sounds like "educating students".

OK, sure, it does sound like educating students, IF you accept that teaching people to spot inconsistencies and trap questions in a ritualized multiple choice test is a skill that transfers to other situations. Unfortunately this is really not the case at all. It's an education in how to navigate specific public-school bureaucracy. It doesn't even teach people to navigate other shitty bureaucracies.

> From what this guy describes, "test prep" sounds like "educating students".

It's educating them in the wrong things. For example, goal of class: better thinker by learning to do multiplication. Because of test, they only learned memorizing multiplication tables.

The tests ask for multiplying 3 digit numbers. That's a lot of memorizing.
that was a simple example. A better example might be memorizing quotes from a book to regurgitate in the test*

* (I had to do this as part of GCSE English in the UK - I expect a similar situation in the US)

Why don't you look at the real tests and see if such tricks are possible?
> "How to spot the fake answers put there to fool you" == "how to see when an answer isn't even in the ballpark". That's a useful skill.

It's a probabilistic skill, and in so far as the test is designed to give a measure of what someone knows, and not how lucky they've been, it's gaming the system.

Is it a useful skill? Well, yes. To an extent. So is knowing your addition table, but we expect students to have moved somewhat beyond that by the time they're in secondary education. Just as we'd expect someone educated for five years, six or seven hours a week, 28-40 weeks a year, to have advanced somewhat beyond the need for discarding comedy answers as a viable test strategy.

By secondary level, we'd expect them to know, (or be capable of running the calculation,) to decide among those answers that are actually in the ball park. Approaches for which the cost is more or less constant regardless of how many answers are on the page: You trust your calculation or memory to have given you the answer and discard all others by default.

The thing you are blatantly ignoring is this: standardized tests also teach children the lesson that there will be exactly one correct answer in all of life's situations, and that there will be exactly one OK method for every possible challenge faced. No exceptions.

Which is sometimes true sure. But it is frustrating working with academic "stars" who internalize this once they come to the "real world". They are focused so much on getting the "right answer" within some narrowly defined context of right, that they can't see the better solution by reinterpreting the problem or recasting the assumptions in a slightly different order. They are unable to combine bits of knowledge from different buckets, because the test questions are all neatly siloed.

For example, I've had this argument with fresh grads many, many times:

me: you need to limit your UDP packets to 512 bytes (or 8K depending on the situation).

them: but my teachers told me UDP packet size is a 16 bit integer.

me: yeah, but many stacks cut off shorter, because there is a different standard that says routers can drop packets bigger than their preferred size, the only minimum is 512 bytes.

them: my teacher told me that the packet size is a 16 bit field. Why are you talking about routers?

me: because you need to combine information to actually solve a problem?

them: whatever, I need to figure out what the bug in my code is causing these packets to be dropped.

Or -

me: hey $intern, let's figure out a few ways to solve approach this problem. expounds on the problem, lays out a few things that might work. The goal here is to try a few different techniques so we can work them into the bigger design. Any questions?

intern: no.

a few days later

intern: Hey I think i solved the problem, is this one solution right?

me: it's one way. It has some good stuff and bad stuff, but we want to try a few solutions to determine how to think about this.

intern: looks like a lost puppy but is it right?

Following conversation about multiple solutions and exploring solutions resembles "who's on first"

The biggest problem with standardized testing is there is no room for the idea that outside of school, it isn't always about doing the rote thing, the simple siloed task in front of you, but rather incorporating various bits of knowledge, about applying the bits of knowledge in ways that allow task completion for tasks that aren't extremely well defined with a pre-arranged solution.

In fact - the lack of a pre arranged solution is what defines most work outside of menial jobs. The idea that there is more than one approach or solution to something is antithetical to the core of standardized testing.

(keep in mind - that for the statistics to be meaningful, the tests can't allow for grading criterial other than "one strictly correct answer" or you end up with issues in the numbers as the result of graders being different.)

I don't think standardized testing has that much impact on how people think. I suppose test prep teaching is less likely to break people out of lazy thinking, but I don't think it inculcates it.
There's pretty strong evidence suggesting this isn't a problem. Or if it's a problem, it's not one that schools can solve.

Teaching "critical thinking" is basically a waste of time. You can't do it. It would be nice if you could, but you can't. "Critical thinking" simply doesn't transfer. (Well, they do a tiny bit, if they are done right, but there's more fine print than Facebook's ToS to any claim that you can teach students how to think.)

Let's say you took all those "creative thinking" skills you learnt in networking, did a course on photography, then got a job with a really good photographer. Guess what - you might have decent communication skills, but you'd still come off as a clueless idiot who can't "think creatively" or "solve problems", because you don't have the domain skills and knowledge.

If they've got a solid core of domain skills and knowledge, they can actually think for themselves. If they don't, they'll be clueless, and just try to memorise answers.

Anyone who can tie their own shoelaces knows "there's more than one way to solve a problem". Kids can actually think for themselves, if and only if they understand the domain.

Now, maybe the schools are teaching really badly, and the tests are geared towards forcing students to answer questions rather than solve problem - that's a problem. As in machine learning, getting students to memorise training data just leads to brittle learning. That might be the real problem - the blind are leading the blind, and some teacher who can't network is telling kids to memorise whatever was in the book, because no-one in the class has a clue. That's a recipe for incompetence.

And we know that high stakes tests with rewards for "good" teachers are like paying programmers per LoC. But that's not a problem with standardised tests anymore than code metrics are a problem. Idiots in management can cause issues, though.

I disagree completely with "you must already have domain knowledge to be able to apply basic learning skills within that domain". I've seen people enter new domains and do well, and other enter new domains and do poorly. The difference seems to be the ability to ask "how do the things I do already know interrelate?"

It is a matter of metacognition (thinking about what I know and how it applies) and not being paralyzed by fear of "getting the wrong answer". The former can be taught, and there are teaching methods that show success around the concept. The latter is something that is hard to overcome when people spend 16 formative years being punished when they don't "find the exact, single, and exclusive" answer and not being rewarded for "learning a few ways". (although research also shows that tests that are not binary - that is all points or no points - do a good job of helping with the fear e.g. multiple choice tests that have "wrong" answers that suggest conceptual understanding even if there is a calculation error.)

I defined what it meant to game a test, and gave definitions which fit the example. You have rejected my definition, without giving an alternative.

"That's a useful skill" is not a useful educational criterion. Knowing how to change a tire on a car is a useful skill, but it's not appropriate for a math course.

"Educating students" is also a useless criterion. Education is a never-ending process. I'm still learning things now. Schools by necessity must restrict themselves to certain topics. A Spanish teacher cannot simply use "I'm educating students" as an excuse to spend four weeks on Canadian politics in the 1970s.

If test prep is so important, why isn't it its own course, where the teachers are trained for it, and where there are specific curriculum goals?

"This is why tests are standardized, not left up to the schools or teachers" - are you willfully ignoring the point? Someone defines the standards. The page I linked to suggests that the standards for this New York test were defined by people who want the public school systems to fail, as part of the general effort to privatize public school.

As points out:

> The bottom line is that there are tremendous financial interests driving the agenda about our schools — from test makers, to publishers, to data management corporations — all making tremendous profits from the chaotic change. When the scores drop, they prosper. When the tests change, they prosper. When schools scramble to buy materials to raise scores, they prosper. There are curriculum developers earning millions to created scripted lessons to turn teachers into deliverers of modules in alignment with the Common Core (or to replace teachers with computer software carefully designed for such alignment). This is all to be enforced by their principals, who must attend “calibration events” run by “network teams.”

You even used the passive "that's why tests are standardized" - who standardized the tests, what political and financial goals influence them, and how transparent is the standardization process?

When the schools and teachers define the tests, then these issues are much clearer, and any failures are limited to just the school or teacher, and not systemic to the entire state.

Hence why it's possible, and likely, that some test standardizers have gamed the test results, under my concrete definition of "gaming."

I agree that if you want to define education in the course material as "gaming", the tests can indeed be gamed. Knowing how to reject an obviously false answer (e.g., "23.6 x 10.9 = ? a) 1,000,000, b) 0.000000001, c) 257.24 d) 527.24) is part of learning math.

As for who defines the standards, the answer is our politicians or whoever they delegate to. And if you define gaming as "defining a standard", then yes, the test creators also game the system.

So far, you haven't actually pointed to any part of the standard that you object to. Nor have you pointed out any sort of gaming other than "teaching the material on the tests, including how to ballpark answers".

I defined gaming as "aspects which influence the test results other than proficiency in the subject matter."

I did not define it as "education in the course material". Please don't make that assumption.

It's impossible to evaluate your example without defining the pedagogical goal. Your example test question cannot distinguish between proficiency in multiplying two three-digit numbers, and proficiency in selecting from one of four possible answers, where two are obviously incorrect.

That said, this question is biased in favor of students who have been taught estimation techniques, in this case, round, compute 20 * 10, and look for the closest answer. They will be able to answer more of these types of questions than students who can actually multiply the numbers, but haven't learned the approximation methods.

Had the answers been "1) 257.24, 2) 256.24, 3) 247.34, 4) 248.34" then the other class of students would fare better. Then again, those who learned casting-out-nines would be able to reject two of these quickly.

It's clear that sometimes ballpark answers are better than exact ones. In bookkeeping, it's clear that exact answers are better. It's possible to teach students both ... by taking time away from other skills which are also part of mathematical proficiency. A standardized testing system encourages monoculture teaching, so that all students are primarily taught the method most likely to be on the test, on the assumption that the test defines proficiency.

It appears that you have defined "proficiency in the subject matter" as "ability to pass a standardized test." If so, then by definition it's impossible to game the system, making this discussion pointless. Is that your definition of proficiency?

P.S. Here's another way to game the test system - expel your worst students before the state tests. In that way, your school gets the money (for the student) but doesn't have to be responsible for the poor grades, or even make an effort to educate them. See for examples.
> I don't get how referencing a singular event often is actually an issue.

Because it's irrational and doesn't represent the real probability of an event happening again. The argument is therefore that we are shaping policy (with ramifications on economics, privacy and politics) based on poor statistical analysis. I'd recommend reading Kahneman's Thinking, Fast and Slow if you're interested in understanding how irrational our minds are.

I wonder if that makes you more conscious of them.

I'm thinking about a study cited in Thinking Fast, Thinking Slow[1] where the participants did better on a test when the questions where in a blurry font. I believe it was suggested that being forced to exert mental effort to read the questions forced their brain into "actual thinking mode" as opposed to "pattern recognition" mode.

ie. If we know it's probably not an ad, but it looks like an ad, is it a more effective bit of non-ad? (well, it is an ad, but it's an ad we want to see.)


Thinking Fast and Slow at least mentions many similar experiments and the results are often interesting, so it is probably worth reading if the topic is of interest (and it probably should be).

I have't looked at the actual research behind the topics they are covering, but at least there seems to be a good reference list in the book so it should be possible dig out the articles.,_Fast_and_Slow

HN Books is an independent project and is not operated by Y Combinator or
~ [email protected]
;laksdfhjdhksalkfj more things ~ Privacy Policy ~
Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.