HN Books @HNBooksMonth

The best books of Hacker News.

Hacker News Comments on
Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are

Seth Stephens-Davidowitz · 3 HN comments
HN Books has aggregated all Hacker News stories and comments that mention "Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are" by Seth Stephens-Davidowitz.
View on Amazon [↗]
HN Books may receive an affiliate commission when you make purchases on sites after clicking through links on this page.
Amazon Summary
An Economist Best Book of the Year A PBS NewsHour Book of the Year An Entrepeneur Top Business Book An Amazon Best Book of the Year in Business and Leadership New York Times Bestseller Foreword by Steven Pinker, author of The Better Angels of our Nature Blending the informed analysis of The Signal and the Noise with the instructive iconoclasm of Think Like a Freak, a fascinating, illuminating, and witty look at what the vast amounts of information now instantly available to us reveals about ourselves and our world—provided we ask the right questions. By the end of an average day in the early twenty-first century, human beings searching the internet will amass eight trillion gigabytes of data. This staggering amount of information—unprecedented in history—can tell us a great deal about who we are—the fears, desires, and behaviors that drive us, and the conscious and unconscious decisions we make. From the profound to the mundane, we can gain astonishing knowledge about the human psyche that less than twenty years ago, seemed unfathomable. Everybody Lies offers fascinating, surprising, and sometimes laugh-out-loud insights into everything from economics to ethics to sports to race to sex, gender and more, all drawn from the world of big data. What percentage of white voters didn’t vote for Barack Obama because he’s black? Does where you go to school effect how successful you are in life? Do parents secretly favor boy children over girls? Do violent films affect the crime rate? Can you beat the stock market? How regularly do we lie about our sex lives and who’s more self-conscious about sex, men or women? Investigating these questions and a host of others, Seth Stephens-Davidowitz offers revelations that can help us understand ourselves and our lives better. Drawing on studies and experiments on how we really live and think, he demonstrates in fascinating and often funny ways the extent to which all the world is indeed a lab. With conclusions ranging from strange-but-true to thought-provoking to disturbing, he explores the power of this digital truth serum and its deeper potential—revealing biases deeply embedded within us, information we can use to change our culture, and the questions we’re afraid to ask that might be essential to our health—both emotional and physical. All of us are touched by big data everyday, and its influence is multiplying. Everybody Lies challenges us to think differently about how we see it and the world.
HN Books Rankings

Hacker News Stories and Comments

All the comments and stories posted to Hacker News that reference this book.
> Is this actually useful?

IKR.

Pennebaker thinks so. He's a big proponent of "computational linguistics".

https://en.wikipedia.org/wiki/Computational_linguistics

I recently read his book Secret Life of Pronouns.

http://secretlifeofpronouns.com

I thought it'd be fun to replicate the book's claims. Like maybe analyze political speeches.

But Pennebaker's current rulesets apparently aren't available. And I didn't have the gumption to figure how to use the tools which are available.

Another book about truthiness is Everybody Lies by Seth Stephens-Davidowitz. https://www.amazon.com/Everybody-Lies-Internet-About-Really/...

There's got to be some overlap, right?

There are stats available from porn sites. E.g.:

Now we’re not saying that size matters, but 2018 was an impressively big year for Pornhub and its users. Visits to Pornhub totaled 33.5 billion over the course of 2018, an increase of 5 billion visits over 2017. That equates to a daily average of 92 million visitors and at the time of this writing, Pornhub’s daily visits now exceed 100 million.

2018 saw Pornhub’s average visit duration grow by 14 seconds to 10 minutes and 13 seconds.

https://www.pornhub.com/insights/2018-year-in-review

Relevant: https://www.theguardian.com/technology/2017/jul/09/everybody...

Book: https://www.amazon.com/Everybody-Lies-Internet-About-Really/...

hawaiianbrah
Unless I missed it, I didn’t see something as plain as “X million in the US watch porn at least 1x a week” or something like that, which is what I’m curious about.
coldtea
They say "Once again, the United States continues to be the country with the highest daily traffic to Pornhub".

The give a graph of top 20 countries, which they say represent an aggregated 80% of their traffic.

The US is first, and take around 40-50% of that traffic.

Let's go with 40%. Of 80% that's ~ 32% of total.

They also say: "Pornhub’s daily visits now exceed 100 million".

This means 32% of 100M, so around 32M americans visit Pornhub alone each day.

And that's just one porn network. Along with Pornhub (which is however the most visited) there are also XVideos, XHamster, XNXX, Redtube in the list of Top 50 most popular websites.

hawaiianbrah
Yeah, I’m curious about the distribution therein. It’s obviously not the same 32M every single day, though I’m sure there are people who indulge daily. Are 20% of them daily users? And 60% weekly? etc
its publication has caused quite a few folks I know around here to renew their effort to buy local

"Buying local" is the kind of thing everyone is in favor of and no one actually does. Well, okay, not "no one," but far fewer than favor it in theory. Like "giving up Facebook," "switching to Linux" and "always using a condom," the number of people who make the claim in order to signal something about themselves is much smaller than the number actually engaging a behavior.

See further Seth Stephens-Davidowitz, Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us About Who We Really Are ( https://www.amazon.com/Everybody-Lies-Internet-About-Really/... ).

bartread
I'm fortunate enough to be on a motorcycle trip through New Zealand's South Island right now, and I can see the evidence that plenty of people (albeit the overall population is small) do continue to buy local in every small town I pass through: the archetypal general store is everywhere, not to mention hardware merchants, and many other local businesses serving their local communities. It just doesn't happen much in our bubble.
pandler
Your observation is pretty spot on, but I want to add some context. If you’re just doing a trip through NZ (which is awesome and I’m envious of you right now by the way), I’m assuming you haven’t tried to buy anything online.

My experience with shopping online in New Zealand is not like in the US. International shipping prices to NZ are typically insane and shipments take forever. Depending on the item, it’s usually cheaper and faster to just pick it up at the store.

The exception to that is of course online shopping from NZ-based retailers, but I only came across online shops for specific verticals, like computers (e.g. Newegg.com), Trademe (Craigslist), or simply a a brick and mortar shop that also delivers (same price as in store, plus delivery fee). Keep in mind, NZ has a population of something like 5 million people, so it’s harder to get economies of scale.

Furthermore, if you live in a rural area (I don’t know what the cutoff is) without a regular parcel service, you also have to pay an additional delivery fee through NZ Post. This looks like it’s at least $3.70, and maybe more for packages [1].

My point is that I think that the economic incensitves that cause Kiwis to shop locally are the same economic incentives that cause Americans to buy online. I don’t think it’s out of any heightened sense of altruism or local affiliation.

[1] https://www.nzpost.co.nz/tools/rate-finder/sending-nz/rural-...

ganeshkrishnan
>"switching to Linux"

Been using Ubuntu for close to 5 years now. I tried, I really tried to love it but the drivers are horribly outdated. Nvidia graphics broke everything in my Dell 9560. Even a fresh install of Ubuntu LTS is practically unusable. Each keystroke takes around 3 to 7 seconds to register. The screen lag is horrible. I tried fresh install and still everything looks pathetic compared to OSX or even Windows 10.

After Windows 10 got native Ubuntu 16, I just moved to it and though I have to run Linux startup services manually I am pretty satisfied with it.

We can't fight the inevitable just because of personal vendetta. Walmart was a small time store once. They got big because of the ease of use. Same with Amazon. To me it's just stupid to go out of the way to "buy local" when it's a losing game.

michaelmrose
Even though way more machines work great out of the box it's pretty clear your machine isn't one of them. Unfortunately since manufacturers often don't care about linux support on a particular box buying a machine and hoping its supported isn't an optimal strategy on linux or mac.

There is nothing about linux that would imply screen lag or keystrokes its pretty clear that something was clearly broken with your gpu driver. For example using software rendering rather than your gpu. Possibly something that could have been corrected by using drivers that aren't "horribly outdated" which are certainly available.

You didn't happen to think all the people using linux are just dealing with keysstrokes that take seconds to show up and pretending its normal did you?

danso
People weren't objecting to Walmart solely out of a "personal vendetta". They ostensibly wanted local retail because it makes for a more tightly-knit/healthier community. You can call it sentimentality but it's not something with no value, especially in small communities.
Fjolsvith
Having rolled my own ecommerce store on Ubuntu LAMP for several years, I finally got frustrated at the difficulty of recovering from power failures/spikes that hose it's file system after the UPS died. On average it takes 8 days to reconfigure and restore the new installation with all the lineup of products being tweaked to perfection.

This last time, I put Windows 10 and WAMP on the server and was amazed. I was tweaking products after a couple of hours. Things like ssl certificate, smtp/pop3 server, remote desktop all installed and worked with no arduous arcane CLI commands or hours spent researching tasks required.

Bonus: Robust antivirus and easy backups with a simple DOS script run from the task scheduler.

With only one buyer a week and 250 unique daily visitors, I don't need the speed of LAMP for my estore.

slezyr
I use Linux for long time. And newer stayed at ubuntu for long time. Current LTS release comes with broken VirtualBox, which hangs system and they have no plans to fix it.

Ubuntu is worst linux distro.

mixmastamyk
I have the same laptop with intel gfx, works like a dream, no spyware.
acct1771
You never hear about the 999 who don't have a single issue.
psyc
I’ve also earnestly tried switching to Linux many times, over 15 years. My most recent attempt was just a month ago, after becoming infuriated by Windows Update.

It’s not terrible, but taking all of my daily work into account, it’s still not feasible on the whole. And the problems are about the same as always. Hardware compatibility, arduous installations and config (yak shaving), and stuff just not working. And the pervasive pattern, inherited from open source itself, that things have recursive dependencies, each involving their own incredible journey of configuration and workarounds. Always the journey, never the destination of getting work done. In the end, I clean installed Windows, set it up to minimize annoyance, and got on with work.

I feel that, much like Blender, Linux requires a certain mindset. I suspect that mindset resembles either masochism or Stockholm Syndrome. I still have no complaints about MacOS.

None
None
michaelmrose
You don't buy a random laptop and hope it supports mac os right? How about buy something with linux instead of buy the cheapest junk you can get and hope it supports linux?
psyc
Hardware compatibility is a good start. Your tone aside, you have a point. But that's only the first of many problems with Linux.
michaelmrose
While there are problems with linux I'm not sure you have correctly identified them.

"And the pervasive pattern, inherited from open source itself, that things have recursive dependencies, each involving their own incredible journey of configuration and workarounds."

You get software from repos. Your system figures out the dependencies for you I literally have no idea where you could possibly be coming from.

"Hardware compatibility"

Buy compatible hardware.

"arduous installations"

Don't know what you are talking about. Given well supported hardware installation is incredibly easy and has been for over a decade.

" feel that, much like Blender, Linux requires a certain mindset. I suspect that mindset resembles either masochism or Stockholm Syndrome. I still have no complaints about MacOS."

You accept that MacOS doesn't run on just any hardware, buy hardware from a vendor who sell you MacOS preinstalled on supported hardware and experience no issues. Can I propose that you would have similar lack of issues with a vendor who sells prepackaged linux solutions?

Linux is bad in some ways but you are mostly kinda bad at linux. People installing hackintoshes seem to exist in a world that is fraught with pitfalls and workarounds but nobody expects MacOS to work out of the box on whatever. Linux by virtue of actually working on 99% of machines catches flack for only working optimally on 80%.

To boot you wonder why apple who has received thousands of dollars from you works better than a product which you provide zero support for. Its super likely that if you and others who wanted open source solutions to work for them had spent 15 years giving open source software 10% of the support provided to closed source solutions you might have something closer to what you want.

psyc
I’m as baffled by your perspective on configuration as you are by mine. I’d spend half my time working to improve userspace if I could financially afford to. “You’re bad at Linux” is the worst way I can think of to win anyone over.
michaelmrose
Nobody is paying me to sell you Linux. I'm honestly more interested in countering what I see as low quality information borne of lack of experience and knowledge.
zaarn
>Buy compatible hardware.

A lot of people have hardware and aren't looking to buy something shiny and new just to use Linux.

It's an immediate no-go for me to recommend anyone Linux when their machine is not support out of the box (sans nvidia blob, some people insist on it)

StudentStuff
Buying shiny new hardware isn't needed, any old Thinkpad is bound to have good Linux support, plus it'll be upgradeable and durable!

Use the right tool for the job, avoid assuming you need the most expensive tool possible!

zaarn
There is plenty of old hardware that won't properly work for modern linux distros, either by missing firmware packages or software not running on the CPU anymore (such as on an old Thinkpad of mine from 1998, modern distro's don't properly run, a lot of stuff fails with invalid opcodes or segfaults. An ancient copy of knoppix runs (with kernel 2.3))

Even then, most people don't buy with "it must run linux" in mind, they grab whatever has good sounding numbers on it.

StudentStuff
Heh, wasn't thinking of going that old, more like a Thinkpad T430 (can be had for $150) or newer depending on budget. Any of that line will run your distro of choice without issue.
michaelmrose
The manufacturer of the machine or components thereof who has all the specs and time put in the work that enables windows to support the hardware.

Free software development time is finite and information on some hardware is notably absent.

If you desire better support pool money to pay for improvements to particular hardware. Anything else is hoping people are willing to expend their resources for free so that you don't have to.

In the meantime I'm guessing most people intend to eventually buy a new machine. Upgrade time is a great time to think about trying Linux. Instead of

Linux it will run on whatever terrible machine you got on sale at Walmart with a Vista certified sticker.

Think instead...

Make your next computer a Linux machine.

zaarn
Microsoft pays good money to vendors to improve the hardware for windows.

Outside of hardware primarly used under Linux, you won't have much luck paying any vendor money to improve that situation. You'll have to suck it up and make it work anyway by patching the kernel.

>Linux it will run on whatever terrible machine you got on sale at Walmart with a Vista certified sticker.

Vista? I have upgraded people to Linux with 98 stickers and earlier. These people won't buy new machines, the current machine will have to work.

There won't be a "next computer" for some of them.

secabeen
Linux runs fine on desktop. I haven't had a significant issue in years on hardware that has the benefit of reliable wall power.
psyc
That’s good to know. It’s true my last attempt was on a low end laptop. My qualms about my daily applications remain. I’m sure I’ll keep trying.
SlowBro
Linux SysAdmin here with 20 years’ experience: I love it on the server and hate it on the desktop. Always use the right tool for the job.

But that’s a good analogy for this article. UNIX-like devices did eventually dominate the world, just not in the form factor we were expecting. I’m writing this on my pocketable UNIX device. You’ve probably heard of it; it’s called an iPhone. If one adds up all the devices in the world, Microsoft only owns a fraction of all of the operating systems installed.[1] Like the store keepers in this article, Microsoft failed to adapt but continued to rely on an increasingly failing business model (desktops). There will still be a place for Windows in 20 years but it will be a niche product like macOS is today. Unless they quickly adapt.

I certainly prefer to Amazon something than go to a store. This story is one of struggling to protect a failing business model. Were I one of those business owners I would spend all of my (increasingly) free time learning to either sell online or tweak my service so that it’s so compelling that people would willingly dress, drive out, risk crowds, and pay more. I would get online (hah!) and find the world’s best retailers, find out what they’re doing, and emulate their success.

There are still buggy whip manufacturers[2] but they learned what was needed to adapt. Adapt or get run over.

[1] https://en.m.wikipedia.org/wiki/Usage_share_of_operating_sys...

[2] https://www.uswhip.com/product/buggy-whips/

ganeshkrishnan
> I love it on the server and hate it on the desktop

It's an equal nightmare to develop on windows and deploy on Linux. But it's true. I have Linux systems running for couple of years with no downtime at all.

SlowBro
Not even for kernel updates?
ganeshkrishnan
Yes it requires an restart for kernel but for one machine I never did for an year
user9182031
I've found Linux usually works great on low end desktops with limited use (such as web browsing) but I've had many problems with modern GPUs and multiple displays. I upgraded to a 4k monitor and wanted to use my old monitor as a secondary and it required manual editing of configurations to support a setup with two monitors and two different resolutions. This is stuff that has worked on Windows or Mac for decades that Linux still hasn't figured out.
Retra
I think "buying local" went out the window with "manufacture local" and "produce local."
smileysteve
+1 I have friends who buy things from the local retailer instead of Walmart, but it's the Walmart brand.
EggsOnToast
I always imagine an unspoken "at a certain price" when people say they believe in buying locally. I live a short distance from a grocery store in my neighborhood, but I rarely go there because I like to cook and their prices for meat and produce are always higher than the big chains. In the case of their meat the difference has sometimes been as much as $1.50/lb. which is huge if you're someone who prefers a frugal lifestyle.

Edit: As for condoms, I think the situation is a little different. They ruin sex compared to going without a condom, marketing from Trojan be damned. I'm not trying to endorse the mindset, but there are many people who are primarily just trying to avoid getting someone/themselves pregnant so if someone's on the pill that's good enough for them.

grogenaut
Welp, gave up facebook a year after they opened, been using linux as a main home computer since 2001 (tho I am multi-os), always used a condom, and I always buy local (amazon, in seattle) and starbucks (same)...
tinymollusk
I didn't find "Everybody Lies" particularly compelling. It was an interesting piece of exploratory research, but I didn't see a lot of effort on generalizability. Several of his examples were using rare occurrence rates.

In short, the whole book felt like he got great access to data, and poked around in it until something interesting popped up. Then overarching conclusions were drawn without significant effort to describe whether or not the data was representative.

HN Books is an independent project and is not operated by Y Combinator or Amazon.com.
~ yaj@
;laksdfhjdhksalkfj more things
yahnd.com ~ Privacy Policy ~
Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.