No. 35 Statistical Fallacies
Survivorship Bias
In Turkish: Hayatta Kalan Yanlılığı
You only see the ones that lasted, because the ones that didn’t are in the tip.
Fridges used to last forty years. My mother’s is still running.
She’s right. The new ones are gone in five.
And where are the other fridges bought the same year, Fadime? In the tip. The fridge in the tip doesn’t talk.
What is it?
Judging from the successful examples of something while ignoring the failures, because you never see them. If the old fridge still runs, it may not be because old fridges were built well but because the ones that broke were thrown out long ago. The eliminated don’t speak; you only hear the survivors.
The pattern
- Every X I’ve seen is successful / sturdy / rich.
- Therefore X makes you successful / sturdy / rich.
- (The X’s that failed are out of sight.)
Why it fails
What we see passes through a sieve: failed companies don’t write books, scrapped fridges don’t stand in kitchens, dropouts who made nothing don’t give talks. What’s left looks like the whole. The famous case is wartime bombers: someone counted the bullet holes on the planes that made it back and wanted to armour those spots; the spots that needed armour were the ones without holes. The planes hit there hadn’t come back.
How to spot it
- Every example is a success; the failures aren’t around, because they’re exactly the ones that disappeared.
- Phrases like “look, it’s still going”, “he did the same and it worked”.
- Nobody asks how many there were at the start, or how many fell by the wayside.
What to say
Ask about the invisible ones: “How many were there, how many are left? What happened to the rest?” Without the starting number, the number of survivors means nothing. Then ask for the reverse: “Is there anyone who did the same and failed?” There is; they just aren’t telling.
When it’s not a fallacy
Looking at the survivors is perfectly legitimate if you count the eliminated too: “ninety out of a hundred fridges lasted forty years” is evidence. Taking one long-lasting example as proof that “this is possible” isn’t a fallacy either. The fallacy is generalising from the survivors without counting the ones that didn’t survive.
Examples
- Work“Read the stories of three famous founders who dropped out, then drop out.” The tens of thousands who dropped out and became nothing never got a book.
- Health“My grandfather is ninety and drank rakı every morning, so it can’t be harmful.” The ones who drank rakı every morning and never saw ninety aren’t here to tell it.
- Shopping“This brand’s old models are still on the road, they were built to last.” Without knowing how many cars from that year are still on the road, three survivors say nothing about sturdiness.
- Social mediaThousands of videos say “I learned English in three months with this method.” Nobody who failed with the same method makes a video.