All of us draw conclusions based on what we see happening around us. Often what we’re observing is a sample from some larger population of events, and we draw inferences based on the sample without even realizing it. If the sample we observe is not a representative one, our resulting judgments can be seriously flawed, potentially at considerable personal cost.

During World War II, the Allied air forces wanted to analyze data on the damage suffered by aircraft returning from combat missions over Europe. Those in charge had wisely decided to examine returning planes in order to assess what design upgrades might increase survivability. They observed how many hits each had sustained, and where on the aircraft they occurred. The brass was of the opinion that those parts of the plane that received the most hits should be up-armored in order to make them more survivable.

 …