We learned in science class that we should use the scientific method to evaluate hypotheses. We should study the problem, formulate a hypothesis, run a controlled experiment, analyze the resulting data, and then make an objective decision. We have also heard innumerable times that we should make “data-based decisions.” These principles have been around for centuries in one form or another, yet all too often they have been ignored, often with disastrous results.

In this article, which will be presented in three parts, I’ll look at some examples from history and daily life that illustrate key pitfalls in experimentation and data interpretation. I’ll also discuss possible countermeasures that you can apply.

Pitfall 1: Not running an experiment at all

Often we come up with a theory that makes sense to us, and then, instead of testing the theory first, we go ahead and run with it and end up with surprisingly bad results. For example, we’ve all learned the hard way that some food items should not be microwaved. How many times have you turned something into shoe leather and then immediately asked yourself why you didn’t try microwaving a small sample before ruining the entire batch?

 …