Harvest time—vegetables fresh from the local farmer’s market going right into a homemade soup, eaten with fresh bread. Yum! This month, I’d like to talk about a recipe for a Greek alphabet soup, which you had better know about if you plan on doing an experiment. This may be the first recipe that might save your bacon, rather than use it up.

What in the world am I talking about? Alpha, beta, delta, and sigma—that’s right: sample sizes. In my experience, this is a very frequently misunderstood topic that can lead to very bad decisions.

Let me give you an example.

Once upon a time, I was standing in line in a fast-food joint near a major client. A couple of engineers who worked for the business walked up and said, “Hey, you’re one of those consultants, right?”

It’s never a good idea to answer this question “Yes,” but throwing caution to the wind, I said, “Maybe.”

It turns out these guys were in charge of making a change to their process that would result in some nice cost-savings for the business. There was a concern that these changes might result in changes to the strength of the product.

 …