I seem to be thinking about the measurement phase a lot recently. I suppose it’s because I’ve seen some teams working hard on problems that turned out to be nothing more than problems with measurement. Let me give you an example.

Once upon a time, I was responsible for a product that was frequently machined by the end-user. We had a measurement system that purported to predict one aspect if its machinability, which involved the use of a band saw cutting right down the middle of the product. We had complaints from customers who were machining to tight tolerances, so we were looking to design an experiment to make this characteristic better.

Then we noticed something odd. The readings on this device didn’t seem to relate to the complaints that we’d gotten from customers. To complicate matters, not every customer machined and not every lot was tested this way.

So we did a measurement-system analysis on the device, and found out it was good only for generating random numbers. Measuring the same product time and time again resulted in so much variability that there was no way to predict which lots would work and which ones would not.

 …