Four Control Chart Myths from Foolish Experts

Don’t teach people statistics—teach them to solve problems

Wed, 03/30/2011 - 10:00

There are four statements regarding control charts that are myths and in my experience, just refuse to die. The next time you're sitting in a seminar and someone tries to teach you how to transform data to make them normally distributed, or at any point during the seminar says, “Normal distribution” twice within 30 seconds, leave. You’ve got better things to do with your time.

The four myths

When you attend statistical seminars, do some statistical calculations seem like this? (2 minutes)

Are you “taught” these four things about control charts?
1. Data must be normally distributed before they can be placed on a control chart.
2. Control charts work because of the central limit theorem.
3. Data must be in control before you can plot them on a control chart.
4. Three standard deviation limits are too conservative.

April fool!

 …

Want to continue?
Log in or create a FREE account.
By logging in you agree to receive communication from Quality Digest. Privacy Policy.

Comments

Many processes, especially those in which the critical to quality characteristic is an undesirable random arrival (particles, impurities, pollutant levels) do not follow the normal distribution. It is therefore correct to say that the distribution need not be normal for the process to be in control. "In control" means simply that the distribution's shape, scale, and threshold parameter (if applicable) remain constant and that assignable causes do not produce outliers.        

 

 

The x-bar chart is somewhat robust in the presence of non-normality because of the Central Limit Theorem but the chart for individuals is definitely not. I can supply examples of X charts in which the false alarm rate exceeds 1 percent versus the 0.135% promised by the Shewhart chart's assumptions.              

 

 

Data transformations (e.g. square root or cube root for the gamma distribution) will often however make the result sufficiently normal for plotting on a traditional 3-sigma chart. Software such as StatGraphics or Minitab (or even Microsoft Excel with some user programming) can fit a gamma or Weibull distribution where appropriate, and the user can then select the 0.00135 and 0.99865 quantiles as Shewhart-equivalent control limits, with the median (not the mean) as the center line.      

 

 

In addition, individuals as opposed to sample averages are in or out of specification so assumption of normality will deliver totally unrealistic estimates of the process performance index. The result can be off by orders of magnitude (in terms of the nonconforming fraction) for a gamma distribution.          

 

 

None of this is to say that non-normal data can't be plotted on a preliminary SPC chart with normal control limits; doing so may make the non-normal distribution quite obvious up front, as will a histogram and a normal probability plot. Deployment for routine use on the shop floor requires however that the control limits and center line be changed to reflect the non-normal distribution or that a transformation be used to make it behave like a normal distribution.

I'm sure in your industry that there is a need for such minutiae.  What I'm trying to address in my article is the rampant "belt" sub-culture being created, especially in service industries, regarding organization-wide implementation of "quality fad du jour."  As I said, only 1-2% of people need advanced statistical knowledge. 

I stand by my statements about normality and ask you to be careful:  Deming HATED the use of probability limits.  I saw him once growl at a quesioner during a seminar, "Normal distribution?  I've never seen one!"

I'm with Davis on this one.  But don't take it from us.  Go back and actually read "Economic Control of Quality of Manufactured Product" and you will find that Dr. Shewhart invokes the Tchebychev Inequality in his derivation of control chart structure.

I am writing up a simulation that I just completed, looking into the issue of the robustness of ImR charts. In this simulation I used a random generater to create 197 Binomial Distributions, with 5000 in each. I used values of p set to 0.1, 0.2, 0.3, 0.4, 0.5, 0.6 to 0.7, 0.8, and 0.9; with sample sizes in increments of 5, from n=5 to n=95. The simulated data thus reflected very skewed data sets (where p = .1 and .9 and n was small), through a lot of sets that were much more unimodal, symmetrical, bell-shaped. (I also include p=0.05 at the higher sample sizes). I then ran ImR charts for each data set, and recorded the number of false signals in each chart for each of the four common Western Electric Zone tests. For all four rules, the total number of "signals" was 14,405, for an observed probability of a false signal of 0.014, and an observed ARL of about 69. Most of the indicated signals came from the lower-left and lower-right corners of the space (n and p low); there were a lot of rule three signals (4 out of 5 more than one sigma away on the same side of the centerline); I'm guessing that that is because the binomial is more heavy-tailed than the normal.

Looking at all the data, if we just use rule 1 (point outside the control limits), the empirical probability was 0.0037, ARL 274. When I added rule 2 (run of 9 above or below the centerline), the empirical probability was .0063, and the ARL was 160. This is not bad, when you consider that I included all the signals (including those from the more skewed corners of the space).  The probabilities are much lower, and the ARL much higher, when p is from 0.3 to 0.7, or n is higher than 15. Of course, the probabilities are much higher and the ARL is much lower, when p is very low or very high, and n is very low. These are the most skewed conditions. The point is that ImR is very robust in most cases... 

Contrast this with a similar study of 75,000 randomly-generated data from a "normal" distribution: an ImR chart of those data had an empirical probability of a false signal of .008 for rules one and two, with an observed ARL of 130.

We don't get infinite amounts of noise-free data; we get finite amounts of noisy data. We don't get theoretical curves; we get histograms. We are fortunate that many of our data fit some models closely enough that we can use theory for prediction ("if," as Shewhart said, "we don't examine too carefully what we mean by 'close'"). The ImR chart has proven robust enough in most cases for which I've used it. I haven't found transformation to be particularly useful in SPC (with the possible exception of rational subgrouping to transform the distribution by averaging); it's certainly possible, and I have done it. A few years ago I mentioned that in a discussion. Lloyd Provost asked, "What would it mean if you got a special cause in the transformed data?" While I was still trying to formulate an answer to that, he asked, " What would it mean if you didn't get one?"

Since then, I have relied on Deming's claim that statistical theory is only half of what you need for SPC; the other 50% is process knowledge. I may get a few more false signals in an ImR chart of, say, cycle-time data from a call center; I expect that, and usually investigate only those that go well beyond the upper control limit. I will, however, fit a Weibull or use the "non-normal" option in Minitab when I do a capability study with those call-center data.

There are those who are afraid of clarity because they fear that their ideas may not sound profound.  While you try to dance around the issue, you continue to get the wrong answer.  Transforming your data is simply a fancy way of lying with statistics.  Davis is right.

 

Donald J. Wheeler, Ph.D.
Fellow American Statistical Association
Fellow American Society for Quality

If the purpose of control charts is to detect process changes (special causes) - how can we specify the required chart (or sample size) if we haven't even determined how much of a process shift are we are interested in detecting? In MANY of the applications I have worked on, Individuals charts are COMPLETELY INEFFECTIVE in controlling the process since as you know, Individuals possess much more variation than averages do.  That is, the power to detect changes QUICKLY is too low.  Of course, before charting anything we need to know that the charactistic is worth charting AND have some idea of how much of a process change (mean shift or variation change) is of PRACTICAL importance so that an appropriate chart can be designed.  Without asking and answering these questions, how can the tool (control chart) serve it's purpose?

I agree that in many cases traditional Xbar charts (of appropriate sample sizes) are not viable, so when only Individuals are feasible, CUSUM charts are very useful since they improve sensitivity. 

I see LOTS of data that STABLE - yet NON NORMAL.  If I were to use 3 sigma limits on an I-MR chart, I would expect a large number of false alarms since 3 sigma limits do not account for the skewness (especially on characteristics such as flatness, runout, etc.)

Of course, Normality of the individual data is of little concern when working with averages due the CLT...

I don't know why in the year 2011, with all of the tools available, we need to continue to over simplify (dumb down) everything - just to make it super easy.   

The challenge many statisticians have is communicating methods effectively without resorting to oversimplifications.

 

Such utter nonsense is not even wrong.  You do not have a clue to what you are missing.  In the video they got the answer wrong because they did not understand the concept of place value.  You got your answer wrong because you fail to understand that probability models do not generate your data, processes do.

 

Donald J. Wheeler, Ph.D.
Fellow American Statistical Association
Fellow American Society for Quality

It is true that processes as opposed to probability models generate data; on the other hand, certain distributions are known from experience to model the data accurately. I had workplace experience with (1) particle count data in semiconductor tools and (2) vendor quality reports about impurity levels in semiconductor chemicals. The gamma distribution turned out to be an excellent model for both as shown by the chi square test for goodness of fit and the quantile-quantile plot, and it is backed up by scientific reasoning.           Impurities and particles are undesirable random arrivals, for which the Poisson distribution is the standard model. The gamma distribution is the continuous scale analogue of the Poisson. If one uses a standard 3-sigma Shewhart chart in the production line, the manufacturing operators will soon tire of responding to false alarms at the upper control limit. (Since the quality characteristic is something undesirable, the lower limit is not particularly important.) Even worse, the traditional CPU or PPU will have little to do with the actual nonconforming fraction; it is possible to be off by a factor of 1000 or more. E.g. the PPU may promise 1 defect or nonconformance per million opportunities when the process actually delivers 1 per thousand.            It is important to remember that Shewhart's methods were developed for use in an era when people did not have computers or calculators. The sample range is easy to calculate, and I suspect that the median chart was invented so production workers would not even have to add up the measurements and then divide. Today however it is easy to deploy on an Excel spreadsheet a control chart for a gamma distribution in which the spreadsheet calculates the exact quantile of the measurement and turns the cell red if it exceeds 0.99865 (or any other desired false alarm risk). It is also easy to deploy an attribute control chart that does the same for the quantile of a binomial or Poisson distribution, thus bypassing the requirement of an expected count of 4-6 (almost universally something bad like defects or scrap) for the traditional attribute charts to work. All the operator needs to do is enter the measurement or defect/scrap count and the rest happens automatically. The lesson here is that today's technology allows us to use methods in a production environment that would have been prohibitively time-consuming if not impossible during the early 20th century.

Davis, You have certainly stirred it up...And I totally agree with you!!  Even after nearly 30 years of studying and using Process Behavior Charts, I am still learning and gaining insight into their role in turning data into knowledge as a basis for action.  We are swimming (drowning?) in data these days due to technological advances in data aquisition.  Process Control vendors often try to sell their systems by saying, "You can get a data point every 5 seconds - Isn't THAT wonderful!!!"  No.....it isn't.  Such data from a continuous process is usually so auto-correlated that it is useless.  Enter rational sub-grouping and knowledge of the context of your data.  Then and only then can one properly transform data into knowledge as a basis for action.  All data costs MONEY to collect; therefore, you had better be able to gain some knowledge from it!

Those who howl when the four myths are exposed would do well to educate themselves by reading (and understanding) Shewhart's book "Economic Control of Quality of Manufactured Product".  It is 500 pages of sheer genius.  I walk away from those who profess to be SPC experts, and yet have never read this landmark work.  I can usually tell whether or not someone understands control charts just by how they react to the four myths.

Another source of great knowledge in this arena is Donald Wheeler's book "Normality and the Process Behavior Chart".  Dr. Wheeler definitively blows up Myth 1 in this study of 1143 distributions.

Finally, I must say that the Individuals chart is indeed the "Swiss army knife" of control charts and has many uses in industry.   Yes, it is simple and easy to use - and that is the beauty of the tool!  From Dr. Shewhart: "...the fact that the criterion which we happen to use has a fine ancestry in highbrow theorems does not justify its use.  Such justification must come from EMPIRICAL EVIDENCE THAT IT WORKS."  Imagine that!

  

 

If you can get a data point every five seconds, it might be possible to deploy feedback process control similar to that used in the chemical industry. Box and Luceno's "Statistical Control by Monitoring and Feedback Adjustment"  seeks to connect SPC methods to continuous control techniques such as proportional, integral, and derivative control.

The problem with SPC is the charts are not being used as designed, to hunt down and remove special causes and reduce variation. Just passive process monitoring and wall paper charts for ISO auditors. This HUGE issue is not addressed at all by "Stat Experts" who continue to waste time writting about ARL and false alarm rates etc.....

Being lead astay 1% of the time when looking for a "special cause" is something I should be concerned about? Are you kidding? Process improvement is a result of the interaction of the line operator with the chart and the ACTION he takes. All this other stuff just gets in the way.......

To understand the issues surronding SPC implementation, most authors need to spend more time on the factory floor and less time in their little statistical world.

Rich DeRoeck  

Submitted by pagnonig on Fri, 04/01/2011 - 09:01

I'm not a statistician but someone who strive for spreading continuous improvement culture in my organization.

In practice I've found that Davis and Don teachings can be used as "icebreakers" on people which are inherently suspicious and resistant to change. Believe me, when you are able to create a common improvement language with some easy tools to be applied in almost everyday activities you gain tremendous results.

Also, this new mindset in people help to generate new ideas for improvement which open even more space for increased performance.

And "Quality" or "Excellence" becomes really a common aim for everybody in the group!

Kind regards.

Giuseppe

 

Add new comment