This sure changes how I view the many results that I have had to deal with that were less than the current AIAG Gauge R&R requirements.

However, the real question is: How do you convince customers that are ingrained in the AIAG Gauge R&R process that they need to change how they require the results to be calculated? Or is it to convince the AIAG group to change their published documents?

Kim Howarter
Revcor, Inc.
Carpentersville, IL

I'm hoping Don will weigh in. I remember taking the EMP workshop years ago, and Don told us he had presented this paper at an AIAG or ASQ working group, and they just stared blankly at him...
On the hopeful side, I ran into some British and German engineers a few years ago, and they told me that they had abandoned AIAG GR&R and were all exclusively using Wheeler's EMP.

I recommend doing some simulation work, and then you will discover a few things.

1) The use of the d2* constant was developed for variances. When you use this constant to estimate a standard deviation, you get a biased result. It will show up especially in the attempt to determine reproducibility, where it is generally quite significant. The d2 number should actually be used instead. If you are trying to estimate the variance from a range, or a group of ranges, use d2*, but if you are trying to estimate a standard deviation, use d2. This depends on your research question, and what you wish to do with the numbers generated.

2) Any attempts at determining measurement error as a proportion of observed total variation in a measurement study is bogus, and has no external validity. The problem is the sampling method and the sample size. First, I contend that you should not be taking a random sample of items, but a stratified (non-random) sample to exercise the full range of the measurement system. There is other analysis that you need to do, such as determining measurement error as a function of item size. You certainly want to look at product near your specification ranges, and how the measurement system behaves in those regions. Taking a "random sample" could be quite misleading.

The second problem is the sample size used. Just look at the confidence intervals for variances and standard deviations with a sample size of 10 (a size often used in an "R&R" study)! The sampling error is so large, that any attempt at estimation if futile.

3) And finally, as I looked at 1000's of measurement system applications in industry, in many cases, "reproducibility" should really be treated as a "fixed" factor. That is a complete different analysis, and study. In the case of a fixed factor, you are not interested in estimating the variance from operator/system to operator/system, you are interested in estimating the bias between operators/systems. Often, this is much more useful.

May I suggest also posing this to Minitab and JMP, to allow for alternate calculations in their software?

Thanks for the excellent article. It was timely and have already forwarded it to colleagues who are starting to get the message.

GagePack by PQ Sysyems has already incorporated Don's EMP III methods as part of their calibration software.

Rich DeRoeck

The member companies of ISMI (the manufacturing subsidiary of SEMATECH, the semiconductor industry consortium) have written a white paper to aid companies in our industry to deal with auditors who insist on using AIAG guidelines, even though the MSA Manual states they are only guidelines. We have found that often both internal and external auditors insist on the use of AIAG methods, even though Don Wheeler and others have shown that there are superior measures of the goodness of a measurement system. The white paper can be downloaded free of charge via http://www.sematech.org/docubase/abstracts/35939.htm.

A corresponding white paper for use with SPC system auditors can be found at http://www.sematech.org/docubase/abstracts/35938.htm.

The method Dr. Wheeler is describing is the "Xbar-R Method" in Gage R&R (Crossed) in Minitab and the "honest" ratios are presented as well as the AIAG ratios. The "honest" ones are in the %Contribution column presented first, and the AIAG ratios are presented as %Study Var in the next table.

For what it's worth, I believe most practitioners with stat software are currently using the ANOVA method and not Xbar-R, but the concepts of additivity among %Contribution as compared to %Study Var hold true just the same. I believe the only attractive propoerty of %Study Var is that it is in the original units of the data and at least in that dimension has an easier interpretation, but without a firm understanding of the underlying formulas and concepts the correct interpretation of each is likely lost.

Interesting article. I am curious as to whether any of this has been communicated to the AIAG and what their response is.

Dr. Wheeler,

           I've been reading through your paper(I am somewhat new to R&R) and I've been finding it extremely helpful in getting myself up to speed with what I need to know. For that, I am very thankful. My only issue, and I'm hoping it proves to be a misinterpretation or misunderstanding of the material on my part, is where you get the number 1.906 as the value for d2*. I'm not questioning it's validity in the formula; more so, where can I find such a table of constants?(to aid in practical application on other potential sets of data) Again, I feel as though this may be some kind of misinterpretation of what I'm reading, so any bits that can help me over this hurdle would be greatly appreciated! Thanks! 

Don, I respect your work especially your SPC text, but your continued rant about the AIAG method is dishonest. It is time for it to stop.

First, if you've done your homework you understand why Standard Deviation, not variance was used at the onset of Gauge Studies. It was 1962 and the error involved with doing calculations by hand was greater than the error with using r-bar to estimate s (or s^2). So your "honest" method just squared the result. Wow, you must be a genius! 

And your talking down to us about a + b does not equal c is just plain arrogance.

The problem with your rant is you have no way to deal with the adequacy of a measure where there is a specification. you think that industry just cares about control charts. You can tell you are just an academic.

Your writings are used as an excuse to avoid the hard work of getting an adequate measure by a bunch of lazy MBB's and others. Anyone who has ever used the AIAG methods and found that they can solve unsolvable problems knows your writing in nonsense.

Just my experience.

Dr. Wheeler,

I admire the way you call out and fix flawed methodologies. Fairly recently, you wrote about, but didn't identify, a software package that had some fundamental errors. Do you know if they've been corrected?

May I link this article on LinkedIn?

There are further examples of uncareful R&R studies, especially in the automotive industry, where the AIAG's manual is literally read but not always understood, which is destructive testing. Though expensive they are, destructive tests are used more and more often for safety components and for components made of plastic or rubber or that are kept together by adhesives. Only the latest AIAG's manual warns not to rely on R&R studies for these measures, of course, but, once again, form takes over substance. I'm personally not in favor of R&R studies to test the reliability of a measuring system, I would rather break it down to its components, that is devices, people, methods and environment, which are much easier to audit and to repair, were it the case. But I'm not an AIAG ruler, so I've to go along. Mr. Wheeler is right when raising doubts about the validity of R&R studies, it's high time; for too long their reliability has been taken for granted.

Thank you for your article.  I have wondered why the measurement error sources never added up to 100% for years, and I have struggled to understand why a gauge that measures at 9% is 'acceptable' while '11%' is not.  For that matter, I could never figure out how a gage would be unacceptable when part variation would be 90% plus of the observed variation.  Thanks for the insight.