by Carmen Liuzza and Geza M. Pap

In 1986, the Department of Defense directed the removal of acceptable quality levels (AQLs) from all of its specifications. The implementation of this directive also involved the policy that, when lot acceptance is based on only a sample of the total lot quantity, all of the samples should be defect free for outright acceptance: That is, only "accept on zero" (AOZ) plans are to be used for attributes sampling.

AOZ plans are primarily intended for inspection of quality-type characteristics and not for establishing the product's performance. The AOZ requirement doesn't apply to destructive tests.

Several factors prompted the Defense Department to change to AOZ plans, but industry's example was clearly significant. The policy was set in an era where Harley-Davidson recaptured its eminence through process controls, Detroit fought its way back to profitability through "Quality is Job 1," and Motorola won the Baldrige Award and the bulk of the cellular phone business using total quality management and continuous improvement.

The message was quite clear: Customers have the right to expect quality. The Defense Department just happens to be the world's largest customer. After considering their dealings with industry, the department's decision makers realized that AQL-based specifications sent a message about their quality expectations that was contrary to their objectives.

Although the DOD's contracts had always specified in clear legal language that the department required all-conforming product, it immediately withdrew that "customer demand" by allowing nonconformities within the sample. In other words, the objective evidence that was supposed to prove that the lot met all requirements was permitted to include evidence to the contrary.

The notion of "permissible defect levels" was also contrary to the economics of the Defense Department's increasingly complex weapon systems. Relatively minor variances in $5 parts could cause the failure of $5 million systems. AQLs in specifications then compounded the problem with the inertia they created.

The success of an AQL-based production run is judged by the lot-acceptance rate. Both customer and vendor rest on their laurels if the lots pass. In fact, both sides became quite adept at explaining how failed lots are just statistical aberrations and should be bought on waivers. Every aspect of the AQL system favors inaction and avoidance of continuous improvement.

It became, and still is, the Defense Department's policy to do everything it can to encourage contractors to monitor, control and continuously improve their own processes; minimize variability; achieve high capability; and prevent rather than inspect for nonconformities. The Defense Department recognized that proper use of MIL-STD-105 certainly wasn't disabling these objectives; however, the department realized that it had allowed its suppliers to build an undesirable operational culture around the standard.

The new philosophy is that 100-percent conformance is the contractual and business goal, and that all processes should constantly aim for that goal. Furthermore--and this is the cornerstone concept of the Defense Department's use of AOZ plans--every nonconformance requires action. The old notion of "the lot passed with two defects, just pay the bill" had to be erased from the department's business process.

In practice, realizing that perfection is not always feasible, the Defense Department has said that it is willing to accept product lots whose quality levels are in the neighborhood of parts per million (PPM). This neighborhood is usually understood to be from 1 PPM to 100 PPM, depending on the importance of the characteristic.

This has been interpreted to mean that no nonconformances can be tolerated in any sampling plan used by the Defense Department because any nonconformance in such a sample could be considered inconsistent with the PPM quality goal.

The department is now selecting AOZ plans from MIL-STD-1916, which it developed to reflect its changed view of the business process. This standard actually starts with process control-based acceptance as the preferred method, and defines AOZ-type sampling plans as interim procedures until the vendor can move into the PPM range.

Countering AOZ misconceptions

Misconceptions abound about AOZ sampling plans, and they deserve to be heard and refuted. Keep in mind that AOZ plans don't change the statistics; they change the emphasis from throughput to prevention and encourage suppliers to strive continuously for fully conforming product. Some of the misconceptions include:

1. Zero nonconformances in a sample implies zero nonconformances in the lot.

This isn't true, because one doesn't imply the other. The purpose of requiring zero nonconformances is not to demonstrate that a lot's quality level is perfect or meets PPM goals. The purpose of AOZ sampling is either to accept product as long as no nonconformances are found, thus giving the contractor the benefit of the doubt, or to withhold acceptance of any lot when a sample contains a nonconformance, reasoning that the lot's quality is probably not at a PPM level.

2. AOZ sampling requires that the entire lot be perfect.

It might be unreasonable to expect an entire population to be flawless; however, AOZ sampling requires PPM quality levels, not perfection. The presence of a nonconforming item in a sample (especially if it's a small sample) reveals that the quality level isn't in the PPM range.

An important function of AOZ plans is to communicate to the supplier that the customer will not happily accept nonconforming product without redress. Compare this with a sampling plan of, for example, n = 100, accept-on-one: How could a supplier not get the impression that the customer would be perfectly satisfied to receive product that is 1-percent nonconforming?

Yet how many employees, during a 30-year career, would tolerate one error per 100 pay stubs regarding their social security withholding, vacation time or pension contributions? Wouldn't most employees expect errors like these to be followed by an immediate rework, corrective action and changes made to prevent their recurrence?

3. AOZ sampling plans are inferior because they are not as discriminating as non-AOZ plans.

It's true that non-AOZ plans are more likely to pass lots whose quality levels are near the AQL and more likely to fail lots whose quality levels are near the lot tolerance percent defective (LTPD). However, the AOZ plans presently used by the Defense Department shouldn't be viewed in the same way as AQL plans.

The Defense Department didn't employ AOZ plans to compete with non-AOZ plans. AOZ plans don't utilize an AQL, nor do they select by indexing the table with an AQL or an LTPD. AOZ plans simply say that, whatever sample size is chosen, no nonconformances will be allowed for outright acceptance, and the more important the characteristic is, the larger the sample should be to verify that there are virtually no nonconformances in the lot.

AOZ plans can be thought of as having associated AQLs, but this isn't the way the department uses these plans. Of course, an operating characteristic (OC) curve can be drawn for any given AOZ plan to describe the risks inherent in the plan, but the Defense Department is only secondarily interested in the attributes of an OC curve for a particular AOZ plan.

The Defense Department isn't looking for a particular appearance of the OC curve as it did when it indexed plans by their AQLs. Therefore, the department isn't looking to compare the discrimination ability of one OC curve to another, nor is it using AOZ plans to economize on sample size.

4. Use of AOZ plans for some contracts will result in excessive lot rejection.

In MIL-STD-1916, failure to pass an AOZ plan doesn't necessarily imply lot rejection, but it does mean that lot acceptance will be withheld and that the contractor will be required to take several actions listed in the standard.

Lots with nonconformances in the sample are candidates for a normal part of business, namely, negotiations between the consumer and the supplier based on waiving the contractual requirements for concessions that have balanced value to both. These negotiations, frequently based on evaluations of the subject lot regarding its "fitness for use," drive decisions to reject, screen or rework the lot. MIL-STD-1916 and MIL-HDBK-1916 describe these disposition procedures in more detail.

During negotiations either prior to signing the contract or after a lot fails to meet the AOZ plan, both the customer and the supplier should take into account the reality that nonconformances can occur for various reasons: occasional random failures do occur in a sample even when the process is operating on a low PPM level, the process may have degraded, the state of the art for this process may not be high enough, or drawing limits may be too tight for the capability of the current equipment or process.

Process control

The Defense Department's goal is for its contractors to control their processes so well, and with such a high capability, that lot acceptance via AOZ plans becomes unnecessary.

Keep in mind that the normal contract requires totally conforming product. Samples are just one form of objective evidence. Compared with the information given by other techniques, like SPC charts or error-proofing, samples make for a fairly weak set of evidence. That's why MIL-STD-1916 establishes very clearly that AOZ is not the department's final word on product acceptance.

However, AOZ plans will be the criterion for final lot acceptance as long as contractors have processes that aren't mature enough to generate better objective evidence. How long it takes to make the necessary improvements is the supplier's own choice.

When AOZ plans stand alone, higher quality levels are achieved only because of the supplier's fear of excessive lot rejection. The big picture is that process controls are needed to stabilize, monitor and improve processes.

The intent of MIL-STD-1916 and its accompanying MIL-HDBK-1916 is to give process control primary importance--AOZ plans are secondary.

About the authors

Carmen Liuzza, of U.S. Army TACOM-ARDEC at Picatinny Arsenal, has been a mathematical statistician in quality assurance for 33 years. He has recently worked as an SPC consultant and teacher. For the past two years, Liuzza has chaired an NDIA committee, which has just completed a new MIL-HDBK-1916, a guidance document to MIL-STD-1916. Among his current projects is the international standardization of sampling and SPC documents.

To contact Liuzza or Pap, e-mail cliuzza@qualitydigest.com .