by James D. Jenkins

With so much focus these days on collecting and analyzing data, and implementing Six Sigma and other initiatives to ensure that design goals are met, it’s possible that critical details such as the data’s level of accuracy have gone unnoticed. We might not realize how this inaccuracy translates into false acceptance and rejection risks associated with measurement-based decisions. These risks define the probabilities associated with defective products being accepted while good products are scrapped.

Because measurements in research and development and manufacturing acquire traceability from calibration and, in turn, often determine product quality or a product’s target value through testing, these measurements’ own quality is extremely important. This is where ISO/IEC 17025 accreditation comes in. Fundamentally, the standard is about measurement quality.

You might point out that your existing calibration and testing programs require your providers to produce measurements “traceable” to some national or international reference, like those of the National Institute of Standards and Technology. But what exactly does that guarantee? If, for example, I calibrate a gage block with a NIST-traceable ruler, doesn’t that make the gage block “traceable” to NIST? Although this example exaggerates what happens in the real world, I’ve actually seen a laboratory calibrate gage blocks with a micrometer, rather than the other way around.

Are you beginning to worry about your measurement data yet?

Measurement uncertainty

Measurement data should concern us because no matter what we do, some quantity of unknown error in measured values will exist because measurement provides only an estimation of the measurand (i.e., the particular quantity subject to measurement). This unknown error creates doubt in a measurement’s result. The probable quantity of unknown error can be defined only through a further estimate known as measurement uncertainty. This is calculated in accordance with the ISO Guide to the Expression of Uncertainty in Measurement (GUM).

GUM defines measurement uncertainty as “parameter, associated with the result of a measurement, that characterizes the dispersion of the values that could reasonably be attributed to the measurand.”

GUM further states, “The word ‘uncertainty’ means doubt, and thus, in the broadest sense, ‘uncertainty of measurement’ means doubt about the validity of a measurement.”

The statement to the left tells us that the laboratory measured the 1 in. gage block and found it to be 2.2 µin. over nominal size, with an uncertainty, expressed at the 95 percent level of confidence, of 3.6 µin. In other words, the lab determined that a 95 percent probability exists that the measured value is in error by not more than 83.6 µin.

Recipients of measurement values and compliance statements based on measurement values are entitled to an estimate of the amount of uncertainty associated with the measured values. Without a realistic estimate of measurement uncertainty, the client can’t judge the measurement result’s quality. An unknown uncertainty means the measured value must be considered unreliable and, hence, useless. As NIST’s Ted Doiron aptly observes about this dilemma, “A calibration without a valid uncertainty estimate is simply exercise, the result of which is properly expressed in calories.”

If this information is new to you, it shouldn’t be. The requirement for an estimate of measurement uncertainty isn’t new. In fact, ISO 9001:1994 requires it in Section 4.11: “Inspection, measuring and test equipment shall be used in a manner which ensures that the measurement uncertainty is known and is consistent with the required measurement capability.”

Although ISO/IEC 17025 doesn’t--and indeed can’t--mandate perfect measurements, it does provide some assurance that laboratory measurement quality is evaluated, defined and controlled. It’s then our responsibility to use this assurance to determine if labs meet our needs.

Upon successful laboratory accreditation, the accreditation body issues a scope of accreditation that usually defines either the tests a lab has been accredited to perform or the best-measurement capability a calibration lab has demonstrated in accredited measurement disciplines.

The scope of accreditation might not include the measurement uncertainty that exists for measurements performed on your devices and/or objects. However, to be compliant with ISO/IEC 17025, the lab must calculate and maintain measurement uncertainty estimates for all data provided to clients or used in determining “in tolerance” or “out of tolerance” status. The standard’s Section 5.4.6.1 states, “A calibration laboratory, or a testing laboratory performing its own calibrations, shall have and shall apply a procedure to estimate the uncertainty of measurement for all calibrations and types of calibrations.” Section 5.4.6.2 adds, “Testing laboratories shall have and shall apply procedures for estimating uncertainty of measurement.” (Certain exceptions are given for some testing activities.)

The process of uncertainty estimation is somewhat subjective and requires adequate, unbiased technical review to ensure credibility. This review is achieved through expert third-party assessment from accreditation bodies. Because all aspects of the measurement process feed into measurement uncertainty, adequately assessing the measurement process is also required. Combine this need with that of consistency as well as continuous improvement as realized through compliance to ISO 9001, and you arrive at ISO/IEC 17025 accreditation.

Traceability

How many organizations do you know that claim their measurements are “traceable?” What exactly does that mean? ISO’s Vocabulary of International Metrology defines traceability as “property of the result of a measurement or the value of a standard whereby it can be related to stated references, usually national or international standards, through an unbroken chain of comparisons all having stated uncertainties.”

This means that for a measurement to be traceable, uncertainty must be stated. How many certificates have you seen that claim traceability yet provide no stated uncertainty? ISO/IEC 17025 accreditation resolves this problem by requiring uncertainties to be stated.

Current state of accreditation in the United States

Ideally, all ISO/IEC 17025-accredited laboratories should be in full compliance to the standard. Unfortunately, my experience has shown that some aren’t, but considering the magnitude of the task, labs and accreditation bodies are doing fairly well and continuously improving. The requirements are involved, and practical solutions for compliance are in constant development.

During the interim, you have a better chance of receiving quality measurements from an accredited laboratory than from one that doesn’t meet ISO/IEC 17025 requirements. Moreover, with a bit of knowledge, you can then compare laboratory capabilities based on their measurement uncertainty.

Accredited services

ISO/IEC 17025 accreditation’s fundamental purpose is to ensure competency in testing and calibration measurement processes. However, client involvement is critical to ensure that adequate services are received.

ISO/IEC 17025 can empower a lab’s client. With this power comes responsibility, and without adequate knowledge, clients might still not get the service they need--although under those circumstances, they’d have no one to blame but themselves. Thus, it’s important to become familiar with the requirements in order to use ISO/IEC 17025 to your advantage--for example, knowing which questions to ask a potential supplier.

The following concepts address several issues that you should consider when evaluating a testing or calibration supplier, given the current state of accreditation:

Estimated measurement uncertainty. Inquire as to whether the lab provides measurement uncertainty with the data. These values are needed when determining calibration or test adequacy and subsequent conclusions.

Ask whether the provided “uncertainty” is really the measurement uncertainty as required by ISO/IEC 17025, or if it’s the reference equipment’s uncertainty or some other value used to fill in the blank because the lab hasn’t gotten around to fully meeting the standard.

For organizations that have done their homework and understand how to analyze measurement uncertainty, ask for a copy of the analysis. If the lab claims it’s proprietary information, ask if you can view the analysis at its location.

Calibration and/or test data. Prior to submitting a device or object for testing or calibration, ask for a list of the measurements that will be taken and a sample data report--especially if you’re requesting in-tolerance or out-of-tolerance reporting. For calibration work, you will also want to ensure the measurements taken during the performance test adequately cover the functions and ranges you use.

Ensuring quality of calibration and testing results. ISO/IEC 17025 lists many methods that labs can use to ensure the quality of their calibration and testing results. One of these, proficiency testing, has since become a requirement by many accreditation bodies. Others are recommended for use by labs to add reliability and credibility to measurement results.

Proficiency testing is a professional service that puts a lab’s measurement uncertainty claims to the test. This is done by providing an artifact that has a known value with an uncertainty smaller than the participating lab’s. The lab reports its measurement and uncertainty to the provider. The results are statistically analyzed and reported along with a “satisfactory” or “unsatisfactory” result.

If you’re really concerned about the accuracy of a lab’s estimated measurement uncertainty, ask whether the given measurement process has been proficiency tested.

Note: To limit initial accreditation costs, many accreditation bodies allow labs to spread their proficiency testing program over a four-year period.

Witness some of your contracted tests and/or calibrations. ISO/IEC 17025 provides lab clients with the right to witness contracted tests and calibrations, assuming the lab isn’t violating other clients’ confidentiality. Use this to your benefit by taking the time to witness the measurements for yourself. Ask the technician questions and become familiar with your provider. Although you might not have the technical expertise to assess the lab’s measurement quality, you’ll undoubtedly develop an impression that will either enhance your confidence or cause you some concern regarding your provider’s capability.

Summary

Without ISO/IEC 17025 accreditation, “traceability to a national standard” is only a phrase printed on a report. With accreditation, clients can assume some validity to the measurements provided by their calibration and testing suppliers, defined via a valid estimate of the measurement uncertainty. ISO/IEC 17025 accreditation offers a major step in bringing a valid “traceability chain” to general industry’s front door.

Because measurements are usually the fundamental basis of most manufactured product-quality evaluations, the concepts of measurement quality, traceability and uncertainty put forth by ISO/IEC 17025 are useful wherever measurement-based decisions are made. In the manufacturing arena, measurement uncertainty, combined with process capability and other quality assurance data, can facilitate measurement-based decision risk analysis. Risk analysis defines the probabilities associated with defective product being accepted and good product being scrapped. These sciences can give quality departments greater capability in producing proactive, rather than reactionary, solutions.

Organizations selecting calibration and testing suppliers must understand the concepts of measurement quality. Because accreditation is based on periodic assessments rather than continuous monitoring, clients of these services must take active roles in monitoring their suppliers. Any apparent noncompliance observed by recipients of accredited services should be reported to the labs’ accreditation body. Let’s not forget that quality is primarily customer-driven, but customers must be clear about the nature of the quality they seek.

About the author

James D. Jenkins is president and founder of QUAMETEC, an organization that assists companies pursuing ISO/IEC 17025 accreditation. Jenkins developed the Uncertainty Toolbox for Microsoft Excel and is the author of Measurement Uncertainty Analysis Fundamentals, available at measurementuncertainty.com. He also chairs the Accreditation Resources Committee for NCSLI. Letters to the editor regarding this article can be e-mailed to letters@qualitydigest.com.