| by Hershal C. Brewer
Companies that require calibration services must choose among three different delivery methods: internal calibration, in which the company itself provides the service; external calibration, in which the service is outsourced; and some combination of the two. Deciding which method is best, and managing it so that it delivers cost-effective and quality results, can be difficult. This article will look at the three options and offer some basic guidance to determine which choice is best for your organization.
Calibration Terms
 Certification--In the straightforward sense of the term, certification applies to equipment or personnel. For the purpose of this article, certifying equipment means using a master or standard to verify the accuracy and integrity of an instrument for its intended use. Certifying personnel requires an independent evaluation (by a recognized body) of the technical knowledge of an individual.
Registration--As it applies to ISO 9001 or similar standards, registration is the independent evaluation of an organizations’ quality management system, or QMS. Registration doesn’t ascertain a laboratory’s or other organization’s level of technical proficiency.
Accreditation--This is the process a calibration laboratory undergoes to become certified to the calibration standard ISO/IEC 17025. The standard also requires an independent evaluation of an organization’s QMS, followed by practical demonstrations of the technical proficiency of company personnel. Someone who’s also proficient in the same disciplines must evaluate these demonstrations.
Traceability--If the calibration provider is in the United States, for example, it will have documented evidence that every calibration is traceable to the National Institute of Standards and Technology. Without evidence, however, this statement means little. The International Vocabulary of General and Specific Terms in Metrology defines traceability as having (1) an unbroken chain of comparisons to national or international standards and (2) stated uncertainties at each step. The difference between the two represents the evidence, which is gathered during an accreditation assessment. Traceability can also be achieved through a chain different than a country’s national measurement institute (NMI). A laboratory in Texas, for example, might have traceability for some of its calibration through CENAM, which is Mexico’s NMI. It’s just as valid as traceability through NIST.
Competent--In the accreditation and calibration world, there’s “competent” and “not competent”--meaning technically proficient or not technically proficient.
Uncertainty--Any measurement, no matter how precise or accurate, has some amount of error. That amount is the uncertainty of measurement.
Calibration laboratory or calibration provider--These are often used interchangeably. They’re both defined as professional bodies that provide calibration services. |
Many organizations perform some or all of their calibrations internally. That is, they have items known as “masters” or “standards” that are used to calibrate working tools. This approach is common in machine shops, where master “Jo blocks” are used to certify calipers, micrometers and similar items.
Other organizations, such as testing laboratories and midsized to large manufacturing organizations, might also use an internal calibration methodology. Managing an internal calibration operation often includes justifying the operation’s expense, and consequently internal calibration should periodically undergo a cost-benefit analysis. This should include the combined sum of the overhead for assigned calibration personnel and costs of the standards’ calibration and depreciation, divided by the number of items to be calibrated.
The resulting figure is fairly easy to obtain if the internally calibrated items are consistent. For example, the analysis for calipers and micrometers is easy. However, if the items include calipers and micrometers, oscilloscopes, digital multimeters and several other items, the cost per item will sharply increase because of the number and cost of the standards required to support the internal calibration. The cost-benefit analysis must include a comparison of in-house costs against those incurred from managing external calibration.
The issues associated with managing external calibration providers include, but aren’t limited to, accreditation, traceability, cost and delivered performance.
A company will often depend on its quality manager to evaluate an external calibration provider. Though related, quality and metrology aren’t the same. A quality manager might find that a competitive bid process works best. However, he or she, along with top management, must understand that calibration work is like any other high-end service, in that proper, well-qualified and competent service will often require a higher premium than what the lowest bidder might quote. Managers usually have no issue with licensed electricians or plumbers; the same should be true for metrology professionals.
The question, then, is: “How can a quality manager select the most qualified calibration provider to support his or her company’s needs?” I recommend a competitive bid process with three basic steps.
The first step includes disseminating the bid package and basic qualifying documentation. This process largely depends on the company. If its business requires it to be registered to ISO/TS 16949, the calibration provider must be accredited under ISO/IEC 17025. However, it’s important to be aware of the requirements, because QS-9000 requires accreditation for calibration providers under ISO/IEC Guide 25, a standard retired some time ago. A company that only requires ISO 9001 registration can use a calibration provider accredited under ISO/IEC 17025 because the accreditation process easily meets ISO 9001’s documentation requirements.
I recommend that companies require accreditation as a precondition of the bid qualification. Delivering the calibration service is the prime consideration, and accreditation under ISO/IEC 17025 means that the technical provision of metrological service has been evaluated by a similarly qualified metrology professional. This includes an evaluation of the lab’s and personnel’s technical proficiency as well as calibration traceability. The bidding calibration provider should provide evidence of its accreditation. This must include or be followed by the scope of accreditation, also called the scope of work.
When considering a calibration provider that is accredited to ISO/IEC 17025, the organization that accredited the calibration provider should be a recognized body, that is, a body that has been evaluated nationally or internationally and invited to sign a mutual recognition arrangement (MRA) as a recognition of its technical proficiency and quality commitment. The easy way to find out if the accrediting body is recognized is to go to the appropriate Web site and see if it is a signatory. The Web sites are: www.ilac.org, www.aplac.org, www.european-accreditation.org, http://iaac-accreditation.org and www.nacla.net.
You might also want to require a bidder to provide both positive and negative references. Any calibration provider is likely to have one or more dissatisfied clients, although this shouldn’t be the sole criteria for selection. Knowing a potential calibration provider’s good and bad points will give you a better understanding of the quality of service you can expect.
The second phase of the bidding process should include a minimum number of documents. The quality manual and procedure manual should be provided, along with an uncertainty budget. This document, also known as the scope of accreditation, describes a calibration provider’s best performance. According to the International Laboratory Accreditation Cooperation, best performance or best measurement capability (BMC) is the lowest amount of uncertainty associated with a given measurement, given nearly ideal equipment and conditions.
A typical example of the calibration certificates your organization will receive should also be provided. If the certificate doesn’t have the logo of the laboratory’s accrediting body on it (e.g., IAS, NVLAP, A2LA), the laboratory should be asked to provide a certificate that does have one. Sometimes additional costs are involved with providing a certificate that includes a logo because the lab has to take additional readings and provide measurement uncertainties for them. Other documents might include evidence of liability and worker’s compensation insurance.
The third phase is an on-site evaluation. This might be reduced a bit for an accredited organization because it’s been evaluated by an external organization. That doesn’t mean you should blindly accept the provider’s accreditation. Select a representative sample of your company’s equipment that the calibration provider will service, then perform an audit of the procedures required to calibrate the equipment based on the sample. Each calibration provider should have sufficient manuals and/or procedures available to calibrate the equipment. To ensure consistency, the same sample of items should be used to evaluate all calibration providers.
The last consideration should be cost, although for top management, this probably won’t be the least significant consideration. Therefore, the evaluation report should highlight calibration providers’ qualifications and services that will support your own company’s requirements. Present a formal report to top management outlining the bid process and the discovery of information regarding the calibration providers. Include a recommendation, even a ranking of the top three providers. Have the supporting documentation available in case management requests it.
The third approach is a combination of internal and external calibration. It allows the company itself to calibrate the master items. With this approach, it’s important to choose which instruments to calibrate internally and which to calibrate externally. Assuming that the guidelines regarding internal and external calibration are followed, the only consideration would be the cost ratio, which is driven by the ratio of what’s calibrated internally versus externally. After all, internal costs are often hidden, while external costs are considered direct or obvious ones.
Whether a company’s solution involves internal calibration, external calibration or a combined approach, the bottom line is cost. Executives will look only at cost--and to be fair, that’s what they’re paid to do. However, others in the company who produce or review produced items must ensure that these meet customers’ needs. Calibration should be considered part of the built-in cost.
Still, for those who are willing to compromise on this particular cost, perhaps a review of the costs of nonconforming product or of massaging the supply network into compliance, compared with the cost of accredited calibration (which requires very little additional control) would be beneficial. This is known as “risk control.”
That approach might sound harsh, but let’s get real: Products affect people. As quality management and metrology management professionals, our job is to reduce risk to our employers. Part of the challenge of that job is reducing risk that can be attributed to errant measurements. Part of that can be attributed to what’s known as “measurement uncertainty” (MU).
The intent of the calibration provider to report uncertainty is to inform the customer of errors relating to a measurement, so that a reasonable assessment of any risk from that measurement--and any related internal measurements--can be made.
Risk control, as it concerns calibration management, includes understanding what the risks to the organization are and how they are defined and presented. MU is a risk to any measurement the organization may make. Understanding what MU is and why it is a risk can be considered a significant component of effective calibration management.
The concept of MU is very simple. MU is a mathematical description of the errors associated with any given measurement. Calibration providers in the United States typically will present MU in the calibration certificates provided to customers. The certificate may include a simple statement like “Uncertainty is reported at k = 2 to approximate the 95-percent confidence level.” The certificate may also report specific readings for measurements and report a related uncertainty for that measurement.
MU comes in two varieties: systemic and random. Some parts, such as environmental or instrumental, may well be systemic in nature, while others, such as the direct readings, are considered random in nature. Systemic influences affect the measurement as a type, not the specific measurement. Random influences affect the specific measurement and have less influence on the general parameters of the measurement.
If the user understands that any given measurement has some error and accepts this as part of the normal risk of use, then MU is a tool and not the only criteria on which to base a decision. In the United States, a system known as the four-to-one rule (4:1) is key. It means that the ratio of higher-level standards compared to a specific calibration level is four to one. That is, the standards are at least four times more accurate than the item they will be used to certify and/or calibrate. This is allowed and even directed under the U.S. national standard ANSI/NCSL Z540-1-1994. In the United States, laboratories operating under a 4:1 rule need only report specific uncertainties if their clients requests them and if they can provide evidence of maintaining that ratio.
Before recommending a calibration process, a quality manager must factor in cost and quality of service. Whether measurement uncertainty, the 4:1 rule or some other approach is appropriate, the quality of calibration service must be the overriding concern.
Hershal C. Brewer is an accreditation assessor for the International Accreditation Service, performing assessments of calibration and testing laboratories to ISO/IEC 17025, and of inspection agencies to ISO/IEC 17020. He is the former corporate lead auditor and corporate metrologist for Newport Corp., and the primary author of the Measurement Uncertainty Primer, available through the International Code Council (ICC).
|
