| by Mamdouh Halawa, Ph. D.
Essential Terminology
Following are phrases to know that relate to lab proficiency:
• Accrediting bodies: A body or organization that assesses conformity assessment bodies under national or international standards
• Conformity assessment bodies: A generic term covering all labs, inspection bodies or certification bodies
• Assigned value: Value attributed to a particular quantity and accepted, sometimes by convention, as having an uncertainty appropriate for a given purpose (See also VIM:1993, 1.20 and notes 1 and 2.)
• Coordinator: Organization (or person) with responsibility for coordinating all of the activities involved in the operation of a proficiency testing scheme
• Interlaboratory comparisons: Organization, performance and evaluation of tests on the same or similar test items by two or more laboratories in accordance with predetermined conditions
• Precision: Closeness of agreement between independent test results obtained under prescribed (i.e., stipulated) conditions (ISO 3534-1)
• PT staff: The staff involved in providing that the scheme should have, or collaborate closely with those holding, adequate qualifications and experience in the design, implementation and reporting of interlaboratory comparisons. They should include appropriate technical, statistical and administrative skills.
• Reference laboratory: Laboratory that provides reference values on a test item
• Traceability: Property of the result of a measurement or the value of a standard whereby it can be related to stated references (usually national or international standards) through an unbroken chain of comparisons, all having stated uncertainties (VIM:1993, 6.10)
• Trueness: Closeness of agreement between the average value obtained from a large series of test results and an accepted reference value (ISO 3534-1)
• Uncertainty of measurement : Parameter associated with the results of a measurement that characterizes the dispersion of the values that could reasonably be attributed to the measurand (VIM:1993, 3.9)
|
Largely due to the requirement of ISO/TS 16949 that companies must use accredited testing or calibration labs whenever materials testing, safety testing or gauge calibration is required, there has been a heightened visibility of the ISO/IEC 17025 standard. This is good because metrologists and engineers understand that ISO/IEC 17025 provides an internationally recognized base for evaluating the competence of testing and calibration labs. Additional accredited labs means a higher degree of certainty that lab testing as a whole is reliable. When a testing or calibration lab says that it's accredited to ISO/IEC 17025, a lot rides on that statement. It means that the lab has been examined and found to be proficient within the scope of its registration.
ISO/IEC 17025 puts it this way in subclause 5.9.1:
"The laboratory shall have quality control procedures for monitoring the validity of tests and calibrations undertaken. The resulting data shall be recorded in such a way that trends are detectable and, where practicable, statistical techniques shall be applied to the reviewing of the results. This monitoring shall be planned and reviewed and may include, but not be limited to, the following:
a) regular use of certified reference materials and/or internal quality control using secondary reference materials;
b) participation in interlaboratory comparison or proficiency-testing programs;
c) replicate tests or calibrations using the same or different methods;
d) retesting or recalibration of retained items;
e) correlation of results for different characteristics of an item."
Of particular interest is subclause 5.9.1.b regarding proficiency testing. Unlike the type of proficiency testing that personnel might undergo to gain a professional certificate (typically a written test and direct observation of their working skills), testing-lab proficiency looks at the lab as a whole and compares real-world results of the lab under test to other labs performing the same types of tests. Lab proficiency testing ensures that a lab's personnel, equipment and procedures are capable of reliably and reproducibly performing calibrations or measurements.
The requirements for accrediting bodies performing accreditation of conformity assessment bodies, such as calibration and testing laboratories, are identified in ISO/IEC 17011—"Conformity assessment—General requirements for accreditation bodies accrediting conformity assessment bodies." This standard governs organizations, such as the International Laboratory Accreditation Cooperation and the Asia Pacific Laboratory Accreditation Cooperation, that are responsible for ensuring that labs meet the ISO/IEC 17025 standard. ISO/IEC 17011 states that:
" • accreditation bodies shall encourage laboratories to participate in proficiency testing;
• accredited labs shall participate in proficiency testing as required by the accreditation body and shall meet the performance requirements;
• the proficiency testing may be organized by the accreditation body or by another body that is judged competent;
• proficiency testing should be consistent with ISO/IEC Guide 43—'Proficiency testing by interlaboratory comparisons.'"
Most, if not all, ISO/IEC 17025-accredited laboratories are required to participate in proficiency testing or interlaboratory comparison (PT/ILC) schemes that have been developed to support various "program specialty areas," a broad term often interpreted as a field or discipline. For example, soils testing can be considered a program specialty.
The important aims of proficiency testing can be summarized as follows:
• Assist in maintaining the calibration of instruments
• Achieve confidence in testing performance of accredited laboratories
• Assist in detection of the reasons for interlaboratory differences
• Check the quality of a candidate in a specific area of testing/calibration
• Identify the laboratories that have to take corrective actions to increase their quality of testing/calibration.
Proficiency testing is the formal use of interlaboratory comparisons to determine the performance of individual laboratories for specific tests or measurements. In other words, proficiency testing is the use of samples distributed by an independent proficiency testing scheme's organizer thatenables a statistical evaluation and continual assessment of laboratory performance. According to ISO/IEC Guide 2, it is the "determination of laboratory testing performance by means of interlaboratory comparisons."
In general, proficiency testing techniques depend upon the nature of the test item, the method in use and the number of laboratories participating. The following are common types of proficiency testing schemes:
• Testing laboratory comparison schemes . Interlaboratory testing schemes involve randomly selected samples (anything from micrometers to pieces of steel) distributed simultaneously to participating testing laboratories for concurrent testing. After completion of the testing, the results are returned to the coordinating body and compared with the assigned values to give an indication of the performance of the individual laboratories and the group as a whole. This scheme serves the testing of food, body fluids, water, soils and other environmental material. In general you can find two types of laboratory comparisons:
— Intralaboratory comparison: If two or more staff members do the same tests regu- larly, the results should be close to each other.
— Interlaboratory comparison : From an international perspec- tive, if different lab- oratories perform identical tests, it is even more important that results are consis- tent.
In both cases, the results of the lab under test must fall within an acceptable zone compared to the results of the other labs. If they don't, this is an indication that something is wrong with the process, personnel or equipment at that lab. Results falling outside the acceptable zone, plus or minus the uncertainty value, are known as "outliers."
• Measurement laboratory comparison schemes. Measurement comparison schemes involve the item to be measured or calibrated being circulated successively from one participating laboratory to the next. Most commonly this involves comparing the individual measurement results with the reference values established by the reference laboratory. In this case, the coordinator takes into consideration the claimed measurement uncertainty of each participating laboratory. This is a critical component of the PT/ILC. Measurement uncertainty is a mathematical description of the errors with respect to a measurement. The claimed uncertainty is known as the best measurement capability (BMC) for that lab's scope or, in other words, the lowest achievable uncertainty. Different labs will have different BMCs, that value being dependent on a variety of factors, including equipment and environment. The items (e.g., measurement artifacts or the traveling standard) used in this type of proficiency testing include reference standards (e.g., standard resistors, standard capacitor, standard voltage cell, gauges and specific instruments).
• Known-value schemes . This is a special type of proficiency testing scheme. It may involve the preparation of test items with known amounts of the measurand under test. It is then possible to evaluate the capability of an individual laboratory to test the item and provide numerical results for comparison with the assigned value. This type of proficiency scheme does not need the involvement of multiple laboratories.
Metrologists may need to use other types of proficiency testing schemes, such as split-sample testing schemes, qualitative schemes and partial-process schemes. For more details about these types, please refer to "Proficiency Testing by Interlaboratory Comparisons" CAN-P-43, Standards Council of Canada, November 2001.
The following documents are related to proficiency testing:
• ISO/IEC Guide 43-1:1997—"Proficiency testing by interlaboratory comparisons (Part 1: Development and operation of proficiency testing schemes)"
• ILAC-G13:2007—"Guidelines for the Requirements for the Competence of Providers of Proficiency Testing Schemes"
• EN ISO/IEC 17020:2004—"General criteria of the operation of various types of bodies performing inspection"
• Eurachem Guide—"Selection, use and interpretation of proficiency testing schemes by laboratories"
• EN ISO/IEC 17025:2005—"General requirements for the competence of testing and calibration laboratories"
• ILAC-P9:2005—"ILAC Policy for Participation in National and International Proficiency Testing Activities"
Successful participation in a proficiency test means that the lab is performing consistently compared to other similar labs, and successful participation is an accreditation requirement. The following are important aspects that must be considered to accomplish successful proficiency testing:
• The test-method accuracy. The planned test method for the comparison should be clearly defined, technically accepted and reviewed for accuracy.
• The smallest detectable difference between the results . The smallest detectable difference between the results is necessary to evaluate the competence between the participants. This and the measurement uncertainty will demonstrate the performance of the lab, as reflected in the report.
• The number of participating laboratories. The proficiency test can be organized between two labs (i.e., bilateral comparison) or among any number of participants (i.e., multiple comparison). There is some discussion regarding the ideal number of labs for a proficiency test, but a larger number of laboratories will provide less variation resulting from statistical influence.
• The number of times a test is repeated. This number, for example 10 times, should be known to the participants and may be written into the protocol of the comparison as well. Repeating the test or calibration 10 times should produce a result with less variation than repeating it three times.
• The method of assigned value establishment. This method is suggested by the general coordinator (i.e., pivot lab) and is also mentioned in the protocol.
Proficiency test results can appear in many forms. There are, in general, three steps common to all proficiency tests when participants' results are to be investigated:
• Determination of the assigned value of the artifacts
• Calculation of performance statistics
• Evaluation of performance
Only the first item will be discussed here.
The assigned value of a sample should facilitate the evaluation of the lab's performance with respect to the standard or other labs, and at the same time encourages interlaboratory and intermethod agreement. The assigned value should be known clearly by the participants. Any misunderstanding may affect the results for one or more participants. It should also fall within the measurement capabilities of the participants. Where appropriate, the uncertainty of assigned values should be determined using procedures described in ISO Guide 98—"Guide to the expression of uncertainty in measurement (GUM)."
In general, there are three categories of the assigned value establishment:
• Type A (the known value)
• Type B (the certified reference material)
• Type C (the consensus value)
The known value usually is identified through expert agreement or specialized sampling (for example, the known composition). An example of a certified reference value is the hardness testing block, where the Rockwell C includes the parametric statistics (i.e., mean value of results) and robust statistics (i.e., median of all stated results). A consensus value may not have an absolute value or even be traceable, but everyone accepts the value. A prime example is the Rockwell block, which until recently was a consensus standard but now has a value, thanks to NIST's efforts.
Proficiency testing is the use of interlaboratory comparisons to determine the performance of individual laboratories for specific tests or measurements. Although individual laboratories may use such comparisons for internal verifications, accreditation bodies use proficiency testing as part of the overall assessment of a laboratory's ability to perform tests competently. Where it is required as part of the accreditation program, proficiency testing complements the on-site laboratory assessment by technical specialists and provides information to support the accreditation-decision process.
Proficiency testing is a key element in ensuring that a lab can accurately and reproducibly test or calibrate products to meet its customers' needs.
Mamdouh Halawa, Ph.D., is the coordinator of the regional NCSLI organization in Egypt. For 18 years, Halawa has been a researcher at the National Institute for Standards in Cairo, Egypt. He has a Ph.D. in electrical engineering from Ain Shams University in Cairo and is a member of numerous technical societies. He is the Egyptian contact for EUROMET, a technical assessor for the National Laboratories Accreditation Bureau (NLAB-Egypt—a full membership in ILAC/IEC) and a lecturer of engineering physics at Helwan University in Cairo.
Special thanks to Hershal Brewar of International Accreditation Service Inc. for his editorial help.
|
