- Email: [email protected]
Accuracy, repeatability, reproducibility
Accuracy, Repeatability, Reproducibility By Francis Reilly, Director o Marketing, Matrix Metrologies 1340-6 Lincoln Avenue, Holbrook, N Y 11741; 631-580-1290; www.matrixmetrologies.com
ccuracy, repeatability, and reproducibility are three terms that are used extensively yet are poorly understood. In manufacturing a product, the producer must be responsible for delivering accurate output in accordance with a given specification. This must be performed to within a certain level of tolerance --precision. It must be able to be duplicated - - reproducible. In order for a process to consistently output accurate, precise, and reproducible product, these parameters must be understood and monitored (metrics). In general, manufacturers will use some collection of analytical tools and gauges to gather the metrics required to perform this task. Each of these devices will measure some parameter of the process to determine that specific process variables' accuracy, repeatability, and reproducibility attributes. U n d e r s t a n d i n g this information allows subsequent process improvement. The following definitions describe accuracy, repeatability, and precision within the context of measuring variations within a prQcess:
A
• Accuracy: Refers to how closely a m e a s u r e m e n t value obtained by an instrument conforms to the actual ("absolute") value of the sample.
1
i
• Repeatability: The ability of an i n s t r u m e n t to produce consistent results when performing multiple measurements on the same sample area. Also referred to as precision. • Reproducibility: The ability of an instrument to obtain consistent m e a s u r e m e n t results w h e n measuring the same sample at different times, or made by different operators, or by using different instruments of the same type. ILLUSTRATING THE DIFFERENCES The classical bull's eye chart is a great way to envision and r e m e m b e r the differences between these concepts. In the following charts the x represents the output of a specific process, such as the plating thickness of a part. Successive xs are additional parts coming off of the line. The goal of this process is to be accurate (in the middle of the range) and to be able to be accurate again and again (precise). Random error represents the inability to be consistent. Sometimes you hit the t a r g e t and sometimes you don't. Systemic error, on the other hand, r e p r e s e n t s a consistent problem or bias in the process. You are repeatable but you are not accurate. For example, you consistently plate too thick.
11
Figure 1. Summarizes the interaction of accuracy, repeatability, and reproducibility. Series 1 Series 2 Neither Accurate Precise but Inaccurate Nor Precise Random error controlled; Random and Systematic Systematic error still present Errors both present 8
1
Series 3 Accurate and Precise Both Random and Systematic errors under control Metal Finishing
rESTING & CONTROl
Figure 2. Diagram compares a highly repeatable series of measurements in the top series of bullseyes and a not-reproducible series of measurements in the bottom targets. Think of each bullseye as representing measurements of sequential batches. Series I Series 2 Series 3 Time 1 Time2 Time 3
REPRODUCIBILITY Reproducibility is perhaps the most misunderstood concept of the three. It is effectively the ability to reproduce results at a different time or place. Figure 2 shows both a highly reproducible and and a nonreproducible process. In this example, each measurement set is the exact same set of measurements repeated at future intervals. View this as three successive batches coming out of a process. The concepts in this article apply to a wide range
of processes. The specific p a r a m e t e r s being measured can be anything - - current, voltage, pH, concentrations of specific chemistry, t e m p e r a t u r e , thickness, harness, reflectivity. Controlling the quality of the work passing through a shop demands constant vigilance. It requires knowing w h a t to test, and it requires knowing what balance of accuracy, precision, and reproducibility are required to meet customer expectations.
mf
'~ihe i i i d i t ~ i f y s i~ecog|tized | i i t e f l i a | l o n a J a u i ~ o f i i y s i n c e | 9 0 3 " Be a part of this tradition, ideas and manuscripts welcome.
Metal Finishili 9 Mayaziiie 360 Park Avenue South, New York, NY 10010 212-633-3100
May 2004
9
ccuracy, repeatability, and reproducibility are three terms that are used extensively yet are poorly understood. In manufacturing a product, the producer must be responsible for delivering accurate output in accordance with a given specification. This must be performed to within a certain level of tolerance --precision. It must be able to be duplicated - - reproducible. In order for a process to consistently output accurate, precise, and reproducible product, these parameters must be understood and monitored (metrics). In general, manufacturers will use some collection of analytical tools and gauges to gather the metrics required to perform this task. Each of these devices will measure some parameter of the process to determine that specific process variables' accuracy, repeatability, and reproducibility attributes. U n d e r s t a n d i n g this information allows subsequent process improvement. The following definitions describe accuracy, repeatability, and precision within the context of measuring variations within a prQcess:
A
• Accuracy: Refers to how closely a m e a s u r e m e n t value obtained by an instrument conforms to the actual ("absolute") value of the sample.
1
i
• Repeatability: The ability of an i n s t r u m e n t to produce consistent results when performing multiple measurements on the same sample area. Also referred to as precision. • Reproducibility: The ability of an instrument to obtain consistent m e a s u r e m e n t results w h e n measuring the same sample at different times, or made by different operators, or by using different instruments of the same type. ILLUSTRATING THE DIFFERENCES The classical bull's eye chart is a great way to envision and r e m e m b e r the differences between these concepts. In the following charts the x represents the output of a specific process, such as the plating thickness of a part. Successive xs are additional parts coming off of the line. The goal of this process is to be accurate (in the middle of the range) and to be able to be accurate again and again (precise). Random error represents the inability to be consistent. Sometimes you hit the t a r g e t and sometimes you don't. Systemic error, on the other hand, r e p r e s e n t s a consistent problem or bias in the process. You are repeatable but you are not accurate. For example, you consistently plate too thick.
11
Figure 1. Summarizes the interaction of accuracy, repeatability, and reproducibility. Series 1 Series 2 Neither Accurate Precise but Inaccurate Nor Precise Random error controlled; Random and Systematic Systematic error still present Errors both present 8
1
Series 3 Accurate and Precise Both Random and Systematic errors under control Metal Finishing
rESTING & CONTROl
Figure 2. Diagram compares a highly repeatable series of measurements in the top series of bullseyes and a not-reproducible series of measurements in the bottom targets. Think of each bullseye as representing measurements of sequential batches. Series I Series 2 Series 3 Time 1 Time2 Time 3
REPRODUCIBILITY Reproducibility is perhaps the most misunderstood concept of the three. It is effectively the ability to reproduce results at a different time or place. Figure 2 shows both a highly reproducible and and a nonreproducible process. In this example, each measurement set is the exact same set of measurements repeated at future intervals. View this as three successive batches coming out of a process. The concepts in this article apply to a wide range
of processes. The specific p a r a m e t e r s being measured can be anything - - current, voltage, pH, concentrations of specific chemistry, t e m p e r a t u r e , thickness, harness, reflectivity. Controlling the quality of the work passing through a shop demands constant vigilance. It requires knowing w h a t to test, and it requires knowing what balance of accuracy, precision, and reproducibility are required to meet customer expectations.
mf
'~ihe i i i d i t ~ i f y s i~ecog|tized | i i t e f l i a | l o n a J a u i ~ o f i i y s i n c e | 9 0 3 " Be a part of this tradition, ideas and manuscripts welcome.
Metal Finishili 9 Mayaziiie 360 Park Avenue South, New York, NY 10010 212-633-3100
May 2004
9