- Email: [email protected]
Variation and Control of Process Behavior
Int. J. Radiation Oncology Biol. Phys., Vol. 71, No. 1, Supplement, pp. S210–S214, 2008 Copyright Ó 2008 Elsevier Inc. Printed in the USA. All rights reserved 0360-3016/08/$–see front matter
doi:10.1016/j.ijrobp.2007.05.096
QA FOR RT SUPPLEMENT
VARIATION AND CONTROL OF PROCESS BEHAVIOR TODD PAWLICKI, PH.D.,* AND MATTHEW WHITAKER, B.S.y * Department of Radiation Oncology, University of California, San Diego, La Jolla, CA; and y Radiological Imaging Technology, Inc., Colorado Springs, CO The purpose of this work was to highlight the importance of controlling process variability for successful quality assurance (QA). We describe the method of statistical process control for characterizing and controlling a process. Traditionally, QA has been performed by comparing some important measurement (e.g., linear accelerator output) against a corresponding specification. Although useful in determining the fitness of a particular measurement, this approach does not provide information about the underlying process behavior over time. A modern view of QA is to consider the time-ordered behavior of a process. Every process displays characteristic behaviors that are independent of the specifications imposed on it. The goal of modern QA is, not only to ensure that a process is ontarget, but that it is also operating with minimal variation. This is accomplished by way of a data-driven approach using process behavior charts. The development of process behavior charts, historically known as control charts, and process behavior (action) limits are described. The effect these concepts have on quality management is also discussed. Ó 2008 Elsevier Inc. Process behavior, Variation, Statistical process control, Action limits.
acceptable or unacceptable, and no attention was paid to the variations within the specifications. Statistical process control might be new to radiotherapy but it is not new to healthcare (2, 3). In radiotherapy QA, process behavior charts were recently applied to daily linear accelerator output verification as a pilot project (4). Other earlier applications of process behavior charts in radiotherapy included investigating the variability in target volume delineation for breast cancer (5, 6). As a part of their data analysis, these latter studies also used other QA tools such as the Gage R&R and Pareto analyses. SPC can play a role in error reduction or performance improvement strategies (7, 8). Financial performance improvement and monitoring naturally lend themselves to application of process behavior charts and SPC techniques (9). A patient’s medical performance can be analyzed using SPC for making data-based clinical decisions. SPC, for example, has proved beneficial to monitoring peak expiratory flow rates in asthma care (10). The application of SPC has been suggested for patient behavioral analysis (11) and debated in published studies (12, 13). More research is needed for this application of SPC and process behavior charts. As we discuss, adopting SPC as a QA tool changes the way one views quality. Process behavior charts facilitate the idea of characterizing a process and continually trying to improve that process by minimizing variations. The purpose of this
INTRODUCTION In this work, we describe statistical process control (SPC) as a tool for quality assurance (QA). The history of SPC can be traced back to the introduction of the control chart (or process behavior chart) by Walter A. Shewhart on May 16, 1924 (1). Shewhart was working for Bell Telephone Laboratories at the time and was engaged in research on problems of quality associated with mass production. A major emphasis of this research was on the application of probability and statistics to quality. SPC was applied in practice during the post-World War II reconstruction of Japan. The effect of SPC can be dramatic, as evidenced by Japan being known as a world leader in quality of manufactured products long before the United States and other Western nations reached Japan’s standard of quality. Shewhart’s insight was to provide a simple, robust graphic tool to assist process operators in making real-time decisions about process behavior. As we explain, the process behavior chart is a time-ordered graph with action limit lines drawn about the historical average of the subgrouped data from a process. The lines are calculated such that the probability is very low that any data points found outside the action limits will result from chance. The application of process behavior charts leads to an understanding that the goal of QA is operating on-target with minimal variance. This is in contrast to the conventional wisdom in which quality was either
92093-0843. Tel: (858) 822-6058; Fax: (858) 822-6078; E-mail: [email protected] Conflict of interest: none. Received March 9, 2007. Accepted for publication May 21, 2007.
Reprint requests to: Todd Pawlicki, Ph.D., Department of Radiation Oncology, University of California, San Diego, Moores Cancer Center, 3855 Health Sciences Dr., P.O. Box 0843, La Jolla, CA S210
Variation and Control of Process Behavior d T. PAWLICKI AND M. WHITAKER
report is to provide an overview of SPC, the development of process behavior charts, and the implications of adopting a process view of QA. TRADITIONAL APPROACH TO QA The traditional approach to QA is shown in the flowchart of Fig. 1. In this approach, a measurement is performed (for which there are specifications) that is important for successful production from a process. The measurement is compared with the specification. If the specification is met, all is well and work continues. If the specification is not met, time and effort are expended in remeasuring and determining the cause of the failure. Although useful for sorting individual outcomes as acceptable or unacceptable, the shortcoming of this approach is that only a binary decision can be made as a result of the measurement. This leads to periods of benign neglect of the process followed by occasions of (possibly intense) troubleshooting when a result does not meet the specification. The reason for this is that the focus is on whether the result is within specifications without concern for how the process varies within those specifications. The inspection of individual outcomes has been the mainstay of QA and continues to persist in radiotherapy quality control. Achieving and measuring true process improvement is not possible with the traditional approach to QA. MODERN QA TOOL Statistical process control is a method for using time-ordered data to draw inductive conclusions about the underlying process. The characteristics of a process are used to derive action limits. The actions limits are completely independent of the specifications imposed on the process. The method of SPC consists of continuously charting data from the process, and thereby monitoring the process, while taking steps to minimize process variability. The process behavior charts allow one to distinguish between two contributors to process variation. One contributor to variation is routine and the other is exceptional. Routine variation is the regular ‘‘noise’’ within the process. Exceptional variation is a ‘‘signal’’ (i.e., an event whose cause can probably be determined through immediate investigation). In principle, it is possible to determine the exact reason for every variation in a set of measurements; however, it can be difficult and is rarely worth the effort if the magnitude of variation is not clinically relevant. A process is ‘‘in control’’ when it displays only routine variation. In modern terminology, a process in control is said to be predictable. This is because, if the process is in control and nothing in the process changes, one can predict that the process will vary randomly within the action limits. In essence, the action limits on process behavior charts (process behavior limits) allow one to separate exceptional and routine variations. If one had perfect knowledge, all process variation would be a signal and could be eliminated. Perfect knowledge, however, is not a realistic goal in the real-world clinical environment.
S211
Begin
Determine specifications
Test the production or function
Not within specifications (unacceptable) Inspect result
Determine reason
Within specifications (acceptable) Document
End
Fig. 1. Flow diagram depicting conventional approach to quality, which leads to binary view of quality where a measurement is within specifications (don’t worry about it) or outside specifications (something is wrong, fix it, and remeasure).
Two process behavior charts are defined to gain insight into process behavior. One chart provides a measure of location for the data and another chart provides a measure of dispersion for the data. The general development of process behavior charts using the mean as a measure of location and the range as a measure of dispersion is given below. Other measures of location and dispersion can also be used such as the median, root mean square, or standard deviation. Consider a QA process from which data points, x1, x2, x3,., can be measured. An example of this would be daily beam symmetry verification by sampling four points in the open field for which one obtains data points at 5 cm from the central axis along both principal axes. Four data points would be measured each day. All measurements taken together after several days would produce a data set with a mean m(x) and standard deviation s(x). To create the process behavior charts, one gathers data into homogeneous subgroups as the data points come from the process. In this example, all four measurements would be taken as a homogeneous subgroup, because all four data points would be expected to be approximately equal for a single measurement of a symmetric open beam (e.g., single exposure of a diode array or film). For each subgroup, the mean, x, is calculated
S212
I. J. Radiation Oncology d Biology d Physics
and the range, R, is calculated. The process behavior limits for process location are calculated by mðxÞ t,sðxÞ and the process behavior limits for process variation are calculated by mðRÞ t,sðRÞ The parameter t is a scaling factor for the limits and was empirically set to a value of 3 in Shewhart’s original work (14). The question, then, is how to easily estimate the summary statistics mðxÞ, sðxÞ, mðRÞ, and sðRÞ. The charts were developed to be used by workers typically on the factory floor doing the QA measurements. Those workers were not usually educated in statistical applications. Furthermore, at the time the charts were developed (1924), computer software to do the calculations was not available. All these factors led to the final form of the equations to calculate process behavior limits as
and
¼ R X 3, pffiffiffi d2 n
R 3,
d3 R d2
The parameters d2 and d3 vary according to the subgroup size n and relate m(R) and s(R) to s(x), which is the standard de is the average viation of the original measured data set (15). X of subgroup averages used to calculate the process behavior limits. R is the average of the subgroup ranges used to calculate the process behavior limits. X is the centerline for the location process behavior chart, and R is the centerline for the dispersion process behavior chart. When a subgroup exceeds an action limit, one should immediately investigate and remove the cause. One can see that the action limits on the process behavior charts provide an operational definition for what constitutes a random progression of a process. The limits are a decision rule for interpreting whether the data are subject to exceptional (nonrandom) variation. Other criteria are available to help identify exceptional variation on process behavior charts in addition to a subgroup falling outside the process behavior limits (t = 3). Three other criteria for detecting exceptional variation on process behavior charts are as follows (16): 1. Two of three successive subgroups falling on the same side of the centerline for t = 2 limits. 2. Four of five successive subgroups falling on the same side of the centerline for t = 1 limits. 3. At least eight successive subgroups falling on the same side of the centerline. In addition to these, other detection rules have been developed over the years (17). Each detection rule carries a small risk of producing a false alarm. As the number of rules increases, so does the probability of responding to false alarms. The number of rules should be limited to keep this in check.
Volume 71, Number 1, Supplement, 2008
The optimal use of the detection rules depends on the purposes of the chart and any a priori knowledge of possible failure modes. This development of process behavior charts makes no assumption about the underlying distribution of data coming from the process. Charts can also be constructed, however, according to theoretical probability distributions. In this case, the parameters d2 and d3 are replaced by a single statistic to create the action limits. The caveat of this approach is that one must know the distribution model of the data before constructing the charts. An inappropriate theoretical model will yield incorrect limits. The limits would be incorrect because they would result in either too many false alarms or would miss too many signals. However, some benefits exist to using the theoretical approach instead of the empirical approach for process behavior charts. The appropriateness of the theoretical vs. empirical process behavior charts in radiation oncology is an area that requires investigation. Homogeneous subgroups simply mean that several measurements (n $ 1) of the process are obtained under the same conditions. Determining subgroup homogeneity is a critical step in creating useful process behavior charts. The subgroups capture and display systematic changes in the process. If the subgroups are statistically consistent over time, the process will be stable. If the subgroups are not statistically consistent over time, the process will be unstable and subject to both routine and exceptional variations. SPC AND QUALITY MANAGEMENT Statistical process control is the basis for the view that a process can be in one of four possible states (17) (Fig. 2). The four possible states of a process are I, predictable and within specifications; II, predictable and out of specifications; III, unpredictable and within specifications; and IV, unpredictable and out of specifications. The diligent use of process behavior charts is a method of improving process results. The first steps in SPC are to determine the measurements critical to monitor the process and to determine an effective method of measurement. Every clinic has limited resources, and everything cannot be measured. Therefore, other techniques of modern QA such as Failure Modes and Effects Analysis and cause-and-effect diagrams can be used to suggest which measurements are most suitable for process monitoring. Once the measurements have been obtained, the subgroups are plotted on the process behavior charts and used to calculate the action limits. Subgroups outside the action limits indicate that exceptional variation has occurred in the process. If process behavior charts have not been previously used, a process is likely to be in State III or IV. A departure from conventional QA wisdom is that in SPC one should work to eliminate exceptional variation even when the process is performing within specifications. This creates an environment of continual improvement. SPC is an inductive and interactive method and is performed as the process is running, in contrast to a global analysis several days, weeks, or months after the data have been collected.
Variation and Control of Process Behavior d T. PAWLICKI AND M. WHITAKER
measurements can be investigated as candidates for the SPC method.
State I Within specifications & Predictable
DISCUSSION
State II Out of specifications & Predictable
Three acts are included in the operation of control (18):
(only sources of routine variation are present)
If a process is neglected
S213
Continual improvement (sources of exceptional variation are present)
State III Within specifications & Unpredictable State IV Out of specifications & Unpredictable
Fig. 2. A process can be in one of four states as shown in this schematic.
Over time, as the causes of exceptional variation are eliminated, the process will contain only routine variation. The process is then predictable within the action limits. At this time, the process will be in either State I or State II. Exceptional variation could occur from time to time, and the process behavior charts will demonstrate that. At this point, working to minimize routine variation in the process will be much more effective, and one can then effectively compare the variability of the process to the specifications. A number of measurements are available to do this. The most commonly used indexes are process capability and centered process capability (16). If the variability of a predictable process is still greater than or equal to the specifications, the process is unacceptable in its present state (State II). In this case, changes must be made to the underlying processes to reduce variability. This can be thought of as re-engineering the process with improved equipment and/or procedures. Process behavior charts will also indicate whether the process re-engineering has been effective. This continues until the process variability has been reduced to much less than the specification width and the process has reached State I. The process then continues to be monitored indefinitely. Once a process has been demonstrated to be stable for an extended period, it is reasonable to reduce the frequency of collecting subgroups for process analysis. Stopping measurements altogether, however, is not advisable. Left to its own devices, a process will naturally deteriorate and end up in State III or State IV. The ultimate goal is a process centered on target with variability small enough that even when the occasional exceptional variation occurs the likelihood of exceeding the specifications is remote. A predictable process can be corrected when the action limits are exceeded much easier than when the specifications are exceeded. When a process takes much less time to monitor and get under control, new critical
1. Mental judgments that two or more observations are made under the same conditions. 2. Mathematical operations to construct the criteria of control. 3. Physical operations to look for assignable causes when the subgroups do not satisfy the criteria of control. From this, one can see that expert process knowledge is required to achieve optimal quality. Someone without expert process knowledge cannot reasonably be expected to identify an assignable cause in a process that displays exceptional variation. SPC and process behavior charts are based on the science of probability and statistics but require experience to use properly. The use of these techniques does not diminish the role of the person doing the QA but engages the person and focuses their efforts effectively. The cycle of charting homogeneous subgroups, identifying sources of exceptional variation, and removing them is continued indefinitely to identify and reduce the variation in the process. Often in radiotherapy research, retrospective analysis of measurements is done using global dispersion statistics that assume homogeneity of the entire data set. Grouping data together in this manner can lead to the burying of the signals contained in the data. In clinical practice, it is not possible to know without process measurements whether a process is subject to only routine variation or both routine and exceptional variation. To bring out the process characteristics, one must use a localized dispersion statistic. This is a reason subgroups are used. Correct construction of process behavior charts requires data homogeneity only within each individual subgroup rather than in an entire data set. Thus, SPC is similar to the statistical techniques of analysis of variance by using local variation to distinguish a signal from the background noise of a process. Data points present themselves sequentially, and each data point requires a decision about process control. Process behavior charts assist the process operator in making this decision in a rational, consistent, and effective manner. The techniques of hypothesis testing (e.g., t test or chi-square tests) can be used retrospectively to analyze data but do not assist in real-time decision-making. Quality is a real-time venture that requires intervention immediately on detecting a significant change in the process. CONCLUSION The goal of using process behavior charts is to reduce process variability far within the clinical requirements (specifications) so that even when a data point shows lack of process control, the process is still functioning within the specifications. This allows the process to continue even as the process
S214
I. J. Radiation Oncology d Biology d Physics
operator searches for the causes of the out-of-control behavior. SPC is one of many tools available for optimal QA, and it should be used routinely with other QA tools, such as Pareto
Volume 71, Number 1, Supplement, 2008
charts, Failure Modes and Effects Analysis, cause-and-effect diagrams, flowcharts, histograms, and process capability studies.
REFERENCES 1. Best M, Neuhauser D. Walter A Shewhart, 1924, and the Hawthorne factory. Qual Saf Health Care 2006;15:142–143. 2. Carey RG, Lloyd RC. Measuring quality improvement in healthcare: A guide to statistical process control applications. Milwaukee: ASQ Quality Press; 2001. 3. Kelley DL. How to use control charts for healthcare. Milwaukee: ASQ Quality Press; 1999. 4. Pawlicki T, Whitaker M, Boyer AL. Statistical process control for radiotherapy quality assurance. Med Phys 2005;32: 2777–2786. 5. Holli K, Laippala P, Ojala A, et al. Quality control in health care: An experiment in radiotherapy planning for breast cancer patients after mastectomy. Int J Radiat Oncol Biol Phys 1999; 44:827–833. 6. Pitkanen MA, Holli KA, Ojala AT, et al. Quality assurance in radiotherapy of breast cancer—Variability in planning target volume delineation. Acta Oncol 2001;40:50–55. 7. Carey RG, Teeters JL. CQI case study: Reducing medication errors. J Qual Improvem 1995;21:232–237. 8. Clark DE, Cushing BM, Bredenberg CE. Monitoring hospital trauma mortality using statistical process control. J Am Coll Surg 1998;186:630–635. 9. Smolko JR, Greisler DS. Understanding medical group financial and operational performance: The synergistic effect of linking
10. 11. 12.
13. 14. 15. 16. 17. 18.
process control and profit and loss. J Health Care Finance 2001;27:64–75. Boggs PB, Wheeler DJ, Washburne WF, et al. Peak expiratory flow rate control chart in asthma care: Chart construction and use in asthma care. Ann Allerg Asthma Im 1998;81:552–562. Pfadt A, Wheeler DJ. Using statistical process control to make data-based clinical decisions. J Appl Behav Anal 1995;28: 349–370. Hantula DA. Disciplined decision making in an interdisciplinary environment: Some implications for clinical applications of statistical process control. J Appl Behav Anal 1995;28: 371–377. Hopkins BL. Applied behavior analysis and statistical process control? J Appl Behav Anal 1995;28:379–386. Shewhart WA. Economic control of quality of manufactured product. New York: D. Van Nostrand Company, Inc.; 1931. Wheeler DJ. Normality and the process behavior chart. Knoxville: SPC Press; 2000. Wheeler DJ, Chambers DS. Understanding statistical process control. 2nd ed. Knoxville: SPC Press; 1992. Wheeler DJ. Advanced topics in statistical process control. Knoxville: SPC Press; 1995. Shewhart WA. Statistical methods from the viewpoint of quality control. Mineola: Dover Publications; 1986.
doi:10.1016/j.ijrobp.2007.05.096
QA FOR RT SUPPLEMENT
VARIATION AND CONTROL OF PROCESS BEHAVIOR TODD PAWLICKI, PH.D.,* AND MATTHEW WHITAKER, B.S.y * Department of Radiation Oncology, University of California, San Diego, La Jolla, CA; and y Radiological Imaging Technology, Inc., Colorado Springs, CO The purpose of this work was to highlight the importance of controlling process variability for successful quality assurance (QA). We describe the method of statistical process control for characterizing and controlling a process. Traditionally, QA has been performed by comparing some important measurement (e.g., linear accelerator output) against a corresponding specification. Although useful in determining the fitness of a particular measurement, this approach does not provide information about the underlying process behavior over time. A modern view of QA is to consider the time-ordered behavior of a process. Every process displays characteristic behaviors that are independent of the specifications imposed on it. The goal of modern QA is, not only to ensure that a process is ontarget, but that it is also operating with minimal variation. This is accomplished by way of a data-driven approach using process behavior charts. The development of process behavior charts, historically known as control charts, and process behavior (action) limits are described. The effect these concepts have on quality management is also discussed. Ó 2008 Elsevier Inc. Process behavior, Variation, Statistical process control, Action limits.
acceptable or unacceptable, and no attention was paid to the variations within the specifications. Statistical process control might be new to radiotherapy but it is not new to healthcare (2, 3). In radiotherapy QA, process behavior charts were recently applied to daily linear accelerator output verification as a pilot project (4). Other earlier applications of process behavior charts in radiotherapy included investigating the variability in target volume delineation for breast cancer (5, 6). As a part of their data analysis, these latter studies also used other QA tools such as the Gage R&R and Pareto analyses. SPC can play a role in error reduction or performance improvement strategies (7, 8). Financial performance improvement and monitoring naturally lend themselves to application of process behavior charts and SPC techniques (9). A patient’s medical performance can be analyzed using SPC for making data-based clinical decisions. SPC, for example, has proved beneficial to monitoring peak expiratory flow rates in asthma care (10). The application of SPC has been suggested for patient behavioral analysis (11) and debated in published studies (12, 13). More research is needed for this application of SPC and process behavior charts. As we discuss, adopting SPC as a QA tool changes the way one views quality. Process behavior charts facilitate the idea of characterizing a process and continually trying to improve that process by minimizing variations. The purpose of this
INTRODUCTION In this work, we describe statistical process control (SPC) as a tool for quality assurance (QA). The history of SPC can be traced back to the introduction of the control chart (or process behavior chart) by Walter A. Shewhart on May 16, 1924 (1). Shewhart was working for Bell Telephone Laboratories at the time and was engaged in research on problems of quality associated with mass production. A major emphasis of this research was on the application of probability and statistics to quality. SPC was applied in practice during the post-World War II reconstruction of Japan. The effect of SPC can be dramatic, as evidenced by Japan being known as a world leader in quality of manufactured products long before the United States and other Western nations reached Japan’s standard of quality. Shewhart’s insight was to provide a simple, robust graphic tool to assist process operators in making real-time decisions about process behavior. As we explain, the process behavior chart is a time-ordered graph with action limit lines drawn about the historical average of the subgrouped data from a process. The lines are calculated such that the probability is very low that any data points found outside the action limits will result from chance. The application of process behavior charts leads to an understanding that the goal of QA is operating on-target with minimal variance. This is in contrast to the conventional wisdom in which quality was either
92093-0843. Tel: (858) 822-6058; Fax: (858) 822-6078; E-mail: [email protected] Conflict of interest: none. Received March 9, 2007. Accepted for publication May 21, 2007.
Reprint requests to: Todd Pawlicki, Ph.D., Department of Radiation Oncology, University of California, San Diego, Moores Cancer Center, 3855 Health Sciences Dr., P.O. Box 0843, La Jolla, CA S210
Variation and Control of Process Behavior d T. PAWLICKI AND M. WHITAKER
report is to provide an overview of SPC, the development of process behavior charts, and the implications of adopting a process view of QA. TRADITIONAL APPROACH TO QA The traditional approach to QA is shown in the flowchart of Fig. 1. In this approach, a measurement is performed (for which there are specifications) that is important for successful production from a process. The measurement is compared with the specification. If the specification is met, all is well and work continues. If the specification is not met, time and effort are expended in remeasuring and determining the cause of the failure. Although useful for sorting individual outcomes as acceptable or unacceptable, the shortcoming of this approach is that only a binary decision can be made as a result of the measurement. This leads to periods of benign neglect of the process followed by occasions of (possibly intense) troubleshooting when a result does not meet the specification. The reason for this is that the focus is on whether the result is within specifications without concern for how the process varies within those specifications. The inspection of individual outcomes has been the mainstay of QA and continues to persist in radiotherapy quality control. Achieving and measuring true process improvement is not possible with the traditional approach to QA. MODERN QA TOOL Statistical process control is a method for using time-ordered data to draw inductive conclusions about the underlying process. The characteristics of a process are used to derive action limits. The actions limits are completely independent of the specifications imposed on the process. The method of SPC consists of continuously charting data from the process, and thereby monitoring the process, while taking steps to minimize process variability. The process behavior charts allow one to distinguish between two contributors to process variation. One contributor to variation is routine and the other is exceptional. Routine variation is the regular ‘‘noise’’ within the process. Exceptional variation is a ‘‘signal’’ (i.e., an event whose cause can probably be determined through immediate investigation). In principle, it is possible to determine the exact reason for every variation in a set of measurements; however, it can be difficult and is rarely worth the effort if the magnitude of variation is not clinically relevant. A process is ‘‘in control’’ when it displays only routine variation. In modern terminology, a process in control is said to be predictable. This is because, if the process is in control and nothing in the process changes, one can predict that the process will vary randomly within the action limits. In essence, the action limits on process behavior charts (process behavior limits) allow one to separate exceptional and routine variations. If one had perfect knowledge, all process variation would be a signal and could be eliminated. Perfect knowledge, however, is not a realistic goal in the real-world clinical environment.
S211
Begin
Determine specifications
Test the production or function
Not within specifications (unacceptable) Inspect result
Determine reason
Within specifications (acceptable) Document
End
Fig. 1. Flow diagram depicting conventional approach to quality, which leads to binary view of quality where a measurement is within specifications (don’t worry about it) or outside specifications (something is wrong, fix it, and remeasure).
Two process behavior charts are defined to gain insight into process behavior. One chart provides a measure of location for the data and another chart provides a measure of dispersion for the data. The general development of process behavior charts using the mean as a measure of location and the range as a measure of dispersion is given below. Other measures of location and dispersion can also be used such as the median, root mean square, or standard deviation. Consider a QA process from which data points, x1, x2, x3,., can be measured. An example of this would be daily beam symmetry verification by sampling four points in the open field for which one obtains data points at 5 cm from the central axis along both principal axes. Four data points would be measured each day. All measurements taken together after several days would produce a data set with a mean m(x) and standard deviation s(x). To create the process behavior charts, one gathers data into homogeneous subgroups as the data points come from the process. In this example, all four measurements would be taken as a homogeneous subgroup, because all four data points would be expected to be approximately equal for a single measurement of a symmetric open beam (e.g., single exposure of a diode array or film). For each subgroup, the mean, x, is calculated
S212
I. J. Radiation Oncology d Biology d Physics
and the range, R, is calculated. The process behavior limits for process location are calculated by mðxÞ t,sðxÞ and the process behavior limits for process variation are calculated by mðRÞ t,sðRÞ The parameter t is a scaling factor for the limits and was empirically set to a value of 3 in Shewhart’s original work (14). The question, then, is how to easily estimate the summary statistics mðxÞ, sðxÞ, mðRÞ, and sðRÞ. The charts were developed to be used by workers typically on the factory floor doing the QA measurements. Those workers were not usually educated in statistical applications. Furthermore, at the time the charts were developed (1924), computer software to do the calculations was not available. All these factors led to the final form of the equations to calculate process behavior limits as
and
¼ R X 3, pffiffiffi d2 n
R 3,
d3 R d2
The parameters d2 and d3 vary according to the subgroup size n and relate m(R) and s(R) to s(x), which is the standard de is the average viation of the original measured data set (15). X of subgroup averages used to calculate the process behavior limits. R is the average of the subgroup ranges used to calculate the process behavior limits. X is the centerline for the location process behavior chart, and R is the centerline for the dispersion process behavior chart. When a subgroup exceeds an action limit, one should immediately investigate and remove the cause. One can see that the action limits on the process behavior charts provide an operational definition for what constitutes a random progression of a process. The limits are a decision rule for interpreting whether the data are subject to exceptional (nonrandom) variation. Other criteria are available to help identify exceptional variation on process behavior charts in addition to a subgroup falling outside the process behavior limits (t = 3). Three other criteria for detecting exceptional variation on process behavior charts are as follows (16): 1. Two of three successive subgroups falling on the same side of the centerline for t = 2 limits. 2. Four of five successive subgroups falling on the same side of the centerline for t = 1 limits. 3. At least eight successive subgroups falling on the same side of the centerline. In addition to these, other detection rules have been developed over the years (17). Each detection rule carries a small risk of producing a false alarm. As the number of rules increases, so does the probability of responding to false alarms. The number of rules should be limited to keep this in check.
Volume 71, Number 1, Supplement, 2008
The optimal use of the detection rules depends on the purposes of the chart and any a priori knowledge of possible failure modes. This development of process behavior charts makes no assumption about the underlying distribution of data coming from the process. Charts can also be constructed, however, according to theoretical probability distributions. In this case, the parameters d2 and d3 are replaced by a single statistic to create the action limits. The caveat of this approach is that one must know the distribution model of the data before constructing the charts. An inappropriate theoretical model will yield incorrect limits. The limits would be incorrect because they would result in either too many false alarms or would miss too many signals. However, some benefits exist to using the theoretical approach instead of the empirical approach for process behavior charts. The appropriateness of the theoretical vs. empirical process behavior charts in radiation oncology is an area that requires investigation. Homogeneous subgroups simply mean that several measurements (n $ 1) of the process are obtained under the same conditions. Determining subgroup homogeneity is a critical step in creating useful process behavior charts. The subgroups capture and display systematic changes in the process. If the subgroups are statistically consistent over time, the process will be stable. If the subgroups are not statistically consistent over time, the process will be unstable and subject to both routine and exceptional variations. SPC AND QUALITY MANAGEMENT Statistical process control is the basis for the view that a process can be in one of four possible states (17) (Fig. 2). The four possible states of a process are I, predictable and within specifications; II, predictable and out of specifications; III, unpredictable and within specifications; and IV, unpredictable and out of specifications. The diligent use of process behavior charts is a method of improving process results. The first steps in SPC are to determine the measurements critical to monitor the process and to determine an effective method of measurement. Every clinic has limited resources, and everything cannot be measured. Therefore, other techniques of modern QA such as Failure Modes and Effects Analysis and cause-and-effect diagrams can be used to suggest which measurements are most suitable for process monitoring. Once the measurements have been obtained, the subgroups are plotted on the process behavior charts and used to calculate the action limits. Subgroups outside the action limits indicate that exceptional variation has occurred in the process. If process behavior charts have not been previously used, a process is likely to be in State III or IV. A departure from conventional QA wisdom is that in SPC one should work to eliminate exceptional variation even when the process is performing within specifications. This creates an environment of continual improvement. SPC is an inductive and interactive method and is performed as the process is running, in contrast to a global analysis several days, weeks, or months after the data have been collected.
Variation and Control of Process Behavior d T. PAWLICKI AND M. WHITAKER
measurements can be investigated as candidates for the SPC method.
State I Within specifications & Predictable
DISCUSSION
State II Out of specifications & Predictable
Three acts are included in the operation of control (18):
(only sources of routine variation are present)
If a process is neglected
S213
Continual improvement (sources of exceptional variation are present)
State III Within specifications & Unpredictable State IV Out of specifications & Unpredictable
Fig. 2. A process can be in one of four states as shown in this schematic.
Over time, as the causes of exceptional variation are eliminated, the process will contain only routine variation. The process is then predictable within the action limits. At this time, the process will be in either State I or State II. Exceptional variation could occur from time to time, and the process behavior charts will demonstrate that. At this point, working to minimize routine variation in the process will be much more effective, and one can then effectively compare the variability of the process to the specifications. A number of measurements are available to do this. The most commonly used indexes are process capability and centered process capability (16). If the variability of a predictable process is still greater than or equal to the specifications, the process is unacceptable in its present state (State II). In this case, changes must be made to the underlying processes to reduce variability. This can be thought of as re-engineering the process with improved equipment and/or procedures. Process behavior charts will also indicate whether the process re-engineering has been effective. This continues until the process variability has been reduced to much less than the specification width and the process has reached State I. The process then continues to be monitored indefinitely. Once a process has been demonstrated to be stable for an extended period, it is reasonable to reduce the frequency of collecting subgroups for process analysis. Stopping measurements altogether, however, is not advisable. Left to its own devices, a process will naturally deteriorate and end up in State III or State IV. The ultimate goal is a process centered on target with variability small enough that even when the occasional exceptional variation occurs the likelihood of exceeding the specifications is remote. A predictable process can be corrected when the action limits are exceeded much easier than when the specifications are exceeded. When a process takes much less time to monitor and get under control, new critical
1. Mental judgments that two or more observations are made under the same conditions. 2. Mathematical operations to construct the criteria of control. 3. Physical operations to look for assignable causes when the subgroups do not satisfy the criteria of control. From this, one can see that expert process knowledge is required to achieve optimal quality. Someone without expert process knowledge cannot reasonably be expected to identify an assignable cause in a process that displays exceptional variation. SPC and process behavior charts are based on the science of probability and statistics but require experience to use properly. The use of these techniques does not diminish the role of the person doing the QA but engages the person and focuses their efforts effectively. The cycle of charting homogeneous subgroups, identifying sources of exceptional variation, and removing them is continued indefinitely to identify and reduce the variation in the process. Often in radiotherapy research, retrospective analysis of measurements is done using global dispersion statistics that assume homogeneity of the entire data set. Grouping data together in this manner can lead to the burying of the signals contained in the data. In clinical practice, it is not possible to know without process measurements whether a process is subject to only routine variation or both routine and exceptional variation. To bring out the process characteristics, one must use a localized dispersion statistic. This is a reason subgroups are used. Correct construction of process behavior charts requires data homogeneity only within each individual subgroup rather than in an entire data set. Thus, SPC is similar to the statistical techniques of analysis of variance by using local variation to distinguish a signal from the background noise of a process. Data points present themselves sequentially, and each data point requires a decision about process control. Process behavior charts assist the process operator in making this decision in a rational, consistent, and effective manner. The techniques of hypothesis testing (e.g., t test or chi-square tests) can be used retrospectively to analyze data but do not assist in real-time decision-making. Quality is a real-time venture that requires intervention immediately on detecting a significant change in the process. CONCLUSION The goal of using process behavior charts is to reduce process variability far within the clinical requirements (specifications) so that even when a data point shows lack of process control, the process is still functioning within the specifications. This allows the process to continue even as the process
S214
I. J. Radiation Oncology d Biology d Physics
operator searches for the causes of the out-of-control behavior. SPC is one of many tools available for optimal QA, and it should be used routinely with other QA tools, such as Pareto
Volume 71, Number 1, Supplement, 2008
charts, Failure Modes and Effects Analysis, cause-and-effect diagrams, flowcharts, histograms, and process capability studies.
REFERENCES 1. Best M, Neuhauser D. Walter A Shewhart, 1924, and the Hawthorne factory. Qual Saf Health Care 2006;15:142–143. 2. Carey RG, Lloyd RC. Measuring quality improvement in healthcare: A guide to statistical process control applications. Milwaukee: ASQ Quality Press; 2001. 3. Kelley DL. How to use control charts for healthcare. Milwaukee: ASQ Quality Press; 1999. 4. Pawlicki T, Whitaker M, Boyer AL. Statistical process control for radiotherapy quality assurance. Med Phys 2005;32: 2777–2786. 5. Holli K, Laippala P, Ojala A, et al. Quality control in health care: An experiment in radiotherapy planning for breast cancer patients after mastectomy. Int J Radiat Oncol Biol Phys 1999; 44:827–833. 6. Pitkanen MA, Holli KA, Ojala AT, et al. Quality assurance in radiotherapy of breast cancer—Variability in planning target volume delineation. Acta Oncol 2001;40:50–55. 7. Carey RG, Teeters JL. CQI case study: Reducing medication errors. J Qual Improvem 1995;21:232–237. 8. Clark DE, Cushing BM, Bredenberg CE. Monitoring hospital trauma mortality using statistical process control. J Am Coll Surg 1998;186:630–635. 9. Smolko JR, Greisler DS. Understanding medical group financial and operational performance: The synergistic effect of linking
10. 11. 12.
13. 14. 15. 16. 17. 18.
process control and profit and loss. J Health Care Finance 2001;27:64–75. Boggs PB, Wheeler DJ, Washburne WF, et al. Peak expiratory flow rate control chart in asthma care: Chart construction and use in asthma care. Ann Allerg Asthma Im 1998;81:552–562. Pfadt A, Wheeler DJ. Using statistical process control to make data-based clinical decisions. J Appl Behav Anal 1995;28: 349–370. Hantula DA. Disciplined decision making in an interdisciplinary environment: Some implications for clinical applications of statistical process control. J Appl Behav Anal 1995;28: 371–377. Hopkins BL. Applied behavior analysis and statistical process control? J Appl Behav Anal 1995;28:379–386. Shewhart WA. Economic control of quality of manufactured product. New York: D. Van Nostrand Company, Inc.; 1931. Wheeler DJ. Normality and the process behavior chart. Knoxville: SPC Press; 2000. Wheeler DJ, Chambers DS. Understanding statistical process control. 2nd ed. Knoxville: SPC Press; 1992. Wheeler DJ. Advanced topics in statistical process control. Knoxville: SPC Press; 1995. Shewhart WA. Statistical methods from the viewpoint of quality control. Mineola: Dover Publications; 1986.