Guidelines for experimental studies

Dent Mater 10:45-51, January, 1994

Guidelines for experimental studies Jacquelyn E. Moorhead 1, Pejaver V. Rao ~, Kenneth J. Anusavice 2

Division of Biostatistics, Department of Statistics, and 2Departmentof Dental Biomaterials, University of Florida, Gainesville,Florida, USA

ABSTRACT Objectives. The purpose of this document is to present guidelines for the training of researchers and for the planning of experiments that ensure optimum interaction and communication among the investigators and statisticians of the research team. Methods. The protocol described in this document is designed to systematically guide a researcher through a sequence of planning steps and checkpoints to ensure proper experimental design, data management, statistical analyses, and interpretation. Results. The investigator and the statistician, as a collaborative team, will be able to determine the most effective and appropriate procedures and tools necessary for the successful completion of a research project. Significance. Planning an experimental study necessitates much thought. Lack of essential communication with a statistician in the initial stages of the experimental design can doom the experiment to failure. Flawed research studies may be associated with one or more of the following: failure of the established testable hypothesis, uncontrolled biases, insufficient sample size, lack of appropriate controls, variability in specimen preparation, improper calibration of instruments or standardization of measurements, incorrect data entries, inappropriate statistical analyses, errors in interpretation of statistical analyses, and erroneously drawn conclusions. Standard guidelines will eliminate many of these potential problems.

INTRODUCTION The purpose of this document is to provide a set of guidelines for the procedures to be followed in the conduct of experimental research studies. The document outlines the issues that are involved in an experimental study, provides guidelines on how to approach these issues as a collaborative team, and yields a foundation for the successful completion of a research project. Further information on this topic can be found in the following references: Marks (1982); Mik~ and Stanley (1982); Mienert (1986); Hulley and Cummings (1988); Ott (1993). This document will show how the investigator and the statistician, as a collaborative team, must think about the data in a

disciplined and ordered manner. Together this team will be able to determine the most effective and appropriate procedures and tools to be utilized in a particular study. The team will know how and when to use these tools to greatest advantage and will be able to understand the variations that will occur in differing situations. Statistics deals with the planning, collection, analysis, interpretation and presentation of data. Thus, this team effort will begin long before the data are collected and will continue until all valid conclusions have been drawn and published. The guidelines in this document are presented in three phases. Phase I will describe pre-data-collection issues, including the research objective, the literature review, research methodology, the study plan, and the research proposal. A flowchart of the topics included in Phase I is shown in Fig. 1. Phase II will describe ongoing-data-collection issues including the collection, recording, and computerization of data; verification and editing of data; and preliminary analyses and interim reports. A flowchart of the topics included in Phase II is shown in Fig. 2. Phase III will describe post-data-collection issues including final data editing and analyses, the final report, and study documentation and publication of results. A flowchart of the topics included in Phase III is shown in Fig. 3. A detailed description of each phase follows.

METHODS PHASE I: PRE-DATA-COLLECTION ISSUES

The Research Objective. Before planning actually begins, there must be a clear statement of the objective of the study. This statement will specify what scientific questions the study is designed to answer. This statement can be in the form of questions that the study is intended to answer, hypotheses to be tested and/or effects to be estimated. The statement of objectives is essential for selecting the factors to be investigated (material, environmental condition, etc.), the response variables to be measured, the data needed to describe the effects of the factors, and the kind of statistical analysis required. There must also be a clear explanation of why the experiment is being conducted. Is this a study to confirm, refute or extend previous findings or to provide new findings? Is it designed Dental Materials~January 1994 45

Phase I Specific Aims

TABLE 1: EXPERIMENTALDESIGNTOPICS

STATISTICIAN

RESEARCHER

1

RESEARCH OBJECTIVE DESIGN ~IETHODS LITERATURE REVIEW

STUDY PLAN

RESEARCH PROPOSAL

Planning the measurements: measurement selection measurement classification description/relationship of variables validity and reliability Choosing the sample: sampling techniques evaluating bias inclusion/exclusion criteria recruitment goals Estimating sample size & power: sample size determination precision and accuracy assessment clinical/statistical significance Planning data management & analyses: collecting/recording/editing data statistical analyses methods Implementing the study: pilot study/pretesting quality control techniques

Fig. 1. Phase I flow chart of pre-data-collection issues.

primarily to test a research instrument? Is this a preliminary study for the purpose ofraising questions for future investigation or to answer immediate questions? Will the results be carried into practical use at once, or are they to be used to explain aspects of theory not adequately understood before? Answers to these questions will help to explain the significance and usefulness of the study and will give direction and focus to the study. The Literature Review. An essential part of research is the knowledge gained from earlier studies. A careful search of pertinent literature will: (1) provide background material and may suggest variables, techniques or measuring tools that could be useful and included in the study, (2) avoid unnecessary duplication, (3) include information that can be valuable in interpreting the conclusions of the study, and (4) present unique ideas useful to present and future studies. Every study reviewed in the literature search should be documented. A file should be kept on all studies reviewed. For each study reviewed, the file should include a complete citation, an evaluation of the limitations and strengths of each study, methods used, results obtained, and relevance to the study being planned. A careful, systematic literature review will enable one to (1) establish what has or has not been previously studied, (2) integrate one learned fact into another, (3) construct a structure for new understanding and knowledge of the problem, and (4) document and differentiate between the confirmation of previous findings and new advances in the field. Research Methodology. The research methodology should be decided in collaboration with the statistician. While the investigator is best qualified for making decisions on the experimental material and measurement techniques, the statistician is equipped with the skill and knowledge of experimental design 46 Moorhead et alJGuidelines for experimentalstudies

techniques and can put them into useful form for the researcher. By varying the experimental procedure, the cost, quality, and quantity of information can be varied in an experiment. The statistician contributes to the research methodology in two ways. First, the statistician assists in planning the design of the experiment that will lead to an efficient and successful research project. The design topics focus on: (1) planning measurements to ensure that they are relevant to the aims of the study, (2) choosing study subjects to avoid erroneous conclusions from biased sampling, (3) estimating sample size and power to ensure that the study has a small probability (a level Type I error} of rejecting a research hypothesis that is true, a high probability of supporting a research hypothesis that is true, and/or a high probability (confidence level) of providing estimates of effects with specified bounds on error; similarly, with the proper sample size and power, the study has a small probability of Type II error (~) of supporting a false hypothesis (Dawson-Saunders and Trapp, 1990), (4) planning for data management and analyses to ensure that the data are accurate and complete and that valid conclusions are drawn, and (5) implementing the study to minimize avoidable errors once the study has begun. Table 1 lists these design topics and the items of importance in each category. An accurate determination ofsample size is needed to achieve the research objective. This process can be very complex and even difficult for relatively simple research designs. There are several decisions that must be made to determine the sample size, and the investigator must provide the following information when meeting with the statistician: (1) statement of the research question, (2) the inference to be made (estimation or hypothesis testing) (3) expected variability between individual measurements of the response variable (standard deviation), (4) desired

Phase II Specific Aims COLLECT MEASUREMENTS Phase III Specific Aims RECORD MEASUREMENTS FINAL DATA EDITING YES

ERRORS

Researcher and Statistician review the completed data set after various edit and logic checks.

~-- NO

CORRECT ENTRY ERRORS

DATA A N A L Y S E S Descriptive Primary Backup

Edit by hand COMPUTERIZE DATA Make backup copy Look for missing, illogical and out-of-range data YES

ERRORS

FINAL REPORT Interpret the findings Draw conclusions Consider implications and applications

~ NO

CORRECT ENTRY ERRORS STUDY DOCUMENTATION Update backup L

copy

ACCUMULATE DATA BASE ON HARD DISK Examine distribution of variables for outliers. Printout to researcher "Examine consistency among variables YES = ERRORS NO

CORRECT ENTRY ERRORS Update backup ~ . copy

PRELIMINARY STATISTICAL ANALYSES

INTERIM REPORTS Fig. 2. Phase II flow chart of ongoing data collection issues.

accuracy for estimates of effects or magnitude of the difference between the effects that will be considered to be practically important, (5) the confidence coefficient for the estimation inference (90%, 95%, or 99%) or the desired values of a (usually .01 or .05) and [~ (usually 0.2, thus, power= 80%) for the hypothesis testing inference, and (6) a limit on the sample size that is practically feasible. Formulas for determining sample sizes will depend upon the experimental design and the assumptions about the distribution(s) of the responses. For example, suppose it is reasonable to assume that the responses have approximate normal distributions with

I Identify location of data files, data management, records, and all reports. PUBLICATION Fig. 3. Phase III flow chart of post-data-collection issues.

a common standard deviation o. Further suppose we need to ensure that an a-level test of the null hypothesis (Ho, that there is no difference between two means) will reject H o if the actual magnitude of the difference is at least as large as 5 = d/o where d is the magnitude of the difference between the effects that is considered practically important. Then the required sample size is the smallest n such that: {t(2(n-1),a/2)+t(2(n-

1), ~} 2

n>2 52 where t (a,b) is such that exactly 100(1- b)% of the values in a tdistribution with a degrees of freedom will be less than t (a,b). (Neter et al., 1985) In practice, n has to be determined using an appropriate trial and error method. For example, in an experiment to compare biaxial flexure strengths of disk specimens, it was estimated that a minimum of six specimens would be required. These sample sizes will ensure that there is at least an 80% probability that a 0.05 level test will show significance if the difference between group means is greater than two standard deviations. For example, if the mean flexure strength is 427.6 MPa with a typical standard deviation 0f62 MPa, there is an 80% chance that a 0.05 level test will detect Dental Materials~January 1994 47

TABLE 2: OUTLINE OF A STUDY PLAN Design issues: objectives and specific aims selection of experimental units specification of independent and dependent variables measurement selection estimation of sample size randomization to control bias replication to increase precision instrument calibration/measurement standardization to improve accuracy data management software data analyses methods anticipated difficulties

Administration: budget organizational chart biosketches of investigators resources, equipment and facilities

Operational issues: study time frame material and equipment budget and staffing training and certification quality control pretest/pilot study manual of operations

Design & statistical methods: overview of design study sample (selection, sampling, recruitment) measurements (independent and dependent variables) preliminary tests/pilot study sample size estimates, data analyses methods data management and quality control time table

Organizational issues: approvals and clearances organizational chart

Ethical considerations: risks and benefits to subjects safety of subjects confidentiality sharing of data

significance if the experimental mean is at least 124 MPa above or below the group mean. Statistical hypothesis tests control the probability of Type I error (rejecting a true null hypothesis). When the data do not support rejection of the null hypothesis, a decision to accept the null hypothesis requires one to know the probability of Type II error (accepting a false null hypothesis). Since this probability cannot be calculated, all one can do is to conclude that the data do not support the research hypothesis. Often useful information can be obtained by constructing a confidence interval for the difference. Such an interval will provide information about what one can say about the actual magnitude of the difference. Second, the statistician selects the appropriate methodology for analyzing the data. The method ofstatistical analysis depends upon the design of the study, the type of variables measured (nominal, ordinal or continuous), and the type of inference desired (estimation, hypothesis test, or prediction). The Study Plan. Planning an experimental study necessitates much thought. The study plan refers to the design of the study and all of the organizational and operational details needed to carry it out. The objective is to design the most feasible and least expensive study that will produce a correct answer to the research question. The study plan depends on (1) the purpose of the experiment, (2) physical restrictions on the process of taking measurements,(3) other restrictions imposed by limitations of time, money, and the availability of material and personnel and (4) the time and effort put into the planning stage of the experiment. Together the researcher and statistician must draft an outline of the study plan. Table 2 lists the essential topics to be in this outline. The study plan should gradually emerge from a repetitive 48 Moorhead et al./Guidelinesfor experimentalstudies

TABLE 3: COMPONENTSOF A RESEARCH PROPOSAL Introduction: abstract table of contents

Goals and rationale: research objective clinical/statistical significance preliminary studies, competence of investigators

Miscellaneous: consultants references appendices

process of designing, reviewing, pretesting and revising. As the pieces of the study plan are fit into place, the research protocol begins to take shape. The research protocol is a detailed document ofthe study which evolves from the outline ofthe study plan. This document presents all of the elements of the study in an organized and clear manner. Just as the study plan is a precursor to the research protocol, the research protocol is the precursor to the research proposal. The Research Proposal. The research proposal is a document written for the purpose ofobtaining funds from granting agencies or approval from institutional review boards. It contains the study protocol and other administrative and supporting information. Normally, the proposal can be divided into a number ofparts including: the introduction, administration, goals and rationale, designs and statistical methods, ethics and miscellaneous items. Each of these parts can then be divided into various subparts. Table 3 lists the items that make up a successful research proposal. PHASE II: ONGOING-DATA-COLLECTIONISSUES Inferences drawn from a study are only as good as the data upon which they are based. Unless the data are complete and accurate, conclusions drawn from them can be misleading. Thus it is important to develop a good plan for data management that can be implemented well. In this section, a step-by-step guideline is provided for dealing with the data from the time they are

measured to the time they are ready for analysis. The goal is to establish and maintain an efficient data entry and data management system that protects the integrity of the data generated in the various projects. Fig. 2 is a flowchart showing the specific aims in Phase II. Collecting the Data. The following guidelines are designed to ensure a timely flow of data measurement, data recording and data entry activities. Verifying study procedures. All data collection personnel must meet with the statistician to review study procedures before data collection begins. The procedure for data collection must ensure that: (1) all measurement techniques are valid, (2) there are no variables that are not measured, (3) data are collected in such a way that facilitates data entry, and (4) the calculated sample sizes are attainable. Maintaining documentation of data. All data collection personnel are required to prepare a notebook/logbook with a detailed description of the methods section of the protocol. This description should tell exactly how to perform all of the procedures in the study. It logs equipment maintenance dates, gives a description of each variable measured, contains the specimen coding guide, and serves as the Raw Data Source. Maintaining a good logbook is essential for reducing random variation and changes in measurement technique over time. Pretesting study procedures. All data collection personnel are required to work through a sample set of data and to pretest the study procedures. The main purpose of this exercise is to (1) ensure that the methods are so constructed that different persons using the same method and tools will obtain the same kind of data, (2) reveal inadequacies or inconsistencies in the data, (3) assess the accuracy, validity and reliability of the instruments, (4) identify and suggest appropriate changes in the protocol before the study begins, and (5) examine alternative methods at an early stage. Preserving quality control. All data collection personnel should (1) always be aware of and identify deficiencies in collecting the data, (2) always adhere to the data collection and testing schedule, (3) establish good sample labeling procedures, (4) check equipment periodically, and (5) record the data at the time the measurements are taken. Once the study has begun, it is essential to adhere to the study protocol. If this is unrealistic, the statistician and the investigator in charge should be consulted before any protocol changes are made. Only the collaborative team can make a change in the data collection protocol. Recording the Data. Direct computerized data entry is usually not practical. Recording of the data should be done in two stages first, in a logbook, then in a computerized data base. Logbook data entry. It is important to record the data at the same time the measurements are made. Alogbook to hold the raw data is needed to document the data collection and entry processes. Maintenance of a logbook will minimize data loss and transcription errors. The logbook should serve as the source of information about the raw data. A logbook should contain a methods section, a description of each variable, and a coding guide. The logbook must always be readily available to the collaborative team. All logbook entries must be (1) written in ink, (2) easy to read, (3) dated, and (4) initialed. Every item must have a unique identifying number. The format and layout must be such that all items are arranged to minimize a split across columns or pages. The unit of measurement of each variable must be specified, and

all recordings of a specified variable should be made using the same unit. Raw data used to make any summary calculations, as well as calculated parameters (means, ratios, etc.) should be recorded. Continuous variables should not be recorded in categorical form. If there is a need to change the classification of a variable, contact the statistician. Items requiring a clock time should indicate whether the time recordings are a.m. or p.m. It is best to use the 24 hour system. Ifa 12 hour recording system is used, use 12 noon/12 midnight. Computerized data entry. A data entry program should be used to generate the data base and must be selected during the design planning stage of each project. Many data entry programs are quite sophisticated, versatile, and easy to use. A data entry program should be selected that has many field checking features, establishes a system for editing data and can create data files that can be readily input into standard statistical software packages. Only trained personnel should be assigned the task of data entry. These individuals will be able to enter, verify, and perform preliminary editing of the data. The data entry program should not be modified without consulting the data manager. All data must be entered into the computer on the same day or within a few days after data collection. Timely data entry prevents accumulation of backlogs, allows early identification of data collection problems, minimizes the possibility of data loss, and permits the investigator to examine periodic printouts of the data. An updated backup copy of the data must be created after every data entry session. Verifying Data. Both logbook data and computerized data should be verified in a timely manner. The data manager should be consulted on the most appropriate method of data verification. Steps to be taken for verifying logbook data are to: (1) review all data collected and recorded in the experiment, (2) report all measurements made, without any censoring (decisions on whether or not to eliminate an outlier should be made by the collaborative team), (3) visually check for missingresponses, (4)visually check for illogical responses, and (5) set up, maintain, and document a system to monitor equipment readings. The following three steps can be taken for verifying computerized data. Step One- Program Checks the data entry program will be programmed to flag missing and out-of-range values. Step Two - Visual Checks for many small studies, when data are collected in small subsets, the best approach is to compare visually the raw data source from the logbook and a computer printout. Step Three -Double Entry Checks when the data are entered in large batches, all the data should be entered a second time. The computer will be programmed to flag those values that are not concordant with the first entries. Editing the Data. A uniform procedure should be followed for correcting errors in logbook and computerized entries. Errors in the logbook should be corrected by hand by the data recording personnel. One single line must be used to cross out the error and the correction initialed. Errors found while using the data entry program should be corrected by the data entry personnel. After the data file is given to the data manager, a final set of corrections should be made using a computerized error checking program. These corrections should be verified by the researcher and the statistician using an updated listing. Preliminary_ Analysis (edit checks) and Interim Reports. Edit checks should be continually applied during the data gathering period. The problems that these checks will detect can occur at any time. Examine the minimum, maximum and frequency Dental Materials~January 1994 49

TABLE 4: COMPONENTS OF A FINAL REPORT Title: relay the purpose of the study limited number of words Introduction: research objective significance of the study review of literature description of theoretical framework discussion of research hypotheses Experimental design & study procedures: details of the study design description of the sample description of the instruments description of the methods and procedures Statistical methodology: descriptive statistics description of statistical techniques analyses of the data Results: summary of the data collected results of all statistical tests tables, graphs and charts Discussion: explanation of findings interpretation of findings comparisons of findings with those of others application for the area of research implication for further study Summary: a brief statement of the problem, the method used, findings, conclusions and implications

distribution for each variable to make certain that all data appear reasonable. Plot the data and look for problems. Make logic checks, e.g., must be ">" instead of">". Interim reports are written during the time that data are being collected. The purposes of these reports are to monitor and assess the quality and character of the data, and to assess the need for creation of new variables by computation or transformation. Quality control. Quality control techniques should be established to maximize the completeness, consistency and quality of the data. These techniques begin during the planning stage and continue throughout the study. Some important considerations in development ofquality control protocols include: (1) specifying overview and responsibilities, (2) maintaining training procedures, (3) using uniform sample labeling, (4) conducting pretesting, (5) documenting equipment maintenance, (6) initiating regular meetings, (7) periodically tabulating measurements, (8) issuing periodic progress reports, (9) conducting external calibration sessions, and (10) scheduling internal and external site visits. 50 Moorhead et aUGuidelines for experimental studies

TABLE 5: OUTLINE FOR A RESEARCH PUBLICATION Title: descriptive title authors selected key words source of financial support Abstract: objective of the study description of treatment groups description of the methodology primary outcome measure/statistical analysis results significance of the findings Introduction: historical background literature review significance objectives Methodology: study population organizational structure treatments randomization outcome measures statistical issues and techniques statistical analyses quality control Results: tables, graphs and charts results of statistical tests Discussion: explanation of the findings comparison of findings with literature implications and significance of the findings brief summary Conclusions: concise statements of inference, significance or consequences; do not repeat results or discussion Acknowledgements: research funding support individuals who were not major contributors other contributions References: authors of related research publications data analyses methods laboratory methods Appendix:* glossary, codes additional tables, analyses sample data forms *Most scientific publications do not accept appendices to journal articles.

PHASE III: POST-DATA-COLLECTION ISSUES Eventually, the data-collection phase will be complete and all of the data will have been entered into the data base. At this point, the data have gone through various edits and logic checks.

The data will be reviewed again by the statistician and the researcher using the most recent computer listing. To complete the editing process, the frequency distribution is examined for each variable collected, looking for faulty values that may have survived previous efforts to edit the data set. Final corrections are then made prior to initiation of the data analyses. At this point of the study, the investigators and the statistician should feel confident about the accuracy and completeness of the data. A flowchart of the aims in Phase III is displayed in Fig. 3. Analysis ofthe Data. The analysis ofthe data includes computing the statistics, drawing conclusions, and deciding on the correct method ofreporting the findings. Usually, three types of analyses are performed. First is the preliminary analysis which is descriptive and/or graphical to familiarize the statistician with the data and to provide a foundation for all subsequent analyses. These analyses may include pie or bar charts, histograms, stem-andleaf or box plots, frequency distributions, numerical measures for describing central tendency (mean, mode, median) and variability (range, variance, standard deviation, percentile), and correlations. Second is the primary analysis which is used to address the objectives of the study. The choice of the analytic approach was mainly determined by the study design and classification of the independent and dependent variables in the study plan. Third is the backup analysis which employs alternative methods for examining the data and confirms the results of the primary analyses. It might include new statistical methods that are not as readily accepted as the more standard ones. Now the investigators and the statisticians are ready to interpret the findings, draw conclusions, and consider implications and applications of the findings. The Final Report. When the statistical analyses have been completed, conclusions must be drawn and the results communicated in a formal written report. This final report should present an objective, complete and thorough account of what was done in each step of the study. The research findings should be readable and available to a wide range of individuals. Requirements for a well written research report are conciseness, clarity, honesty, completeness, and accuracy. Items usually included in a final report are: the title, introduction, experimental design, statistical methodology, results, discussion, summary and conclusions. Table 4 lists these essential items and a description of each. Study Documentation. It is important to provide detailed documentation for all data processing and the statistical analyses so that the data files are readily accessible. The reviewer can then follow what has been done, redo it, or extend the analyses. The study documentation file should include: the study plan, data management information (including the coding guide, the raw data source and the storage of data files), statistical reports, interim reports, and a final report. Publication of Results. An important responsibility of the researcher should be to communicate the study findings to the research community in a published manuscript. The goal of this publication is to provide a clear and concise description of the study and all relevant design details and results. The writing of this manuscript must involve a team effort and include tables, charts and figures. In addition to the sections included in the final report (title, introduction, experimental design, statistical methodology, results, discussion and summary or conclusions), the manuscript submitted for publication should include sections for an abstract, references and acknowledgements. Table 5 lists a typical outline for publication of results.

Decisions to publish or not to publish the results of a study should be based on several factors including the scientific significance and the clinical significance of the results whether or not the null hypothesis has been rejected. In some cases, additional experiments are designed and performed to supplement the original study. For example, the original study results may have shown that the mean strength of material A was significantly different than that of material B. However, to answer the question of why the materials exhibited a different strength behavior, a new set of specimens is needed to test one of several new hypotheses. The data derived from the supplementary experiment should be considered as pilot data for a more comprehensive future study.

SUMMARY This document provides a framework for designing and conducting controlled experimental research. The proposed guidelines followa sequence ofstages including overall objectives, literature review, preliminary experimental design, protocol development, data collection, data recording, data verification, statistical analyses, and preparation of research reports and manuscripts. The systematic use of these guidelines will minimize the risk of common research errors including inadequate justification of a study, poorly focused planning, inappropriate sample size, inefficient data management, improper statistical analyses, and erroneous interpretation of statistical results. By reducing these risks, the frequency and quality of research programs are enhanced and allocation of resources optimized.

ACKNOWLEDGEMENT Preparation ofthis document was supported by Grants DE09307 and DE06672 from the National Institute of Dental Research, Bethesda, MD 20892 USA. Received June 23, 1993 / Accepted August 18, 1993 Address correspondence and reprint requests to: Jacquelyn E. Moorhead Division of Biostatistics Health Science Center P.O. Box 100212 Gainesville, FL 32610-0212 USA

REFERENCES Dawson-Saunders B, Trapp RC (1990). Basic and Clinical Biostatistics. Norwalk: Appleton and Lange. 95-96. Hulley S, Cummings S (1988). Designing Clinical Research. Baltimore: Williams & Wilkins. Marks R (1982). Designing a Research Project. New York: Von Nostrand Reinhold Company. Mienert C (1986). Clinical Trials. New York: Oxford University Press. Ott L (1993). An Introduction to Statistical Methods and Data Analysis. Belmont, CA: Wadsworth Inc. Mik~ V, Stanley KE, Editors, (1982). Statistics in Medical Research. New York: John Wiley & Sons. Neter J, Wasserman W, Kutner M (1985). Applied Linear Statistical Methods, 2nd ed. Homewood, Illinois: Richard D. Irwin, Inc. Dental Materials~January 1994 51