- Email: [email protected]
Telephone surveying with computers: Administrative, methodological and research issues
Evaluation and Program Planning, Vol. 9, pp. 107-l 11, 1986 Printed in the USA. All rights reserved.
0149-7189/86.$3.00 + .OO Copyright o 1986 Pergamon Journals Ltd
TELEPHONE SURVEYING WITH COMPUTERS: ADMINISTRATIVE, METHODOLOGICAL AND RESEARCH ISSUES
RALPH Department of Public Administration,
F. SHANGRAW. JR.
Maxwell School of Citizenship and Public Affairs, Syracuse University
ABSTRACT Computer-Assisted Telephone Interviewing (CATI) is a tool designed to assist interviewers in almost every aspect of a telephone survey. This article reviews the current status of CATI with special attention to microcomputer-based systems. Most of the literature in this area has focused on the implementation and design of CA TI systems. This paper takes a broader perspective by examining the adminktrative, methodological and research issues surrounding the use of CATI. The primary conclusion is that CATI systems can aid in the administration of telephone surveys in addition to reducing sources of survey error. However, these systems are neither inavpensive nor easily implemented. Perhaps the greatest potential for CATI is in the area of research, where their structured use can lead us to a better understanding of the survey research process in general.
INTRODUCTION With the advent of the computer, automated operations of previously manual tasks became possible in a variety of areas. In the field of survey research, the statistical analysis of results relieved researchers of the arduous task of hand calculating statistics. As computer systems matured, other survey research applications increased. One area that has received a great deal of attention in recent years is computer-aided telephone interviewing (CATI). CAT1 is a technique that aIlows almost all aspects of a telephone survey to be conducted with the assistance of a computer. The first systems appeared in the commercial sector around I972, with initial prototype systems developed by Chilton and Amrigon. Government and academic systems grew both out of the success of the commercial systems and out of the need for more efficient and cost-effective telephone surveying techniques,’ The initial justification for CAT1 was reduced costs for survey administration. Recent reports (Shanks, 1983; Groves, 1983) have deemphasized cost savings, especially for first-time users of CATI. Current re-
searchers argue that the real benefits of CAT1 are methodological and sample quality improvements, especially interviewer error reduction. Freeman (1983) claims that “CATI, over the long run, will result in a qualitatively different set of norms and expectations regarding the survey: optimistically, we should be able, through CATI, to improve markedly the art of data collection.” (Freeman, 1983:144) The purpose of this paper is to review of the current status of CATI, with special attention given to microcomputer-based telephone interviewing systems. Microcomputers are playing a greater role in the day-today activities of many researchers yet the academic literature has virtually ignored any discussions of microcomputer CATI systems.’ The first section of the paper will explore the administrative, methodological and research issues surrounding the utilization of CATI systems. In the second part of the paper, the design and implementation of a microcomputer-based CATI system is discussed.
GENERAL IMPLICATIONS OF CAT1 Much of the work on CAT1 systems has concentrated on the development, implementation, and refinement of the computer software that drives these systems. Now that some of the packages have been in use for several years, researchers are moving beyond the nuts
and bolts and into research on the influence of CAT1 on survey methods. The findings from these recent studies can be grouped into three categories: administrative, methodological, and research issues. 107
108
RALPH F. SHANGRAW
Administrative Issues Improved respondent selection is often cited as a primary admi~strative advantage of CATI (Paiit and Sharp, 1983; Shanks and Nicholls, 1981). CAT1 programs improve respondent selection in several ways. First, “random digit dialing” (Cummings, 1979) can be employed to generate random samples. Local telephone exchanges are entered into the CAT1 program and the software generates a random sample of telephone numbers. Second, most programs can also monitor sampling frequencies for individual stratum of a stratified sample. Periodically, the researcher can output a frequency table and immediately identify small cell problems. Third, the growth of computer data bases on survey populations will greatly streamline the conduct of telephone interviews.3 Information from these systems can provide a convenient front end for CAT1 systems. In addition to aiding in sampling procedures, CAT1 can monitor the administration of the survey. When the interviewee is called, the CAT1 program generates a response code indicating if the number is busy, if there is no answer, if a follow-up call is required, or if the interview is refused. The system will prompt the interviewer with the same number for recall if there is a busy number or no answer. These automated scheduling procedures greatly reduce the amount of time that the project coordinator spends on administrative issues. The general consensus on interviewer training and recruitment is that CATI is more costly than manual methods, especially for initial training (Frey, 1983; Shanks and Nicholls, 1981). At least 80 to 90 hours of training over a three week period were required for the California Disability Study before the interviewers were comfortable with the system (Shanks and Nicholls, 1981). Three developments are likely to offset these costs in the future. First, as the population in genera1 becomes more computer literate, less training on the operational aspects of the system will be necessary. Second, the computer-aided instruction (CAI) packages can provide programmed learning modules on CAT1 systems, though currently few survey groups have developed CA1 for CAT1 systems (Shanks, 1983). Third, CAT1 systems are being upgraded so that the interviewer needs to know very little about the computer. As these systems evolve, training costs will reduce. Contrary to early speculation, the costs of CAT1 systems have not been dramatically lower than manual procedures. Researchers have consistently underestimated the time and costs required for software design and debugging. Software costs will remain high as long as these systems continue to be developed for specific survey applications. The current movement is towards “generic” CAT1 systems that can be modified easily for any survey schedule in a variety of survey environ-
ments (marketing research, academic surveys, evaluations studies, etc.). The availability of “off the shelf” software will improve the cost ratio between manual and automated systems. While costs may be higher, interviewer efficiency is expected to improve with CAT1 systems. The University of Wisconsin’s Survey Research Laboratory estimated that “an experienced CAT1 interviewer’s productivity (as measured by the number of sample points contacted per production hour) may be as much as 20% higher than that of the same interviewer using traditional survey methods.” (Palit and Sharp, 1983) In the short term, manual procedures will have to accompany the computerized systems. Paper copies of interviewing schedules are necessary in case of hardware or software failures. One study (Groves and Mathiowetz, 1984) found that interviewers preferred the paper version of the survey schedule because it was easier to correct errors. Supervision will be similar in a manual or automated setting. In the case of CATI, however, the supervisor will be able to monitor both the phone conversation and the CRT input. Methodological Issues Administrative survey research issues cannot be distinguished easily from methodological concerns, especially when improvements in administration coincidentally reduce survey error.4 Reduced interviewer error, for example, is one methodological benefit of CAT1 systems. Error reduction can be grouped into three categories: skip logic or branching; response coding; and aided recall. CAT1 relieves the interviewer of following complicated procedures when the response to one question dictates the asking of another question. Branching procedures, which are generally desirable from a data collection perspective, are often sources of interviewer confusion and error (Shure and Meeker, 1978). These capabilities also allow researchers to develop branching operatons that were never before possible with manual methods. Another advantage of CAT1 is the ability of the system to recognize instantly illegal or inconsistent responses or skipped questions. When an improper i~sponse is recorded at the terminal, the systems responds with an error message prompting the interviewer for a legal response. As with manual systems, it is possible for the interviewer to record an incorrect response in a legal format. Post interview error checking procedures, with the corollary expense of obtaining correct or missing data points, can be eliminated with CAT1 (Shure and Meeker, 1978). It is also possible for the computer to detect “yea-saying” by the interviewee or any other response pattern. Once detected, the CAT1 program can reverse the scale for future questions or ask follow-up questions on previous responses. While aided-recall and question clarification pro-
Telephone surveying with computers cedures have been an integral component of manual systems, these processes are implemented easily on CAT1 systems. Consistent and properly worded additional information can reduce interviewer error frequently described in the literature (for example, see Hanson and Marks, 1958). On-line reference materials and branching clarification algorithms can contribute to the system’s aided-recall or “intelligent probing” capabilities (Fink, 1983). In addition to reductions in interviewer error, CAT1 systems “permit new solutions to old problems” when open-ended questions are included in the survey (Groves, 1983). Of primary benefit is the ability of the CAT1 system to record responses to open-ended questions in a machine-readable format.5 Some CAT1 systems have been designed to automatically update short response categories to open-ended questions when a new response is encountered. Other systems generate a list of all open-ended responses to allow researchers to add new categories on a periodic basis. With the evolution of text analysis software, machine-readable openended responses could be evaluated for occurrences of phrases or keywords and, in the future, for ideas (Groves, 1983). Three other methodological issues can be addressed by CAT1 systems. McFarland (1981) has found that the type of question and the level of specificity will determine if position bias will occur. Position bias, in these instances, can be countered by the random assignment of question order by the computer. Some response bias can be reduced through the personalization of the survey. Pronouns can be replaced throughout the survey schedule to reflect the gender of the in-
109
terviewee or the name of the inteviewee can be inserted to personalize the schedule. Finally, and most obvious, transposition errors are eliminated with CAT1 systems since there is no transfer of the response data from a hard-copy document to a machine-readable format.
Research Issues Freeman (1983) makes a valid point when he suggests that researchers should “use CAT1 as a means of testing our premises about survey procedures rather than simply employing the technology indiscriminately and without evaluation.” (Freeman, 1983:152) As he notes, “CAT1 permits increased control over the interpersonal relationship between interviewer and respondent and the presentation of stimuli.” (Freeman, 1983:148) He suggests that a research agenda be established to test much of the “folklore” that is an integral part of survey research. Some research organizations are initiating empirical studies on CAT1 systems. The U.S. Census Bureau, for example, has completed two field test of computerassisted telephone interviewing systems (Nicholls, 1983). The field tests had three purposes: (a) to compare CAT1 with manual methods especially in terms of response rates, quality of data obtained, and speed of data collection; (b) to evaluate the selection and training of CAT1 interviewers and “to identify factors in their performance;” and (c) to improve CAT1 systems for future field tests and general use (Nicholls, 1983). Results from this research are forthcoming and should provide some interesting insight into the use of CAT1 systems.
CATI IN A MICROCOMPUTER ENVIRONMENT time is virtually instantaneous. On a large mainframe While much of the recent literature has focused on mainframe or minicomputer CAT1 applications, few or minicomputer time-sharing system, the slow reresearchers have addressed the use of microcomputers sponse rate at peak use can be annoying to both the interviewer and interviewee. Screen display, especially in in the survey research laboratory. Palit and Sharp (1983) discussed the implementation of a CAT1 system terms of color and graphics, is generally easier to on a time-shared minicomputer with microcomputers manipulate on microcomputers. Colors can be used to at individual interviewing stations. This configuration emphasize words or questions; graphics can be used allows the interviewer to execute the survey at their into display frequency counts and preliminary statistical dividual stations with the additional benefits of imresults. mediate uploading of the survey data to the minicomThe hardware disadvantages, especially for largeputer for further manipulation. For small to medium scale surveys, greatly outweigh the benefits. Permasize surveys, the microcomputer is an attractive alternent and internal storage are important consideranative to manual systems. The potential use of micros tions. Large data sets will mandate the use or purchase in survey research is increasing as researchers begin to of hard disk storage systems for most users. While it is use microcomputers in other aspects of their work. likely that the interview schedule can be stored on a The following discussion will center on the sole use of diskette, most surveys will require a hard disk for the microcomputers for computer-aided telephone surresponse data. Large internal storage (RAM) will be necessary if a large data base will be sorted or if a veying. statistical package is used to analyze the data. There Hardware Considerations are further problems when several microcomputers are The microcomputer has both advantages and shortused to conduct the same survey. Until Local Area comings. One advantage is that the computer response Network (LAN) software is perfected, data from the
110
RALPH F. SHANGRAW
individual machines will have to be merged before it can be processed. It is imperative that checks are included in the software to eliminate interview redundancy. As microcomputers grow in size, multi-tasking operating systems and multi-processors will enable data input from several terminals. Software Issues At present, most researchers who are interested in using CAT1 on microcomputers will have to design and develop their own software. Three options are available for software development. First, the researcher or research group could program the system using BASIC or another higher level programming language. Since execution speed may be a consideration, the system could be programmed in C or Assembler. Fortunately, the experience with mainframe CATI systems will provide guidance on system design and features. Programming expertise and a long lead time will be required for this approach. Second, the researchers could use the programming capabilities of several widely available data base man-
agement programs6 Though some programming background is necessary, the data base management functions are simple and easily accessed. Researchers at Syracuse University, for example, are developing a CATI system using dBASE11 on an IBM Personal Computer. The third option is to purchase one of the few CATI systems on the market and to adapt the software to meet your needs.’ In addition to the features discussed in the first section of this paper, advanced software should have “schedule editor and interview program generator” modules (Shure and Meeker, 1978). The interview program generator allows the researcher to input the survey text with any text editor. Execution of the interview program generator converts the text into program code. Another module, called a schedule editor, enables the researcher to modify previously generated survey questions or response codes; a common practice after the pre-test has been compieted. Also, the software should have the capability to export the data in a format suitable for mainframe statistical analysis.
CONCLUSIONS Undoubtedly, most of the issues discussed above will be technically obsolete in several years. Advances in computer technology have already had an astounding impact on survey research and this trend will continue for the foreseeable future. During the next decade, videotext systems will allow researchers to utilize visual images while conducting interviews. Artificial intelligence software will assist researchers in a variety of areas including data reduction, response coding,
and sample selection. The basic conclusion is that CAT1 systems can aid in the administration of telephone surveys and they can reduce sources of survey error, but these systems are neither inexpensive nor easily implemented. Perhaps the greatest potential for CAT1 is in the research field, where their structured use can lead us to a better understanding of the survey research process in general.
REFERENCES CUMMINGS, K. M. (1979). Random Digit Dialing: A Sampling Technique for Telephone Surveys. Pubbc Opinion Quarterly, 43, 233-245. FINK, J. C. (1983). CATI’s First Decade: The Chilton Experience. Sociological Methods and Research, 12, 153-168. FREEMAN, H. E. (1983). Research Opportunities Related to CATI. SociologicaI Methods and Research, 12, 143-152. FREY, 3. (1983). Survey Research by Telephone. Beverly Hills: Sage Publications. GROVES, R. M. (1983). Implications of CATI: Costs, Errors, and Organizations of Telephone Survey Research. Sociological Methods and Research, 12, 199-216. GROVES, R. M., % MATHIOWETZ, N. A. (1984). Computer Assisted Telephone Interviewing: Effects on Interviewers and Respondents. Public Opinion Quarterly, 48, 356-369. HANSON, R. H., & MARKS, E. S. (1958). Influence of the Interviewer on the Accuracy of Survey Results. Journal of the American Statistical Association, 3, 635-655.
MCFARLAND, S. G. (1981). Effect of Question Order on Survey Responses. Public Opinion Quarterly, 45, 208-215. NICHOLLS II, W. L. (1983.). CAT1 Research and Development at the Census Bureau. Socio/ogica/ Methods and Research, 12, 191-197. PALIT, C., & SHARP, H. (1983). Microcomputer-Assisted Telephone Inteviewing. Sociological Methods and Research, 12, 169-190. PHILIPP, S. F., & CICCIARELLA, C. F. (1983). An Apple II Package for Computer-Assisted Telephone Interviewing. Behavior Research Methods and Instrumentation, IS, 456-458. SHANKS, J. M. (1983). The Current Status of Computer-Assisted Telephone Interviewing: Recent Progress and Future Prospects. Sociological Methods and Research, 12, 119-142. SHANKS, J. M., & NICHOLLS II, W. L. (1981). The California Disability Survey: Design and Execution of a Computer-Assisted Telephone Study. Sociologicaf Methods and Research, 10, t23-140. SHURE, G. H., &MEEKER,
R. J. (1978). A Minicomputer System
’
Telephone
surveying
for Multiperson Computer-Assisted Telephone Interviewing. Behavior Research Methods and Instrumentation, 10, 196-202.
with computers
111
SUDMAN, S. (1983). Survey Research and Technological Change. Sociological Methods and Research, I2, 217-230.
NOTES 1. University of California at Los Angeles (UCLA), University of California at Berkeley, University of Michigan and University of Wisconsin have led in the development of university-based I CATI systems. The U.S. Bureau of Census and the U.S. Depattment of Agriculture have been the innovators at the federal level. 2. The two exceptions to this statement are the articles by Philipp and Cicciarella (1983) and Palit and Sharp (1983). 3. See Encyclopedia of Information Systems and Services (Detroit, Michigan: Gale Research Company, 1982) or A Directory of Computerized Data Files (Washington, D.C.: National Technical Information Service, 1984) for a comprehensive listing of computerized data bases. 4. This is the case, for example, when computers reduce costs by generating samples through random digit dialing and also pro-
vide a more statistically sound survey population. 5. This capability may add a new dimension to interviewer recruitment in that interviewers will have to possess some keyboard typing skills (Sudman, 1983). 6. Currently, microcomputer data base software packages with pro gramming capabilities include dBASE& Condor 3, KnowledgeMan, R-BASE 5000, PC-Focus, and others. 7. There are several commercial microcomputer CAT1 systems. One system, written for the Apple IIe, costs $35.00 and has been described by Philipp and Cicciarella (1983). Arlington Software and Systems Corporation (Arlington, MA 02174) distributes an $855.00 program for the IBM PC called PCAT. Sawtooth Software, Inc. (Ketchum, Idaho 83340) markets Ci2 System, which is a sophisticated computer interviewing program for the IBM PC.
0149-7189/86.$3.00 + .OO Copyright o 1986 Pergamon Journals Ltd
TELEPHONE SURVEYING WITH COMPUTERS: ADMINISTRATIVE, METHODOLOGICAL AND RESEARCH ISSUES
RALPH Department of Public Administration,
F. SHANGRAW. JR.
Maxwell School of Citizenship and Public Affairs, Syracuse University
ABSTRACT Computer-Assisted Telephone Interviewing (CATI) is a tool designed to assist interviewers in almost every aspect of a telephone survey. This article reviews the current status of CATI with special attention to microcomputer-based systems. Most of the literature in this area has focused on the implementation and design of CA TI systems. This paper takes a broader perspective by examining the adminktrative, methodological and research issues surrounding the use of CATI. The primary conclusion is that CATI systems can aid in the administration of telephone surveys in addition to reducing sources of survey error. However, these systems are neither inavpensive nor easily implemented. Perhaps the greatest potential for CATI is in the area of research, where their structured use can lead us to a better understanding of the survey research process in general.
INTRODUCTION With the advent of the computer, automated operations of previously manual tasks became possible in a variety of areas. In the field of survey research, the statistical analysis of results relieved researchers of the arduous task of hand calculating statistics. As computer systems matured, other survey research applications increased. One area that has received a great deal of attention in recent years is computer-aided telephone interviewing (CATI). CAT1 is a technique that aIlows almost all aspects of a telephone survey to be conducted with the assistance of a computer. The first systems appeared in the commercial sector around I972, with initial prototype systems developed by Chilton and Amrigon. Government and academic systems grew both out of the success of the commercial systems and out of the need for more efficient and cost-effective telephone surveying techniques,’ The initial justification for CAT1 was reduced costs for survey administration. Recent reports (Shanks, 1983; Groves, 1983) have deemphasized cost savings, especially for first-time users of CATI. Current re-
searchers argue that the real benefits of CAT1 are methodological and sample quality improvements, especially interviewer error reduction. Freeman (1983) claims that “CATI, over the long run, will result in a qualitatively different set of norms and expectations regarding the survey: optimistically, we should be able, through CATI, to improve markedly the art of data collection.” (Freeman, 1983:144) The purpose of this paper is to review of the current status of CATI, with special attention given to microcomputer-based telephone interviewing systems. Microcomputers are playing a greater role in the day-today activities of many researchers yet the academic literature has virtually ignored any discussions of microcomputer CATI systems.’ The first section of the paper will explore the administrative, methodological and research issues surrounding the utilization of CATI systems. In the second part of the paper, the design and implementation of a microcomputer-based CATI system is discussed.
GENERAL IMPLICATIONS OF CAT1 Much of the work on CAT1 systems has concentrated on the development, implementation, and refinement of the computer software that drives these systems. Now that some of the packages have been in use for several years, researchers are moving beyond the nuts
and bolts and into research on the influence of CAT1 on survey methods. The findings from these recent studies can be grouped into three categories: administrative, methodological, and research issues. 107
108
RALPH F. SHANGRAW
Administrative Issues Improved respondent selection is often cited as a primary admi~strative advantage of CATI (Paiit and Sharp, 1983; Shanks and Nicholls, 1981). CAT1 programs improve respondent selection in several ways. First, “random digit dialing” (Cummings, 1979) can be employed to generate random samples. Local telephone exchanges are entered into the CAT1 program and the software generates a random sample of telephone numbers. Second, most programs can also monitor sampling frequencies for individual stratum of a stratified sample. Periodically, the researcher can output a frequency table and immediately identify small cell problems. Third, the growth of computer data bases on survey populations will greatly streamline the conduct of telephone interviews.3 Information from these systems can provide a convenient front end for CAT1 systems. In addition to aiding in sampling procedures, CAT1 can monitor the administration of the survey. When the interviewee is called, the CAT1 program generates a response code indicating if the number is busy, if there is no answer, if a follow-up call is required, or if the interview is refused. The system will prompt the interviewer with the same number for recall if there is a busy number or no answer. These automated scheduling procedures greatly reduce the amount of time that the project coordinator spends on administrative issues. The general consensus on interviewer training and recruitment is that CATI is more costly than manual methods, especially for initial training (Frey, 1983; Shanks and Nicholls, 1981). At least 80 to 90 hours of training over a three week period were required for the California Disability Study before the interviewers were comfortable with the system (Shanks and Nicholls, 1981). Three developments are likely to offset these costs in the future. First, as the population in genera1 becomes more computer literate, less training on the operational aspects of the system will be necessary. Second, the computer-aided instruction (CAI) packages can provide programmed learning modules on CAT1 systems, though currently few survey groups have developed CA1 for CAT1 systems (Shanks, 1983). Third, CAT1 systems are being upgraded so that the interviewer needs to know very little about the computer. As these systems evolve, training costs will reduce. Contrary to early speculation, the costs of CAT1 systems have not been dramatically lower than manual procedures. Researchers have consistently underestimated the time and costs required for software design and debugging. Software costs will remain high as long as these systems continue to be developed for specific survey applications. The current movement is towards “generic” CAT1 systems that can be modified easily for any survey schedule in a variety of survey environ-
ments (marketing research, academic surveys, evaluations studies, etc.). The availability of “off the shelf” software will improve the cost ratio between manual and automated systems. While costs may be higher, interviewer efficiency is expected to improve with CAT1 systems. The University of Wisconsin’s Survey Research Laboratory estimated that “an experienced CAT1 interviewer’s productivity (as measured by the number of sample points contacted per production hour) may be as much as 20% higher than that of the same interviewer using traditional survey methods.” (Palit and Sharp, 1983) In the short term, manual procedures will have to accompany the computerized systems. Paper copies of interviewing schedules are necessary in case of hardware or software failures. One study (Groves and Mathiowetz, 1984) found that interviewers preferred the paper version of the survey schedule because it was easier to correct errors. Supervision will be similar in a manual or automated setting. In the case of CATI, however, the supervisor will be able to monitor both the phone conversation and the CRT input. Methodological Issues Administrative survey research issues cannot be distinguished easily from methodological concerns, especially when improvements in administration coincidentally reduce survey error.4 Reduced interviewer error, for example, is one methodological benefit of CAT1 systems. Error reduction can be grouped into three categories: skip logic or branching; response coding; and aided recall. CAT1 relieves the interviewer of following complicated procedures when the response to one question dictates the asking of another question. Branching procedures, which are generally desirable from a data collection perspective, are often sources of interviewer confusion and error (Shure and Meeker, 1978). These capabilities also allow researchers to develop branching operatons that were never before possible with manual methods. Another advantage of CAT1 is the ability of the system to recognize instantly illegal or inconsistent responses or skipped questions. When an improper i~sponse is recorded at the terminal, the systems responds with an error message prompting the interviewer for a legal response. As with manual systems, it is possible for the interviewer to record an incorrect response in a legal format. Post interview error checking procedures, with the corollary expense of obtaining correct or missing data points, can be eliminated with CAT1 (Shure and Meeker, 1978). It is also possible for the computer to detect “yea-saying” by the interviewee or any other response pattern. Once detected, the CAT1 program can reverse the scale for future questions or ask follow-up questions on previous responses. While aided-recall and question clarification pro-
Telephone surveying with computers cedures have been an integral component of manual systems, these processes are implemented easily on CAT1 systems. Consistent and properly worded additional information can reduce interviewer error frequently described in the literature (for example, see Hanson and Marks, 1958). On-line reference materials and branching clarification algorithms can contribute to the system’s aided-recall or “intelligent probing” capabilities (Fink, 1983). In addition to reductions in interviewer error, CAT1 systems “permit new solutions to old problems” when open-ended questions are included in the survey (Groves, 1983). Of primary benefit is the ability of the CAT1 system to record responses to open-ended questions in a machine-readable format.5 Some CAT1 systems have been designed to automatically update short response categories to open-ended questions when a new response is encountered. Other systems generate a list of all open-ended responses to allow researchers to add new categories on a periodic basis. With the evolution of text analysis software, machine-readable openended responses could be evaluated for occurrences of phrases or keywords and, in the future, for ideas (Groves, 1983). Three other methodological issues can be addressed by CAT1 systems. McFarland (1981) has found that the type of question and the level of specificity will determine if position bias will occur. Position bias, in these instances, can be countered by the random assignment of question order by the computer. Some response bias can be reduced through the personalization of the survey. Pronouns can be replaced throughout the survey schedule to reflect the gender of the in-
109
terviewee or the name of the inteviewee can be inserted to personalize the schedule. Finally, and most obvious, transposition errors are eliminated with CAT1 systems since there is no transfer of the response data from a hard-copy document to a machine-readable format.
Research Issues Freeman (1983) makes a valid point when he suggests that researchers should “use CAT1 as a means of testing our premises about survey procedures rather than simply employing the technology indiscriminately and without evaluation.” (Freeman, 1983:152) As he notes, “CAT1 permits increased control over the interpersonal relationship between interviewer and respondent and the presentation of stimuli.” (Freeman, 1983:148) He suggests that a research agenda be established to test much of the “folklore” that is an integral part of survey research. Some research organizations are initiating empirical studies on CAT1 systems. The U.S. Census Bureau, for example, has completed two field test of computerassisted telephone interviewing systems (Nicholls, 1983). The field tests had three purposes: (a) to compare CAT1 with manual methods especially in terms of response rates, quality of data obtained, and speed of data collection; (b) to evaluate the selection and training of CAT1 interviewers and “to identify factors in their performance;” and (c) to improve CAT1 systems for future field tests and general use (Nicholls, 1983). Results from this research are forthcoming and should provide some interesting insight into the use of CAT1 systems.
CATI IN A MICROCOMPUTER ENVIRONMENT time is virtually instantaneous. On a large mainframe While much of the recent literature has focused on mainframe or minicomputer CAT1 applications, few or minicomputer time-sharing system, the slow reresearchers have addressed the use of microcomputers sponse rate at peak use can be annoying to both the interviewer and interviewee. Screen display, especially in in the survey research laboratory. Palit and Sharp (1983) discussed the implementation of a CAT1 system terms of color and graphics, is generally easier to on a time-shared minicomputer with microcomputers manipulate on microcomputers. Colors can be used to at individual interviewing stations. This configuration emphasize words or questions; graphics can be used allows the interviewer to execute the survey at their into display frequency counts and preliminary statistical dividual stations with the additional benefits of imresults. mediate uploading of the survey data to the minicomThe hardware disadvantages, especially for largeputer for further manipulation. For small to medium scale surveys, greatly outweigh the benefits. Permasize surveys, the microcomputer is an attractive alternent and internal storage are important consideranative to manual systems. The potential use of micros tions. Large data sets will mandate the use or purchase in survey research is increasing as researchers begin to of hard disk storage systems for most users. While it is use microcomputers in other aspects of their work. likely that the interview schedule can be stored on a The following discussion will center on the sole use of diskette, most surveys will require a hard disk for the microcomputers for computer-aided telephone surresponse data. Large internal storage (RAM) will be necessary if a large data base will be sorted or if a veying. statistical package is used to analyze the data. There Hardware Considerations are further problems when several microcomputers are The microcomputer has both advantages and shortused to conduct the same survey. Until Local Area comings. One advantage is that the computer response Network (LAN) software is perfected, data from the
110
RALPH F. SHANGRAW
individual machines will have to be merged before it can be processed. It is imperative that checks are included in the software to eliminate interview redundancy. As microcomputers grow in size, multi-tasking operating systems and multi-processors will enable data input from several terminals. Software Issues At present, most researchers who are interested in using CAT1 on microcomputers will have to design and develop their own software. Three options are available for software development. First, the researcher or research group could program the system using BASIC or another higher level programming language. Since execution speed may be a consideration, the system could be programmed in C or Assembler. Fortunately, the experience with mainframe CATI systems will provide guidance on system design and features. Programming expertise and a long lead time will be required for this approach. Second, the researchers could use the programming capabilities of several widely available data base man-
agement programs6 Though some programming background is necessary, the data base management functions are simple and easily accessed. Researchers at Syracuse University, for example, are developing a CATI system using dBASE11 on an IBM Personal Computer. The third option is to purchase one of the few CATI systems on the market and to adapt the software to meet your needs.’ In addition to the features discussed in the first section of this paper, advanced software should have “schedule editor and interview program generator” modules (Shure and Meeker, 1978). The interview program generator allows the researcher to input the survey text with any text editor. Execution of the interview program generator converts the text into program code. Another module, called a schedule editor, enables the researcher to modify previously generated survey questions or response codes; a common practice after the pre-test has been compieted. Also, the software should have the capability to export the data in a format suitable for mainframe statistical analysis.
CONCLUSIONS Undoubtedly, most of the issues discussed above will be technically obsolete in several years. Advances in computer technology have already had an astounding impact on survey research and this trend will continue for the foreseeable future. During the next decade, videotext systems will allow researchers to utilize visual images while conducting interviews. Artificial intelligence software will assist researchers in a variety of areas including data reduction, response coding,
and sample selection. The basic conclusion is that CAT1 systems can aid in the administration of telephone surveys and they can reduce sources of survey error, but these systems are neither inexpensive nor easily implemented. Perhaps the greatest potential for CAT1 is in the research field, where their structured use can lead us to a better understanding of the survey research process in general.
REFERENCES CUMMINGS, K. M. (1979). Random Digit Dialing: A Sampling Technique for Telephone Surveys. Pubbc Opinion Quarterly, 43, 233-245. FINK, J. C. (1983). CATI’s First Decade: The Chilton Experience. Sociological Methods and Research, 12, 153-168. FREEMAN, H. E. (1983). Research Opportunities Related to CATI. SociologicaI Methods and Research, 12, 143-152. FREY, 3. (1983). Survey Research by Telephone. Beverly Hills: Sage Publications. GROVES, R. M. (1983). Implications of CATI: Costs, Errors, and Organizations of Telephone Survey Research. Sociological Methods and Research, 12, 199-216. GROVES, R. M., % MATHIOWETZ, N. A. (1984). Computer Assisted Telephone Interviewing: Effects on Interviewers and Respondents. Public Opinion Quarterly, 48, 356-369. HANSON, R. H., & MARKS, E. S. (1958). Influence of the Interviewer on the Accuracy of Survey Results. Journal of the American Statistical Association, 3, 635-655.
MCFARLAND, S. G. (1981). Effect of Question Order on Survey Responses. Public Opinion Quarterly, 45, 208-215. NICHOLLS II, W. L. (1983.). CAT1 Research and Development at the Census Bureau. Socio/ogica/ Methods and Research, 12, 191-197. PALIT, C., & SHARP, H. (1983). Microcomputer-Assisted Telephone Inteviewing. Sociological Methods and Research, 12, 169-190. PHILIPP, S. F., & CICCIARELLA, C. F. (1983). An Apple II Package for Computer-Assisted Telephone Interviewing. Behavior Research Methods and Instrumentation, IS, 456-458. SHANKS, J. M. (1983). The Current Status of Computer-Assisted Telephone Interviewing: Recent Progress and Future Prospects. Sociological Methods and Research, 12, 119-142. SHANKS, J. M., & NICHOLLS II, W. L. (1981). The California Disability Survey: Design and Execution of a Computer-Assisted Telephone Study. Sociologicaf Methods and Research, 10, t23-140. SHURE, G. H., &MEEKER,
R. J. (1978). A Minicomputer System
’
Telephone
surveying
for Multiperson Computer-Assisted Telephone Interviewing. Behavior Research Methods and Instrumentation, 10, 196-202.
with computers
111
SUDMAN, S. (1983). Survey Research and Technological Change. Sociological Methods and Research, I2, 217-230.
NOTES 1. University of California at Los Angeles (UCLA), University of California at Berkeley, University of Michigan and University of Wisconsin have led in the development of university-based I CATI systems. The U.S. Bureau of Census and the U.S. Depattment of Agriculture have been the innovators at the federal level. 2. The two exceptions to this statement are the articles by Philipp and Cicciarella (1983) and Palit and Sharp (1983). 3. See Encyclopedia of Information Systems and Services (Detroit, Michigan: Gale Research Company, 1982) or A Directory of Computerized Data Files (Washington, D.C.: National Technical Information Service, 1984) for a comprehensive listing of computerized data bases. 4. This is the case, for example, when computers reduce costs by generating samples through random digit dialing and also pro-
vide a more statistically sound survey population. 5. This capability may add a new dimension to interviewer recruitment in that interviewers will have to possess some keyboard typing skills (Sudman, 1983). 6. Currently, microcomputer data base software packages with pro gramming capabilities include dBASE& Condor 3, KnowledgeMan, R-BASE 5000, PC-Focus, and others. 7. There are several commercial microcomputer CAT1 systems. One system, written for the Apple IIe, costs $35.00 and has been described by Philipp and Cicciarella (1983). Arlington Software and Systems Corporation (Arlington, MA 02174) distributes an $855.00 program for the IBM PC called PCAT. Sawtooth Software, Inc. (Ketchum, Idaho 83340) markets Ci2 System, which is a sophisticated computer interviewing program for the IBM PC.