- Email: [email protected]
A client-computer interface for questionnaire data
639
A Client-Computer Murray
E. Maitland,
Interface for Questionnaire Data
MSc, Allun R. Mandel,
PhD
ABSTRACT. Maitland ME, Mandel AR. A client-computer interface for questionnaire data. Arch Phys Med Rehabil 1994;75:639-42. a Our purpose was to assess efficiency and patient acceptance of a simple, inexpensive method of direct clientcomputer interface. A computerized version of the Sickness Impact Protile (SIP) was developed with the standard keyboard replaced by a numeric keypad. Forty-eight volunteer clients of an occupational rehabilitation center completed both the paper format and the computer format in randomized order, then were asked to compare the two methods in four areas. The majority of subjects rated the computer as easier to use and as the preferred method. Most subjects rated the computer and paper formats as equivalent in comfort and understanding. A summary score established that significantly more individuals preferred the computer format over the paper format @ = .02). Efficiency was determined by the mean time to score the test and produce a report (computer 39s, paper 309s). The correlation coefficients for the physical, psychosocial, and overall scores were very acceptable being greater than 0.90. These data indicate that a simple client-computer interface for the SIP is preferred by clients and is more efficient compared to the paper and pencil format. 0 1994 by the American Congress qf Rehabilitation Medicine and the American Academy of Physical Medicine and Rehabilitkion
Client questionnaires can be used effectively in rehabilitation settings to plan treatment, to determine change in the client’s condition, or to provide quality assurance data. Questionnaires are used to measure quality of life, appraise client satisfaction, quantify symptoms (such as pain scales), and assess function. Questionnaires can serve as integral elements of a comprehensive clinical evaluation, but obstacles such as the time required for scoring, data reduction, and data analysis inhibit more widespread use of these measures in rehabilitation settings. The motivation for more extensive use of clinical measures comes from several fronts. Insurance programs and consumers who pay for health care services are demanding more accountability for costs. Also, health care providers are becoming more sensitive to types of information required by institutional administrators to determine resource allocation. From a clinical perspective, health care providers seek answers to clinical questions, such as the rate of patient progress. Finally, researchers endeavor to measure patient attributes by methods that are time and cost effective. Computerization of questionnaires may have advantages that the standard paper and pencil format do not. Obvious benefits are associated with routine scoring, reporting, and data analysis. With the computer no personnel are required for these tasks. Furthermore, reliable reporting is assured because of automated calculation and transcription.’ The advantages of computerization become obvious when there is greater mathematical complexity, longer questionnaires, and From the University Occupational partment
ofCalgarySport
Rehabilitation
of Psychology
Submttted
Centre
(Dr.
for publication
(Dr.
Medicine
Mandel).
Calgary.
June
1993.
IO.
Centre
Mandel),
(Mr.
Maitland),
and the University Alberta,
Accepted
the Western
of Calgary
De-
Canada. in rewed
form
October
26.
1993. This reuxrch District
was cupported
of The Canadian
by the B.E.
Physiotherapy
Schnurr
Memo&
Fund and The Calgary
Association.
No commcrical party having 3 direct iinancial interest zn the results of the research supporting this article has or wll confer a benefit upon the authors or upon any organization with which the authors are associated. Reprmt requests to Murray Maitland. MSc, The University of Calgary, Sport Medicine Centre, 11500 University Drive NW. Calgary, Alberta. Canada T2N IN4. CJ 1994 by the Amencan Congress of Rehabilitation Medicme and the Amerlcnn Academy of Physical Medicine and Rehabilitation 0003.99Y3/94/750&0139$3.00/0
more detailed reports. For example. The Sickness Impact Profile (SIP) used in this study requires the calculation of 15 weighted averages of 135 questions.” Moreover, incorporating current research models into a rehabilitation setting may involve complex mathematical functions. For instance, Carey et al’ suggested that to predict discharge scores from functional test scores at admission would require a thirddegree polynomial. In part, the purpose of this study was to assess the efficiency of a direct client-computer interface. Regardless of potential benefits, few clinical environment\; take advantage of direct client-computer interface for questionnaire data, automated scoring, standardized reporting. and generation of statistics. One problem is the complexity of setting up a system. Clinicians have little experience in this area and can be dismayed by the prospect of such a project. Furthermore, the cost of planning, programming. developing, and supporting the system can be prohibitive. Therefore, we describe a simple client-computer interfacr that can be adapted to many clinical environments. Computerized questionnaire responses might be different somehow because of the format. Huba’ compared computer and paper and pencil administrations of the Western Personnel Test, and concluded that “the computerized version is generally comparable to the paper and pencil version and can be used interchangeably.” A computerized version of‘ the Minnesota Multiphasic Personality Inventory was found to be equivalent to the paper and pencil version.’ Our study provides an estimate of the reliability of the computerized SIP compared with the paper and pencil version. Incorporating computerized questionnaires into health care settings has other potential disadvantages. Clinicians may be concerned about client reaction because negative responses may interfere with the client-care giver relationship or with patient satisfaction. Several studies have assessed patient acceptance of computerized questionnaires. Blood donors. for example, were found to be accepting of computer-based questionnaires.” In a study of 200 university men who completed a sexual abuse questionnaire, 9 1%: said they preferred the computerized method but 3% found the
Arch
Phys
Med
Rehabil
Vol
75, June
1994
CLIENT-COMPUTER
640
computer threatening to use.7 As early as 1968, a descriptive paper reported positive responses of patients to computerized history taking.* An assessment of a psychiatric patient interview system, however, indicated that 10 of 27 subjects were not comfortable interacting with a computer.’ Burke and Normand,’ in a review of computerized psychological testing, concluded that computer-based testing has a high degree of acceptability to clients. Previous studies reported responses to open-ended questions and did not attempt a systematic approach to assessing the client’s perspective. We are unaware of any study that evaluates a client-computer interface in an occupational rehabilitation population. Consequently, our purpose was to assess patient acceptance of an inexpensive method for direct client-computer interface to determine the potential consequences to patient satisfaction. We hypothesized that there would not be a difference in preference between the paper method and the computer method. METHODS
INTERFACE, Maitland
E
16-2021-25 26-3031-35 36-4041-45 46-5051-55 56-6061-65
Fig l-Distribution of client (A) education level, (B) self-rated computer experience, and (C) age.
The Computerized Questionnaire The SIP’ was adapted to a computer format using dBase IV.” The SIP was chosen because it is commonly used as an outcome measure in back injury rehabilitation settings and is used in the setting of the study. The data entry program was intended to be as straightforward as possible for the client, regardless of education level or experience. Each question was presented individually on the computer screen. Responses were entered using a numeric keypadb instead of the standard keyboard. The SIP normally requires the subject to either check a question or leave the question blank, but in the computer version the subject must respond either “yes” or “no.” On the numeric keypad the number “1” was labelled “yes” and number “3” was labelled “no.” The “enter” key was labelled “space” and had been reprogrammed to be a “space” symbol. If the subject entered a response other than “yes” or “no,” an error message appeared and the subject was required to press the space (enter) key to continue.
Sample Subjects (n = 48) were all volunteers who were enlisted from clients of an occupational rehabilitation program and who were undergoing intake evaluations. Subjects completed both forms of the questionnaire and were randomized according to whether they completed the paper format or the computer format first.
Postquestionnaire
Evaluation
After the subjects finished the two formats of the SIP questionnaire they also completed a questionnaire that elicited information about age, educational level, and computer experience. Education level was categorized by completion of grade school levels, participation in university or college courses, completion of college diploma or university degree, and taking postgraduate courses. In addition, subjects were asked to select whether formats were equivalent, or whether the computer format or the paper format was preferable in Arch Phys bled Rehabil Vol75,
June 1994
four areas: comfort, understanding, ease of use, and preferred method overall.
Data Analysis A summary score was calcuIated for each subject from ratings in the four areas: for each question a selection of the paper format was scored as -1, equivalence in preference was scored as 0, computer was scored as 1, and then the responses were summed. The statistical significance of the summary score in the sample population was determined by a two-sided sign test, calculated from the binomial distribution. Although this study was not intended to provide an accurate assessment of reliability between the two formats, we wanted to determine if categorical scores and overall scores in both formats were comparable. As an estimate, the least squares linear regression correlation coefficient was calculated for each of the 12 section scores, the physical and psychosocial scores, and overall scores. RESULTS Forty-eight subjects, 36 men and 12 women, volunteered for the study between September 26 and December 16,1992. Most subjects had graduated from grade 12 but a large percentage (38%) had not reached that educational level. Seventy-three percent of the subjects rated themselves as having very little or no computer experience. The median age category of subjects was 31 to 35 years at the time of the study (fig 1). Twenty-three subjects completed the computer form first and 25 completed the paper form first. The frequency of subject choices (paper format, computer format, or either format being equivalent) for each of the four areas assessed is shown in figure 2. The subjects did not show a preference for either the computer format or the paper format in the areas of comfort or understanding; they most often rated the computer higher in ease of use and overall preference. Calculation of summary scores for each subject showed
CLIENT-COMPUTER
Paper Either Computer Easiest to Understand 301
30-r
Maitland
Sleep and rest Emotional behavior Body care and management Home management Mobility Social integration Ambulation Alertness behavior Communication Work Recreation and pastimes Eating Physical Psychosocial Overall * Not a valid correlation.
Paper Either Computer Preferred Methoh
Paper Either Computer Easiest to Use _
Fig 2-Frequency of client choices after they had completed both formats of the SIP questionnaire.
significantly more individuals on the positive side of this distribution @ = .02). This result indicates that the study sample chose the computer format over the paper format more often in the four areas assessed (fig 3). Retrospectively, summary scores were divided into two age ranges (up to age 35, over age 35) to determine if older individuals respond more negatively to the computer format. In the younger age group 7 of 25 subjects (28%) responded negatively, whereas in the older age group 6 of 21 subjects (29%) responded negatively. Correlation coefficients varied from 0.55 to 0.93, which is comparable to studies of the reliability of the SIP questionnaire (table).’ The correlation coefficients for the physical, psychosocial. and overall scores are very acceptable being greater than 0.90. The correlation coefficient of the “Work” section is not presented in table 1 because of problems with its validity. In answering both the paper and the computer questionnaires many subjects misunderstood directions and incorrectly responded that they were working (question Wl) although, in fact, none of the subjects was working. In marking the paper questionnaire, support staff corrected these
-4 -3 Ailresponser favour paper
Fig 3-Frequency
-2
-1
0
641
Linear Regression Correlation Coefficients For Computer and Paper Formats of SIP Category Scores and Overall Score
25
Paper Either Computer Most Comfortable
INTERFACE,
1
Summary Score
2
3
4
All ~es~onscs favourcompulcr
distribution of summary scores.
.7x .94 .84 .67 .75 .85 .93 .90 .93 * .55 .&I .90 .93 .92
_
See text for details.
responses and reported the corrected scores but they were unable to correct the computer format. DISCUSSION Our results support the development of computerized questionnaires using a direct client-computer interface as a result of two findings. First, clients selected the computer format significantly more often than the paper format in ease of use, understanding, comfort, and preference overall. Second, support staff time required to produce a report was reduced from 5:09 minutes to 39 seconds. Furthermore, efficient collection, analysis, and reporting of questionnaire data should enable improved documentation of programs for justification, administrative, or research uses. Numerous computer interfaces are being used currently: touch screens, voice, bar code readers, mouse, and others. The numeric keypad was chosen because it is inexpensive, easy to program, resilient to heavy use and is believed to be the most familiar to the client. Other investigators have attempted to use touch screens or mouse input.‘” They found many patients were apprehensive about touching the screen or using the mouse. For some clients the computerized method is clearly the method of choice. Previous studies indicate that information may be better acquired by direct client-computer interface if the client is uncomfortable about disclosing the information to another person. For example, significantly more rcports of sexual abuse were related with the use of a computerized questionnaire compared with a paper format7 Similarly, in a study assessing human immunodeficiency virus (HIV) risk characteristics, more HIV-related factors were elucidated by computer interview.’ Level of physical ability may also determine if a particular client-computer interface is expedient or whether it is impractical. In this study all clients were physically able to complete the questionnaire. Difficulties with grasp, coordination, or physical tolerance may be overcome with alternate interfaces.” Otherwise, physically disabled clients may require assistance entering data. Some clients cannot cope with an automated system. For example, Coombs and coworkers” suggested that support Arch Phys Med Rehabil Vol75,
June 1994
CLIENT-COMPUTER
642
staff assist any client over the age of 65 with an automated medical history system. In the current study we did not find preference for either format to be associated with age. Arbitrary criteria such as age guidelines clearly depend on the complexity of the task and other factors. Perhaps more important criteria should be based on literacy, language skills and anxiety, both for the paper and the computer formats. These areas have not been investigated to our knowledge. Burke and Normand’ summarize aspects of the test-taker/ computer interaction that promote effective use by large patient populations. These include clarity of instructions, allowing for practice, and error checking. Though we have focused on the client-computer interface, similar methods may be used by health care professionals and support staff in situations that demand various forms of data entry. For example, clinics may be required to collect patient status data in a computerized format. Similar interfaces can be used at this level to minimize keystrokes, produce the required reports. - i minimize the stress impact on staff. CONCLUSION Subjects on average did not show a preference for the paper format or computer format in the areas of comfort or understanding. Subjects overwhelmingly choose the computer format as easier to use and as the preferred method. These findings support the integration of a direct clientcomputer interface for questionnaire data in an occupational rehabilitation setting where most clients are laborers with grade 12 education or less. Correlation coefficients for paper and computer formats were acceptable for physical, psychosocial, and overall scores. Modification to the “work” section of the SIP questionnaire may improve its validity with regards to a nonworking population. Computerized reporting of scores reduced support staff time to 13% of the time required to produce a report from
Arch Phys Med Rehabil Vol75,
June 1994
INTERFACE,
Maitland
the paper format. This finding indicates a potentially important area for increased staff productivity depending on the environment. Acknowledgment: The authors thank their assistance in collecting the data.
K. Nielsen
and T. Lipovski
for
References I. Burke MJ, Normand J. Computerized psychological testing: overview and critique. Prof Psycho] 1987; I8:42-5 1. 2. Bergner M, Bobbitt RA, Carter WB, Gilson BS. The sickness impact profile: development and final revision of a health status measure. Med Care 1981; 19:787-805. 3 Carey RG. Seibert JH, Posavac ET. Who makes the most progress in inpatient rehabilitation? An analysis of functional gain. Arch Phys Med Rehabil 1988;69:337-43. Huba GJ. Comparability of traditional and computer Western Personnel Test (WPT) versions. Educ Psych Measure 1988;48:957-9. Lushene RE, O’Neil HH. Dunn T. Equivalent validity of a completely computerized MMPI. J Pers Assess 1974;38:353-61. Locke SE, Kowaloff HB, Hoff RG, Safran C. Popovsky MA, Cotton DJ, et al. Computer-based interview for screening blood donors for risk of HIV transmission, JAMA 1992;268: 1301-5. 7 Bagley C. Genius M. Psychology of computer use: XX. Sexual abuse recalled: evaluation of a computerized questionnaire in a population of young adult males. Percept Mot Skills 1991;72:287-8. 8 Slack WV, Van Cura LJ. Computer-based patient interviewing II. Postgrad Med I968;43: I 15-20. 9 Greist JH. Klein MH, Van Cura LJ. A computer interview for psychiatric patient target symptoms. Arch Gen Psych 1973;29:247-53. IO Roizen MF, Coalson D, Hayward RS, Schmittner J, Thisted RA, Apfelbaum JL, et al. Can patients use an automated questionnaire to define their current health status? Med Care 1992;30:MS74-MS84. Il. Sampson JP. Computer-assisted testing and assessment: current status and implications for the future. Measure Eva1 Guidance 1983;5: 293-9. 12. Coombs GJ, Murray WR, Krahn DW. Automated medical histories: factors determining patient performance. Comput Biomed Res 1970;3:178-81. Suppliers a. d Base IV. Borland, 1800 Green Hills Road, PO Box 660001, Scotts (alley. CA 95067-0001, b. lenovations numeric keypad. Genovation Incorporated, 17741 Mitchell, iorth Irvine, CA.
A Client-Computer Murray
E. Maitland,
Interface for Questionnaire Data
MSc, Allun R. Mandel,
PhD
ABSTRACT. Maitland ME, Mandel AR. A client-computer interface for questionnaire data. Arch Phys Med Rehabil 1994;75:639-42. a Our purpose was to assess efficiency and patient acceptance of a simple, inexpensive method of direct clientcomputer interface. A computerized version of the Sickness Impact Protile (SIP) was developed with the standard keyboard replaced by a numeric keypad. Forty-eight volunteer clients of an occupational rehabilitation center completed both the paper format and the computer format in randomized order, then were asked to compare the two methods in four areas. The majority of subjects rated the computer as easier to use and as the preferred method. Most subjects rated the computer and paper formats as equivalent in comfort and understanding. A summary score established that significantly more individuals preferred the computer format over the paper format @ = .02). Efficiency was determined by the mean time to score the test and produce a report (computer 39s, paper 309s). The correlation coefficients for the physical, psychosocial, and overall scores were very acceptable being greater than 0.90. These data indicate that a simple client-computer interface for the SIP is preferred by clients and is more efficient compared to the paper and pencil format. 0 1994 by the American Congress qf Rehabilitation Medicine and the American Academy of Physical Medicine and Rehabilitkion
Client questionnaires can be used effectively in rehabilitation settings to plan treatment, to determine change in the client’s condition, or to provide quality assurance data. Questionnaires are used to measure quality of life, appraise client satisfaction, quantify symptoms (such as pain scales), and assess function. Questionnaires can serve as integral elements of a comprehensive clinical evaluation, but obstacles such as the time required for scoring, data reduction, and data analysis inhibit more widespread use of these measures in rehabilitation settings. The motivation for more extensive use of clinical measures comes from several fronts. Insurance programs and consumers who pay for health care services are demanding more accountability for costs. Also, health care providers are becoming more sensitive to types of information required by institutional administrators to determine resource allocation. From a clinical perspective, health care providers seek answers to clinical questions, such as the rate of patient progress. Finally, researchers endeavor to measure patient attributes by methods that are time and cost effective. Computerization of questionnaires may have advantages that the standard paper and pencil format do not. Obvious benefits are associated with routine scoring, reporting, and data analysis. With the computer no personnel are required for these tasks. Furthermore, reliable reporting is assured because of automated calculation and transcription.’ The advantages of computerization become obvious when there is greater mathematical complexity, longer questionnaires, and From the University Occupational partment
ofCalgarySport
Rehabilitation
of Psychology
Submttted
Centre
(Dr.
for publication
(Dr.
Medicine
Mandel).
Calgary.
June
1993.
IO.
Centre
Mandel),
(Mr.
Maitland),
and the University Alberta,
Accepted
the Western
of Calgary
De-
Canada. in rewed
form
October
26.
1993. This reuxrch District
was cupported
of The Canadian
by the B.E.
Physiotherapy
Schnurr
Memo&
Fund and The Calgary
Association.
No commcrical party having 3 direct iinancial interest zn the results of the research supporting this article has or wll confer a benefit upon the authors or upon any organization with which the authors are associated. Reprmt requests to Murray Maitland. MSc, The University of Calgary, Sport Medicine Centre, 11500 University Drive NW. Calgary, Alberta. Canada T2N IN4. CJ 1994 by the Amencan Congress of Rehabilitation Medicme and the Amerlcnn Academy of Physical Medicine and Rehabilitation 0003.99Y3/94/750&0139$3.00/0
more detailed reports. For example. The Sickness Impact Profile (SIP) used in this study requires the calculation of 15 weighted averages of 135 questions.” Moreover, incorporating current research models into a rehabilitation setting may involve complex mathematical functions. For instance, Carey et al’ suggested that to predict discharge scores from functional test scores at admission would require a thirddegree polynomial. In part, the purpose of this study was to assess the efficiency of a direct client-computer interface. Regardless of potential benefits, few clinical environment\; take advantage of direct client-computer interface for questionnaire data, automated scoring, standardized reporting. and generation of statistics. One problem is the complexity of setting up a system. Clinicians have little experience in this area and can be dismayed by the prospect of such a project. Furthermore, the cost of planning, programming. developing, and supporting the system can be prohibitive. Therefore, we describe a simple client-computer interfacr that can be adapted to many clinical environments. Computerized questionnaire responses might be different somehow because of the format. Huba’ compared computer and paper and pencil administrations of the Western Personnel Test, and concluded that “the computerized version is generally comparable to the paper and pencil version and can be used interchangeably.” A computerized version of‘ the Minnesota Multiphasic Personality Inventory was found to be equivalent to the paper and pencil version.’ Our study provides an estimate of the reliability of the computerized SIP compared with the paper and pencil version. Incorporating computerized questionnaires into health care settings has other potential disadvantages. Clinicians may be concerned about client reaction because negative responses may interfere with the client-care giver relationship or with patient satisfaction. Several studies have assessed patient acceptance of computerized questionnaires. Blood donors. for example, were found to be accepting of computer-based questionnaires.” In a study of 200 university men who completed a sexual abuse questionnaire, 9 1%: said they preferred the computerized method but 3% found the
Arch
Phys
Med
Rehabil
Vol
75, June
1994
CLIENT-COMPUTER
640
computer threatening to use.7 As early as 1968, a descriptive paper reported positive responses of patients to computerized history taking.* An assessment of a psychiatric patient interview system, however, indicated that 10 of 27 subjects were not comfortable interacting with a computer.’ Burke and Normand,’ in a review of computerized psychological testing, concluded that computer-based testing has a high degree of acceptability to clients. Previous studies reported responses to open-ended questions and did not attempt a systematic approach to assessing the client’s perspective. We are unaware of any study that evaluates a client-computer interface in an occupational rehabilitation population. Consequently, our purpose was to assess patient acceptance of an inexpensive method for direct client-computer interface to determine the potential consequences to patient satisfaction. We hypothesized that there would not be a difference in preference between the paper method and the computer method. METHODS
INTERFACE, Maitland
E
16-2021-25 26-3031-35 36-4041-45 46-5051-55 56-6061-65
Fig l-Distribution of client (A) education level, (B) self-rated computer experience, and (C) age.
The Computerized Questionnaire The SIP’ was adapted to a computer format using dBase IV.” The SIP was chosen because it is commonly used as an outcome measure in back injury rehabilitation settings and is used in the setting of the study. The data entry program was intended to be as straightforward as possible for the client, regardless of education level or experience. Each question was presented individually on the computer screen. Responses were entered using a numeric keypadb instead of the standard keyboard. The SIP normally requires the subject to either check a question or leave the question blank, but in the computer version the subject must respond either “yes” or “no.” On the numeric keypad the number “1” was labelled “yes” and number “3” was labelled “no.” The “enter” key was labelled “space” and had been reprogrammed to be a “space” symbol. If the subject entered a response other than “yes” or “no,” an error message appeared and the subject was required to press the space (enter) key to continue.
Sample Subjects (n = 48) were all volunteers who were enlisted from clients of an occupational rehabilitation program and who were undergoing intake evaluations. Subjects completed both forms of the questionnaire and were randomized according to whether they completed the paper format or the computer format first.
Postquestionnaire
Evaluation
After the subjects finished the two formats of the SIP questionnaire they also completed a questionnaire that elicited information about age, educational level, and computer experience. Education level was categorized by completion of grade school levels, participation in university or college courses, completion of college diploma or university degree, and taking postgraduate courses. In addition, subjects were asked to select whether formats were equivalent, or whether the computer format or the paper format was preferable in Arch Phys bled Rehabil Vol75,
June 1994
four areas: comfort, understanding, ease of use, and preferred method overall.
Data Analysis A summary score was calcuIated for each subject from ratings in the four areas: for each question a selection of the paper format was scored as -1, equivalence in preference was scored as 0, computer was scored as 1, and then the responses were summed. The statistical significance of the summary score in the sample population was determined by a two-sided sign test, calculated from the binomial distribution. Although this study was not intended to provide an accurate assessment of reliability between the two formats, we wanted to determine if categorical scores and overall scores in both formats were comparable. As an estimate, the least squares linear regression correlation coefficient was calculated for each of the 12 section scores, the physical and psychosocial scores, and overall scores. RESULTS Forty-eight subjects, 36 men and 12 women, volunteered for the study between September 26 and December 16,1992. Most subjects had graduated from grade 12 but a large percentage (38%) had not reached that educational level. Seventy-three percent of the subjects rated themselves as having very little or no computer experience. The median age category of subjects was 31 to 35 years at the time of the study (fig 1). Twenty-three subjects completed the computer form first and 25 completed the paper form first. The frequency of subject choices (paper format, computer format, or either format being equivalent) for each of the four areas assessed is shown in figure 2. The subjects did not show a preference for either the computer format or the paper format in the areas of comfort or understanding; they most often rated the computer higher in ease of use and overall preference. Calculation of summary scores for each subject showed
CLIENT-COMPUTER
Paper Either Computer Easiest to Understand 301
30-r
Maitland
Sleep and rest Emotional behavior Body care and management Home management Mobility Social integration Ambulation Alertness behavior Communication Work Recreation and pastimes Eating Physical Psychosocial Overall * Not a valid correlation.
Paper Either Computer Preferred Methoh
Paper Either Computer Easiest to Use _
Fig 2-Frequency of client choices after they had completed both formats of the SIP questionnaire.
significantly more individuals on the positive side of this distribution @ = .02). This result indicates that the study sample chose the computer format over the paper format more often in the four areas assessed (fig 3). Retrospectively, summary scores were divided into two age ranges (up to age 35, over age 35) to determine if older individuals respond more negatively to the computer format. In the younger age group 7 of 25 subjects (28%) responded negatively, whereas in the older age group 6 of 21 subjects (29%) responded negatively. Correlation coefficients varied from 0.55 to 0.93, which is comparable to studies of the reliability of the SIP questionnaire (table).’ The correlation coefficients for the physical, psychosocial. and overall scores are very acceptable being greater than 0.90. The correlation coefficient of the “Work” section is not presented in table 1 because of problems with its validity. In answering both the paper and the computer questionnaires many subjects misunderstood directions and incorrectly responded that they were working (question Wl) although, in fact, none of the subjects was working. In marking the paper questionnaire, support staff corrected these
-4 -3 Ailresponser favour paper
Fig 3-Frequency
-2
-1
0
641
Linear Regression Correlation Coefficients For Computer and Paper Formats of SIP Category Scores and Overall Score
25
Paper Either Computer Most Comfortable
INTERFACE,
1
Summary Score
2
3
4
All ~es~onscs favourcompulcr
distribution of summary scores.
.7x .94 .84 .67 .75 .85 .93 .90 .93 * .55 .&I .90 .93 .92
_
See text for details.
responses and reported the corrected scores but they were unable to correct the computer format. DISCUSSION Our results support the development of computerized questionnaires using a direct client-computer interface as a result of two findings. First, clients selected the computer format significantly more often than the paper format in ease of use, understanding, comfort, and preference overall. Second, support staff time required to produce a report was reduced from 5:09 minutes to 39 seconds. Furthermore, efficient collection, analysis, and reporting of questionnaire data should enable improved documentation of programs for justification, administrative, or research uses. Numerous computer interfaces are being used currently: touch screens, voice, bar code readers, mouse, and others. The numeric keypad was chosen because it is inexpensive, easy to program, resilient to heavy use and is believed to be the most familiar to the client. Other investigators have attempted to use touch screens or mouse input.‘” They found many patients were apprehensive about touching the screen or using the mouse. For some clients the computerized method is clearly the method of choice. Previous studies indicate that information may be better acquired by direct client-computer interface if the client is uncomfortable about disclosing the information to another person. For example, significantly more rcports of sexual abuse were related with the use of a computerized questionnaire compared with a paper format7 Similarly, in a study assessing human immunodeficiency virus (HIV) risk characteristics, more HIV-related factors were elucidated by computer interview.’ Level of physical ability may also determine if a particular client-computer interface is expedient or whether it is impractical. In this study all clients were physically able to complete the questionnaire. Difficulties with grasp, coordination, or physical tolerance may be overcome with alternate interfaces.” Otherwise, physically disabled clients may require assistance entering data. Some clients cannot cope with an automated system. For example, Coombs and coworkers” suggested that support Arch Phys Med Rehabil Vol75,
June 1994
CLIENT-COMPUTER
642
staff assist any client over the age of 65 with an automated medical history system. In the current study we did not find preference for either format to be associated with age. Arbitrary criteria such as age guidelines clearly depend on the complexity of the task and other factors. Perhaps more important criteria should be based on literacy, language skills and anxiety, both for the paper and the computer formats. These areas have not been investigated to our knowledge. Burke and Normand’ summarize aspects of the test-taker/ computer interaction that promote effective use by large patient populations. These include clarity of instructions, allowing for practice, and error checking. Though we have focused on the client-computer interface, similar methods may be used by health care professionals and support staff in situations that demand various forms of data entry. For example, clinics may be required to collect patient status data in a computerized format. Similar interfaces can be used at this level to minimize keystrokes, produce the required reports. - i minimize the stress impact on staff. CONCLUSION Subjects on average did not show a preference for the paper format or computer format in the areas of comfort or understanding. Subjects overwhelmingly choose the computer format as easier to use and as the preferred method. These findings support the integration of a direct clientcomputer interface for questionnaire data in an occupational rehabilitation setting where most clients are laborers with grade 12 education or less. Correlation coefficients for paper and computer formats were acceptable for physical, psychosocial, and overall scores. Modification to the “work” section of the SIP questionnaire may improve its validity with regards to a nonworking population. Computerized reporting of scores reduced support staff time to 13% of the time required to produce a report from
Arch Phys Med Rehabil Vol75,
June 1994
INTERFACE,
Maitland
the paper format. This finding indicates a potentially important area for increased staff productivity depending on the environment. Acknowledgment: The authors thank their assistance in collecting the data.
K. Nielsen
and T. Lipovski
for
References I. Burke MJ, Normand J. Computerized psychological testing: overview and critique. Prof Psycho] 1987; I8:42-5 1. 2. Bergner M, Bobbitt RA, Carter WB, Gilson BS. The sickness impact profile: development and final revision of a health status measure. Med Care 1981; 19:787-805. 3 Carey RG. Seibert JH, Posavac ET. Who makes the most progress in inpatient rehabilitation? An analysis of functional gain. Arch Phys Med Rehabil 1988;69:337-43. Huba GJ. Comparability of traditional and computer Western Personnel Test (WPT) versions. Educ Psych Measure 1988;48:957-9. Lushene RE, O’Neil HH. Dunn T. Equivalent validity of a completely computerized MMPI. J Pers Assess 1974;38:353-61. Locke SE, Kowaloff HB, Hoff RG, Safran C. Popovsky MA, Cotton DJ, et al. Computer-based interview for screening blood donors for risk of HIV transmission, JAMA 1992;268: 1301-5. 7 Bagley C. Genius M. Psychology of computer use: XX. Sexual abuse recalled: evaluation of a computerized questionnaire in a population of young adult males. Percept Mot Skills 1991;72:287-8. 8 Slack WV, Van Cura LJ. Computer-based patient interviewing II. Postgrad Med I968;43: I 15-20. 9 Greist JH. Klein MH, Van Cura LJ. A computer interview for psychiatric patient target symptoms. Arch Gen Psych 1973;29:247-53. IO Roizen MF, Coalson D, Hayward RS, Schmittner J, Thisted RA, Apfelbaum JL, et al. Can patients use an automated questionnaire to define their current health status? Med Care 1992;30:MS74-MS84. Il. Sampson JP. Computer-assisted testing and assessment: current status and implications for the future. Measure Eva1 Guidance 1983;5: 293-9. 12. Coombs GJ, Murray WR, Krahn DW. Automated medical histories: factors determining patient performance. Comput Biomed Res 1970;3:178-81. Suppliers a. d Base IV. Borland, 1800 Green Hills Road, PO Box 660001, Scotts (alley. CA 95067-0001, b. lenovations numeric keypad. Genovation Incorporated, 17741 Mitchell, iorth Irvine, CA.