- Email: [email protected]
Internet versus live: Assessment of government documents bibliographic instruction
Journal of Government Information 30 (2004) 561 – 574
Internet versus live: Assessment of government documents bibliographic instruction Jeffrey M. Wilhite* Bizzell Memorial Library, University of Oklahoma, 401 West Brooks, Norman, OK 73019-0528, USA
Abstract The issue of live bibliographic instruction (BI) versus electronic BI is a matter many libraries are facing as more technology becomes available to afford such an option. At the University of Oklahoma Government Documents Collection, a study was administered in March 2001 to determine the relative advantages between these two teaching techniques. Using three groups (live, Internet, and control), a pre- and posttest/evaluation were utilized to produce statistical results. In the end, both the live and Internet groups’ posttests showed statistical significance when compared to the control group but showed no statistical significance when compared to each other. The evaluation and comment section of the posttest were enlightening, as live instruction was preferred over the Internet instruction in most instances. The final result of the study showed no statistical significance between the posttests of the live group and the Internet group, but the subjects preferred live government documents BI to the Internet version. This is important to note as more emphasis is being placed on the possibility of electronic BI for the future of library instruction. D 2004 Elsevier Inc. All rights reserved. Keywords: Bibliographic instruction; Internet; Live
1. Introduction The Internet is changing the way academic librarians teach and offer learning experiences. One of the foremost changes is the delivery of bibliographic instruction (BI) in the form of * Fax: +1 405 325 1841. E-mail address: [email protected]. 1352-0237/$ - see front matter D 2004 Elsevier Inc. All rights reserved. doi:10.1016/j.jgi.2004.10.002
562
J.M. Wilhite / Journal of Government Information 30 (2004) 561–574
Internet tutorials.1 The issue of Internet BI has been under discussion at the University of Oklahoma, Norman, since the technology advanced to a level that would permit such dialogue. The University of Oklahoma (OU) is a second-tier school,2 with an enrollment of 23,800 students. The undergraduate population comprises 81% of total enrollees. Bizzell Memorial Library is the main library for the Norman campus. With a collection of 4.1 million volumes, the Bizzell Library, and its many special collections and branches, serve the entire University community, as well as, secondarily, the city of Norman and surrounding areas. The Government Documents Collection is one of the main departments in the Bizzell Library system. Established in 1893, the Documents Collection is an 89% depository that contains 2.6 million items. As one of the largest and most historic document depositories in the state, the collection serves the University and local community as the main focal point for federal, state, and international document information. In the fall of 2000, the author began discussing with Wallace C. Koehler, Jr., a professor from the OU School of Library and Information Studies (SLIS), video taping, digitizing, and uploading government documents BI sessions to the Internet for the use of his spring 2001 government publications class. The outline of this project involved the video taping of the author’s BI session; the digitizing, editing, and uploading of the video to the Internet; and editing of the final Internet product. The author’s initial interest in this project centered on assisting a fellow faculty member and his students. As the class was being taught completely online, the author realized that the Internet BI session would assist the SLIS students at the Norman campus, as well as distance learners. The videotaping took place during October 2000. The author presented his 45-minute basic documents BI class to the camera. The video taping was completed in segments covering the presentation of the bTop Ten Most Useful Government Document Resources,Q as well as a brief tour of the closed stacks of the Government Documents Collection. The core of this BI session included explanations and demonstrations of OU’s online catalog, the GPO Database (through Firstsearch), the historical subject and title paper indexes (the Monthly Catalog, the Cumulative Subject Index to the Monthly Catalog of United States Publications 1900–1971, and the Cumulative Title Index to United States Public Documents 1789–1976), Congressional Universe, an overview of U.S. Census materials, Statistical Universe, the general reference area of the Documents Collection, periodicals, Internet resources, and interlibrary loan. In January 2001, the video was digitized and edited to produce a draft version. The author reviewed the results, and after some additional editing, a final version of this Internet video was uploaded to the Library School’s Web server in February 2001. The final product consisted of a high-quality 45-minute video presentation accessible through the Internet. This Internet video, unlike normal videotape, offered the advantage of being viewed from any personal computer (PC) without the constraints of having to have a TV and VCR. As this was a germinal effort, it was decided to leave the final product simple—with no additional Web-based components. This decision made the product interactive to the user only in the ability to manipulate the play, review, fast-forward, and stop functions. The only requirement needed to play the video was Real Player, which was a free download on the Internet.
J.M. Wilhite / Journal of Government Information 30 (2004) 561–574
563
Satisfied with the end product, Wallace Koehler now had a useful Internet tool that could be used by his online class. Students in the government publications class responded positively to the video BI class, claiming it to be a useful variation on a live BI session. Impressed with the results, the author ran a link to it from the Government Documents homepage.3 Less than a week after the posting of the Internet BI session on the Government Documents Web page, the author decided to make it the focus of a study. With the session linked to the department’s Web page, an opportunity arose to use the presentation as a supplement to live BI classes to determine if Internet document introduction was as useful, or perhaps better, than an in-person BI session for government documents instruction. As a result, the author decided to run a pre- and posttest/evaluation with three student groups: a control group, a group who received live BI training, and a group who received Internet BI training. An important precept of the Internet side of this study was the student’s ability to monitor individually the Internet video, allowing for the pausing, rewinding, etc. Without the possibility of personal manipulation, the training session would amount to no more than a group video presentation of an Internet BI session. This basic level of Internet interactivity would allow the students to work at their own speed and hopefully would produce a better understanding of the material presented.
2. Literature review Research into Internet BI began as a rousing fusillade in the late 1990s. The development of the Web has greatly increased the electronic options that libraries now have available.4 As Internet technologies began to develop in libraries, the possibility of taking away some of the daily personal BI responsibilities of librarians began to emerge. Online tutorials were hailed as the way to stop the bcatch-as-catch-canQ approach to library instruction. They were praised for allowing librarians more time to concentrate on specific and advanced BI classes. Internet BI seemed designed for the library patron of the late 1990s: available from any PC and accessible 24 hours a day, completely at the patron’s convenience.5 Simple online tutorials, with little interactivity, initially appeared to be a perfect method to handle a diverse university student body, including nontraditional, international, and distance learning students.6 Internet BI was also being proclaimed as appealing to students who would hesitate to ask questions in a normal class but who would flourish in an online environment.7 From the profession’s point of view, Internet BI even appeared to offer the possibility of ceasing some librarian burnout by ending the deleterious repetition of BI sessions.8 As the technology became more sophisticated and available, Internet BI sessions became more interactive. Live sessions or library handouts that had initially been developed into Internet slide shows morphed into video products accessible directly from the Web. These products now have the possibility of even greater interactivity by including Web-based components that allow further user participation directly with the product. The recent advances include embedded hyperlinks to other sources, graphical interfaces, and PowerPoint
564
J.M. Wilhite / Journal of Government Information 30 (2004) 561–574
presentations that incorporate video and sound.9 This accelerated rate of development quickly outdates products, proving that all Web-based products are not equal. The first tier of research on this topic focused on the pros and cons of implementing Internet BI. Although judgment of this issue fell more toward the positive than the negative, there were still certain negative issues discussed. Any level of Internet BI required knowledge of basic Web design to create and maintain such a Web site, as well as the time to implement the project. A more obvious negative was the expense for the technology to start and maintain such a program.10 Another trend in the research wrestled with the pedagogical implications of BI on the Web, embracing discussions of the lack of active learning inherit in some Webbased technologies.11 The second tier of research appeared to focus on the actual implementation of this technology in the library world. Topics included the different levels of Internet BI products that could be created, as well as the specific factors involved in designing each of these products. Some of the basic factors that applied to all levels of Internet BI products were appearance, legibility, layout, and ADA compatibility.12 Software choices began to be debated, with some formats heralded over others. Software products designed specifically for the purpose of online tutorials, such as Macromedia Authorware, were lauded over products that were not as specific or applicable for online library instruction.13 More recent research on this topic has centered around evaluating live versus Internet BI. Some basic evaluation first occurred in 1997 at the University of Louisville (Kentucky). The product that was evaluated was a video taped BI session accessible directly from the Internet. A questionnaire was given to users of this library online tutorial. Results suggested the tutorial was considered clear and understandable and that negative comments tended to reflect individual differences in comprehension or skill level.14 A study at Colorado State University provided a basic assessment of their online library Web pages by analyzing visits to each page, as well as evaluation forms completed by instructors.15 By 2000, more extensive scientific evaluation began to occur. A study at the University of Montana used a Web-based evaluation tool to evaluate users’ opinions of their live BI program. Results showed the value of assessment in the library setting, specifically in Internet applications.16 At the University of North Carolina at Chapel Hill, a study was completed comparing live BI to a self-paced, interactive online tutorial. Students were tested with preand posttests, and an evaluation form was utilized. The study found no statistically significant differences between the two methods of instruction in terms of postinstruction performance, although students did favor the pace of the tutorial.17 The bulk of the Internet BI research appears to have been focused on main reference and its services, not government documents. To be sure, the breadth and depth of research involving government documents and the Internet is enormous, but the research has apparently not been specific to Internet BI for government documents collections. Possibly, government documents was generally considered too specialized to survive in the electronic BI medium. From experience, the author realized that even a general Internet government documents BI session might be more complicated and lengthy than a general reference BI session.
J.M. Wilhite / Journal of Government Information 30 (2004) 561–574
565
3. Methodology The study is based on a pre- and a posttest/evaluation of three groups of students—an Internet BI group, a live BI group, and a control group. The largest classes to participate in government documents BI each semester are those in Business Communications. These sophomore level classes consist of a cross section of 60–70 undergraduate students, from freshmen to seniors. The coordinator of the spring 2001 Business Communication class agreed to participate in the project. The entire study had to be completed in one 75-minute class. The date chosen for the study was March 15, 2001, with a class of 68 students. The business professor had already randomly broken the class into 11 study sections. The author divided these sections into three groups of 22–23 students each. The author made the inperson BI presentation, the OU Social Science Reference Librarian administered the Internet presentation to another group, and the business professor instructed the control group. The live BI group participated in the study in the Government Documents Collection. The Internet BI group met in Bizzell Library’s Electronic BI Room. The control group met in the business professor’s office. The basic outline of the study involved five steps: ! ! ! ! !
Informed Consent Form Government Document Instruction Sessions flyer Pretest Instruction Posttest/Evaluation.
Upon entering their assigned location, the students reviewed and signed an Informed Consent Form. This alerted the students as to the nature of the study and explained their full rights as participants (bParticipation in this study is voluntary. . . . You may discontinue participation at any time.Q). The use of this form is mandatory for any experiments dealing with human subjects at the University according to the Office of Research Administration’s guidelines. These signed forms were then collected. Next, the students were given a flyer stating that if they did not receive the method of instruction they believed they deserved, they could arrange for a special BI session for a later date. This step was added so no student felt denied a certain level of service, especially the members of the control group who would not be receiving any instruction on the day of the study. In the third step of the study, pretests were administered to each student (see Appendix A). The form of the pretest was one page, single sided. The title of the study was at the top. There was also an admonition to bnot place your name on this form.Q The pretest contained five questions, which centered on basic knowledge of the documents collection. Both BI sessions were designed to teach participants a basic knowledge of the Government Documents Collection. The questions were each multiple choice, with three choices for each question. The questions were designed to be challenging, but not impossible to answer
566
J.M. Wilhite / Journal of Government Information 30 (2004) 561–574
Table 1 Control group, pretest versus posttest Group
Number
Mean
T distribution
Significance
Control (pretest) Control (posttest)
13 11
7.40 6.20
1.05
0.323
with a basic knowledge of the documents collection. Ten minutes were allowed for the students to answer and return this pretest. Pencils were provided to the students. The fourth step was the instruction itself. The Internet group was instructed on how to access the Internet video and how to manipulate it (stop, pause, rewind, and volume). Each session had 45 minutes of training. The control group received no training. Finally, each group was given a posttest/evaluation (see Appendix B). The form of the posttest/evaluation was one sheet of paper, double sided. The only difference in the front page of the posttest was the title at the top of the page and a note on the bottom to bOver Please.Q Due to the complex nature of government document collections, the questions on the pre- and posttest/evaluation were the same. Personal information and the evaluation questions were on the reverse side of the page. The personal information section asked three basic demographic questions (college level, gender, and age). The evaluation section contained four questions designed to determine the usefulness of the mode of instruction each group received. The fifth point asked for personal comments. The page ended with the statement, bThank you for your assistance with this study.Q
4. Participant information On the day of the study, 44 students willingly participated. The subjects were mainly sophomores (53%), with juniors (30%), and seniors (17%) also participating. A slight majority of the subjects were male (53%). The subjects’ ages ranged from the late teens to the early twenties: 19 (18%), 20 (45%), 21 (18%), 22 (10%), and other (9%). Thus, the average subject for this study was a 20-year-old male sophomore.
5. Results The three groups were compared on the basis of pretests scores to posttests scores. The control group’s score was not statistically significant at the 0.05 level (see Table 1). The Table 2 Internet group, pretest versus posttest Group
Number
Mean
Internet (pretest) Internet (posttest)
18 16
11.6 12.2
T distribution 0.283
Significance 0.784
J.M. Wilhite / Journal of Government Information 30 (2004) 561–574
567
Table 3 Live group, pretest versus posttest Group
Number
Mean
Live (pretest) Live (posttest)
13 13
7.60 11.6
T distribution 2.30
Significance 0.050
Internet group’s pre- and posttests were not significantly different (see Table 2). This group did have an unexpectedly high mean in their posttest, something the other two groups did not show. There was a statistically significant difference in the live group’s pre- and posttest (see Table 3). The posttest mean results were unexpected. The Internet group’s posttest score was significantly higher than the posttest of the control group (see Table 4). This was also the case with the live group’s posttest score, as it was also significantly higher than the posttest of the control group (see Table 5). There was no statistically significant difference between the posttest of the Internet group and the posttest of the live group (see Table 6).
6. Evaluation The evaluation section of the posttest contained four general questions and a section for comments. Unlike the statistical data produced, the evaluations and comments varied considerably from the live group to the Internet group. Question 1 (see Table 7) asked, bDid you find this library instruction session helpful? Q In the live group (n = 13), 93% of respondents answered byesQ to this question, while only 7% agreed it was bsomewhatQ helpful. In the Internet group (n = 16), 50% of respondents answered bsomewhatQ to this question, while only 25% answered byes,Q and 25% answered bno.Q According to these responses, the live group was more satisfied with their instruction session than the Internet group. Question 2 (see Table 8) asked, bWas the information presented to you in a clear and understandable fashion?Q One hundred percent of the live group answered byesQ to this question. In the Internet group, only 33% answered byesQ while 59% answered bsomewhatQ and 8% answered bno.Q The live group responded more positively with regard to their BI session than did the Internet group. Question 3 (see Table 9) asked, bHow did you find the length of this session?Q In the live group, 93% of respondents answered that the session was the bright length,Q whereas 7% thought it was btoo long.Q In the Internet group, 83% of respondents thought the session was Table 4 Comparison of control and Internet group posttest Group
Number
Mean
Control (posttest) Internet (posttest)
11 16
6.20 12.2
T distribution 3.36
Significance 0.010
568
J.M. Wilhite / Journal of Government Information 30 (2004) 561–574
Table 5 Comparison of control and live group posttest Group
Number
Mean
Control (posttest) Live (posttest)
11 13
6.20 11.6
T distribution 5.69
Significance 0.000
btoo long,Q with only 17% answering that the session was the bright length.Q In a third instance, the live group was again more satisfied with their BI session when compared to the Internet group. Question 4 (see Table 10) asked, bWhich method of library instruction session would you have preferred to take for this session?Q In the live group, 93% responded blive classQ while 7% answered bindependent study.Q In the Internet group, 42% of respondents noted bInternetQ as their preferred method of BI, while 33% noted bindependent studyQ and 25% noted blive class.Q The live group clearly approved of their live BI class and chose it over any other form of instruction. The Internet group was scattered in its choices for BI, with each form of BI receiving a smattering of approval. There was no outstanding statistical preference.
7. Analysis The last section of the posttest/evaluation (see Table 11) asked for comments to questions such as, bWhat was the best or worst part of this library instruction session; What did you wish you had been presented that you did not see,Q etc. A majority of freshmen at OU participate in a live general library training session, so most participants of this study would have had previous experience with a library BI session on the University level. Interestingly, the comments were perfectly divided between the two groups, with 100% of the live group’s comments (n = 5) being positive and 100% of the Internet group’s comments (n = 6) being negative. The positive comments of the live group included bpresentation was good,Q bvery informative and helpful,Q and bI think a live class is better so that you are able to ask questions.Q These questions reflected the live group’s general satisfaction with their BI session as exemplified in the previous evaluation questions. Conversely, the Internet groups’ comments were entirely negative. Whereas most comments focused on issues of technology, two comments did focus on the material presented, bWorst: too long,Q and bToo much detail. . . . If the student wants the extra details then they will ask.Q It is notable that comments about the length were registered from the Table 6 Comparison of Internet and live group posttest Group
Number
Mean
T distribution
Significance
Internet (posttest) Live (posttest)
16 13
12.2 11.6
0.325
0.753
J.M. Wilhite / Journal of Government Information 30 (2004) 561–574
569
Table 7 Question 1: Did you find this library instruction session helpful? Answer
Live group (%)
Internet group (%)
Yes Somewhat No
93 7 –
25 50 25
Internet group and not the live group, as both sessions were timed to be the same length. The Internet group also got to sit through their entire session, while the live group had to walk through the Documents Collection. This level of physical output on the subject’s part might contribute to a negative reaction about the length of the session; however, the nonphysically exerting Internet group responded negatively about the length of their session. The interaction of a live speaker may have offset the negativism of the live group’s necessary physical output, whereas the lack of a live speaker may have contributed to the negative responses of the nonphysically asserting Internet group. A majority of the negative responses from the Internet group centered on the technology with which they were faced. Comments included, bIt was really hard to watch this over the Internet in a classroom with everyone else watching the same thing, it made it hard to focus,Q bCouldn’t understand because of all the other computers,Q and bToo many computers showing movie at once.Q These comments focused on issues of concern when preparing for the Internet BI session. The first concern was that the Electronic BI Room had only 15 PCs. This was about two-thirds the number of computers devised for the study to allow individual student assessment of the BI Internet video. As this was the only Electronic BI Room available, it was anticipated that some students would need to double-up to accommodate the lack of equipment. The second concern of the room was having 15 BI sessions playing at the same time. Headphones were dismissed as an option after testing all 15 PCs synchronously playing the Internet video; the noise was not deemed sufficient to interrupt normal concentration. What had not been tested was the noise level of 18 college students in the room, most of whom were 20-year-old male sophomores. The live group was a quiet and attentive audience. The Internet group was loud and talkative. This certainly needs to be factored into the study results. In replicating this study, headphones should be provided to each Internet subject. Overall, the entire Internet group’s comments were negative, be it because of the material presented or problems with the technology involved in the study. This, compared with the entirely positive comments of the Table 8 Question 2: Was the information presented to you in a clear and understandable fashion? Answer
Live group (%)
Internet group (%)
Yes Somewhat No
100 – –
33 59 8
570
J.M. Wilhite / Journal of Government Information 30 (2004) 561–574
Table 9 Question 3: How did you find the length of this session? Answer
Live group (%)
Internet group (%)
Too short Right length Too long
– 93 7
– 17 83
live group, again draws a stark comparison between the two. Even with the added information from the Internet group, the live group still appears to be more completely satisfied with the form of BI they received.
8. Discussion The statistical results of this study essentially duplicated the findings of the previous Internet versus live reference studies. The Internet and live groups both improved after instruction, and both scored significantly higher than the control group. The live group improved significantly, whereas the Internet group did not. Given this finding, it was surprising that the live versus Internet group, when compared with one another, did not show more statistical significance. The results of this study showed little difference between offering live or electronic BI for government documents. This result was surprising, as the author assumed government documents BI, unlike general reference BI, is complicated and intensive enough to require a live BI session. According to the statistical results, government documents appeared to fare as well as reference in the electronic BI medium. When the evaluation and comments are factored into this, the final conclusion changes slightly. Government documents BI may be able to be presented to subjects in a live or Internet format and produce essentially the same results, but the human response to each of these BI methods may be dramatically different. According to the evaluation and comments, live government documents BI rated more positively than Internet government documents BI. This result needs to be tempered with the added technological concern of the Internet group, specifically the noise level in the room where their section of the study was conducted. On the whole, while it does appears that the subjects fared statistically as well in an Internet BI environment as in a live BI environment, the subjects expressed greater satisfaction with a live BI class than an Internet class for government documents instruction.
Table 10 Question 4: Which method of library instruction session would you have preferred to take for this session? Answer
Live group (%)
Internet group (%)
Internet Live class Independent study
– 93 7
42 25 33
J.M. Wilhite / Journal of Government Information 30 (2004) 561–574
571
Table 11 Comments Answer
Live group (%)
Internet group (%)
Positive N/A Negative
100 – –
– – 100
Government documents librarians should heed the results of this study when considering a decision to present live BI or electronic BI. Both librarians and library administrators need to consider the results of this study before funds are allotted and staff time delegated to create an electronic BI program. The students may learn as much in electronic BI as they do in live BI, but they certainly do not appear to respond as positively to it as to an in-person presentation explaining government documents. 9. Conclusion The University of Oklahoma Government Documents Collection continues to use the Internet BI session in a limited capacity. The link to the Internet BI session on the Web page remains for general patron use and for some student employee training, but the author provides live BI sessions for all classes coming into the documents collection. Live BI classes are informed about the Internet BI session so they can utilize it for review after their live session. In this age of dropping library statistics and 60% of current government publications going directly to the Internet,18 a decision was made to give the patrons what they apparently preferred: human contact and experience of a librarian leading a bibliographic instruction session introducing the Government Documents Collection. Appendix A. Pretest For the study: Internet versus live: Assessment of government documents bibliographic instruction March 15, 2001 Circle the most appropriate answer DO NOT PUT YOUR NAME ON THIS FORM 1. The Government Documents Collection is what percentage of a federal depository? a. Almost 80% b. Almost 90% c. 100% 2. The Government Documents Collection contains items in the following formats: a. Paper only b. Paper and electronic c. Paper, electronics, and others
572
J.M. Wilhite / Journal of Government Information 30 (2004) 561–574
3. Government Document items are not accessible on the Main Library Electronic Catalog Web page: a. Yes b. Some are available, others are not c. No 4. Congressional Universe is an Internet database that allows indexing access and some full-text items to: a. Information about Congress, 1790s to present b. Information about Congress, 1970s to present c. Information about Congress, 1990s to present 5. The Government Documents Collection contains what level of government information: a. Federal, State of Oklahoma, and some International b. Federal and State of Oklahoma c. Federal and some International
Appendix B. Posttest and evaluation For the study: Internet versus live: Assessment of government documents bibliographic instruction March 15, 2001 Circle the most appropriate answer DO NOT PUT YOUR NAME ON THIS FORM 1. The Government Documents Collection is what percentage of a federal depository? a. Almost 80% b. Almost 90% c. 100% 2. The Government Documents Collection contains items in the following formats: a. Paper only b. Paper and electronic c. Paper, electronics, and others 3. Government Document items are not accessible on the Main Library Electronic Catalog Web page: a. Yes b. Some are available, others are not c. No 4. Congressional Universe is an Internet database that allows indexing access and some full-text items to: a. Information about Congress, 1790s to present b. Information about Congress, 1970s to present c. Information about Congress, 1990s to present 5. The Government Documents Collection contains what level of government information: a. Federal, State of Oklahoma, and some International b. Federal and State of Oklahoma c. Federal and some International
PERSONAL INFORMATION (optional) COLLEGE LEVEL: GENDER: AGE:
Freshman
Sophomore
Male ________ (please note)
Female
Junior
Senior
J.M. Wilhite / Journal of Government Information 30 (2004) 561–574
573
EVALUATION 1. Did you find this library instruction session helpful? YES SOMEWHAT NO 2. Was the information presented to you in a clear and understandable fashion? YES SOMEWHAT NO 3. How did you find the length of this session: TOO SHORT RIGHT LENGTH TOO LONG 4. Which method of library instruction session would you have preferred to take for this session? INTERNET LIVE CLASS INDEPENDENT STUDY 5. COMMENTS (e.g., What was the best or worst part of this library instruction session; What did you wish you had been presented that you did not see, etc. . .) Thank you for your assistance with this study.
Notes 1. Tobin, T. and Kesselman, Martin A. (2000). Evaluation of Web-based library instruction programs. INSPEL, 34(2), 67. 2. As defined by the ranking system of U.S. News and World Report Online. (2001, September). Available: http://www.usnews.com/usnews/edu/college/directory/drglance_3184.htm 3. University of Oklahoma Government Documents Web Page: http://libraries.ou.edu/depts/govdocs/ 4. Crowther, Karmen N.T., & Wallace, Alan (2001). Delivering video-stream library orientation on the Web. C&RL News, 62(3), 280. 5. Ardis, Susan (1998). Creating Internet-based tutorials. Information Outlook, 2(10), 17. 6. Kocour, Bruce G. (2000). Using Web-based tutorials to enhance library instruction. College and Undergraduate Libraries, 7(1), 50. 7. Getty, Nancy, Burd, Barbara, & Burns, K. Sarah (2000). Using courseware to deliver library instruction via the Web: Four examples. Reference Services Review, 28(4), 352. 8. Fowler, Clara S., & Dupis, Elizabeth A. (2000). What have we done? TILT’s impact on our instruction program. Reference Services Review, 28(4), 343. 9. Dewald, Nancy H. (1999). Web-based library instruction: What is good pedagogy? Information Technologies and Libraries, 18(1), 26. 10. Murphy, Deborah A. (1998). Building the library starter kit: Developing new technologies for instruction. Reference Services Review, 26(3–4), 111. 11. Dewald, Nancy H. (1999). Web-based library instruction: What is good pedagogy? Information Technologies and Libraries, 18(1). 12. Ardis, pp. 19–20. 13. Johnson, Anna Marie (1998). Too many students, too little time: Creating and implementing a self-paced, interactive computer tutorial for the libraries’ online catalog. Research Strategies, 16(4), 274 – 276. 14. Johnson, p. 279. 15. Lederer, Naomi (2000). New form(at): Using the Web to teach research and critical thinking skills. Reference Services Journal, 28(2), 130. 16. Samson, Sue (2000). What and when do they know? Web-based assessment. Reference Services Review, 28(4). 17. Holman, Lucy (2000). A comparison of computer-assisted instruction and classroom bibliographic instruction. Reference and User Services Quarterly, 40(1), 53. 18. U.S. Government Printing Office (2001). Library programs service update, spring 2001 depository library council meeting. Administrative Notes: Newsletter of the Federal Depository Library Program, 22(6), 9.
574
J.M. Wilhite / Journal of Government Information 30 (2004) 561–574
Jeffrey M. Wilhite is an Associate Professor of Bibliography, Government Documents Reference Librarian, for the University of Oklahoma. He has held his current position since 1995 and was tenured in June 2001. He manages a collection of 2.6 million U.S. Federal, State of Oklahoma, and International documents. The OU documents collection was established in 1893 and is one of the oldest and largest government depository libraries in the state of Oklahoma. He has served as an adjunct professor in the University of Oklahoma School of Library and Information Studies, teaching the Government Publications class. He is currently working on the book, A Chronology of Librarianship, 1960–2000, for Scarecrow Press.
Internet versus live: Assessment of government documents bibliographic instruction Jeffrey M. Wilhite* Bizzell Memorial Library, University of Oklahoma, 401 West Brooks, Norman, OK 73019-0528, USA
Abstract The issue of live bibliographic instruction (BI) versus electronic BI is a matter many libraries are facing as more technology becomes available to afford such an option. At the University of Oklahoma Government Documents Collection, a study was administered in March 2001 to determine the relative advantages between these two teaching techniques. Using three groups (live, Internet, and control), a pre- and posttest/evaluation were utilized to produce statistical results. In the end, both the live and Internet groups’ posttests showed statistical significance when compared to the control group but showed no statistical significance when compared to each other. The evaluation and comment section of the posttest were enlightening, as live instruction was preferred over the Internet instruction in most instances. The final result of the study showed no statistical significance between the posttests of the live group and the Internet group, but the subjects preferred live government documents BI to the Internet version. This is important to note as more emphasis is being placed on the possibility of electronic BI for the future of library instruction. D 2004 Elsevier Inc. All rights reserved. Keywords: Bibliographic instruction; Internet; Live
1. Introduction The Internet is changing the way academic librarians teach and offer learning experiences. One of the foremost changes is the delivery of bibliographic instruction (BI) in the form of * Fax: +1 405 325 1841. E-mail address: [email protected]. 1352-0237/$ - see front matter D 2004 Elsevier Inc. All rights reserved. doi:10.1016/j.jgi.2004.10.002
562
J.M. Wilhite / Journal of Government Information 30 (2004) 561–574
Internet tutorials.1 The issue of Internet BI has been under discussion at the University of Oklahoma, Norman, since the technology advanced to a level that would permit such dialogue. The University of Oklahoma (OU) is a second-tier school,2 with an enrollment of 23,800 students. The undergraduate population comprises 81% of total enrollees. Bizzell Memorial Library is the main library for the Norman campus. With a collection of 4.1 million volumes, the Bizzell Library, and its many special collections and branches, serve the entire University community, as well as, secondarily, the city of Norman and surrounding areas. The Government Documents Collection is one of the main departments in the Bizzell Library system. Established in 1893, the Documents Collection is an 89% depository that contains 2.6 million items. As one of the largest and most historic document depositories in the state, the collection serves the University and local community as the main focal point for federal, state, and international document information. In the fall of 2000, the author began discussing with Wallace C. Koehler, Jr., a professor from the OU School of Library and Information Studies (SLIS), video taping, digitizing, and uploading government documents BI sessions to the Internet for the use of his spring 2001 government publications class. The outline of this project involved the video taping of the author’s BI session; the digitizing, editing, and uploading of the video to the Internet; and editing of the final Internet product. The author’s initial interest in this project centered on assisting a fellow faculty member and his students. As the class was being taught completely online, the author realized that the Internet BI session would assist the SLIS students at the Norman campus, as well as distance learners. The videotaping took place during October 2000. The author presented his 45-minute basic documents BI class to the camera. The video taping was completed in segments covering the presentation of the bTop Ten Most Useful Government Document Resources,Q as well as a brief tour of the closed stacks of the Government Documents Collection. The core of this BI session included explanations and demonstrations of OU’s online catalog, the GPO Database (through Firstsearch), the historical subject and title paper indexes (the Monthly Catalog, the Cumulative Subject Index to the Monthly Catalog of United States Publications 1900–1971, and the Cumulative Title Index to United States Public Documents 1789–1976), Congressional Universe, an overview of U.S. Census materials, Statistical Universe, the general reference area of the Documents Collection, periodicals, Internet resources, and interlibrary loan. In January 2001, the video was digitized and edited to produce a draft version. The author reviewed the results, and after some additional editing, a final version of this Internet video was uploaded to the Library School’s Web server in February 2001. The final product consisted of a high-quality 45-minute video presentation accessible through the Internet. This Internet video, unlike normal videotape, offered the advantage of being viewed from any personal computer (PC) without the constraints of having to have a TV and VCR. As this was a germinal effort, it was decided to leave the final product simple—with no additional Web-based components. This decision made the product interactive to the user only in the ability to manipulate the play, review, fast-forward, and stop functions. The only requirement needed to play the video was Real Player, which was a free download on the Internet.
J.M. Wilhite / Journal of Government Information 30 (2004) 561–574
563
Satisfied with the end product, Wallace Koehler now had a useful Internet tool that could be used by his online class. Students in the government publications class responded positively to the video BI class, claiming it to be a useful variation on a live BI session. Impressed with the results, the author ran a link to it from the Government Documents homepage.3 Less than a week after the posting of the Internet BI session on the Government Documents Web page, the author decided to make it the focus of a study. With the session linked to the department’s Web page, an opportunity arose to use the presentation as a supplement to live BI classes to determine if Internet document introduction was as useful, or perhaps better, than an in-person BI session for government documents instruction. As a result, the author decided to run a pre- and posttest/evaluation with three student groups: a control group, a group who received live BI training, and a group who received Internet BI training. An important precept of the Internet side of this study was the student’s ability to monitor individually the Internet video, allowing for the pausing, rewinding, etc. Without the possibility of personal manipulation, the training session would amount to no more than a group video presentation of an Internet BI session. This basic level of Internet interactivity would allow the students to work at their own speed and hopefully would produce a better understanding of the material presented.
2. Literature review Research into Internet BI began as a rousing fusillade in the late 1990s. The development of the Web has greatly increased the electronic options that libraries now have available.4 As Internet technologies began to develop in libraries, the possibility of taking away some of the daily personal BI responsibilities of librarians began to emerge. Online tutorials were hailed as the way to stop the bcatch-as-catch-canQ approach to library instruction. They were praised for allowing librarians more time to concentrate on specific and advanced BI classes. Internet BI seemed designed for the library patron of the late 1990s: available from any PC and accessible 24 hours a day, completely at the patron’s convenience.5 Simple online tutorials, with little interactivity, initially appeared to be a perfect method to handle a diverse university student body, including nontraditional, international, and distance learning students.6 Internet BI was also being proclaimed as appealing to students who would hesitate to ask questions in a normal class but who would flourish in an online environment.7 From the profession’s point of view, Internet BI even appeared to offer the possibility of ceasing some librarian burnout by ending the deleterious repetition of BI sessions.8 As the technology became more sophisticated and available, Internet BI sessions became more interactive. Live sessions or library handouts that had initially been developed into Internet slide shows morphed into video products accessible directly from the Web. These products now have the possibility of even greater interactivity by including Web-based components that allow further user participation directly with the product. The recent advances include embedded hyperlinks to other sources, graphical interfaces, and PowerPoint
564
J.M. Wilhite / Journal of Government Information 30 (2004) 561–574
presentations that incorporate video and sound.9 This accelerated rate of development quickly outdates products, proving that all Web-based products are not equal. The first tier of research on this topic focused on the pros and cons of implementing Internet BI. Although judgment of this issue fell more toward the positive than the negative, there were still certain negative issues discussed. Any level of Internet BI required knowledge of basic Web design to create and maintain such a Web site, as well as the time to implement the project. A more obvious negative was the expense for the technology to start and maintain such a program.10 Another trend in the research wrestled with the pedagogical implications of BI on the Web, embracing discussions of the lack of active learning inherit in some Webbased technologies.11 The second tier of research appeared to focus on the actual implementation of this technology in the library world. Topics included the different levels of Internet BI products that could be created, as well as the specific factors involved in designing each of these products. Some of the basic factors that applied to all levels of Internet BI products were appearance, legibility, layout, and ADA compatibility.12 Software choices began to be debated, with some formats heralded over others. Software products designed specifically for the purpose of online tutorials, such as Macromedia Authorware, were lauded over products that were not as specific or applicable for online library instruction.13 More recent research on this topic has centered around evaluating live versus Internet BI. Some basic evaluation first occurred in 1997 at the University of Louisville (Kentucky). The product that was evaluated was a video taped BI session accessible directly from the Internet. A questionnaire was given to users of this library online tutorial. Results suggested the tutorial was considered clear and understandable and that negative comments tended to reflect individual differences in comprehension or skill level.14 A study at Colorado State University provided a basic assessment of their online library Web pages by analyzing visits to each page, as well as evaluation forms completed by instructors.15 By 2000, more extensive scientific evaluation began to occur. A study at the University of Montana used a Web-based evaluation tool to evaluate users’ opinions of their live BI program. Results showed the value of assessment in the library setting, specifically in Internet applications.16 At the University of North Carolina at Chapel Hill, a study was completed comparing live BI to a self-paced, interactive online tutorial. Students were tested with preand posttests, and an evaluation form was utilized. The study found no statistically significant differences between the two methods of instruction in terms of postinstruction performance, although students did favor the pace of the tutorial.17 The bulk of the Internet BI research appears to have been focused on main reference and its services, not government documents. To be sure, the breadth and depth of research involving government documents and the Internet is enormous, but the research has apparently not been specific to Internet BI for government documents collections. Possibly, government documents was generally considered too specialized to survive in the electronic BI medium. From experience, the author realized that even a general Internet government documents BI session might be more complicated and lengthy than a general reference BI session.
J.M. Wilhite / Journal of Government Information 30 (2004) 561–574
565
3. Methodology The study is based on a pre- and a posttest/evaluation of three groups of students—an Internet BI group, a live BI group, and a control group. The largest classes to participate in government documents BI each semester are those in Business Communications. These sophomore level classes consist of a cross section of 60–70 undergraduate students, from freshmen to seniors. The coordinator of the spring 2001 Business Communication class agreed to participate in the project. The entire study had to be completed in one 75-minute class. The date chosen for the study was March 15, 2001, with a class of 68 students. The business professor had already randomly broken the class into 11 study sections. The author divided these sections into three groups of 22–23 students each. The author made the inperson BI presentation, the OU Social Science Reference Librarian administered the Internet presentation to another group, and the business professor instructed the control group. The live BI group participated in the study in the Government Documents Collection. The Internet BI group met in Bizzell Library’s Electronic BI Room. The control group met in the business professor’s office. The basic outline of the study involved five steps: ! ! ! ! !
Informed Consent Form Government Document Instruction Sessions flyer Pretest Instruction Posttest/Evaluation.
Upon entering their assigned location, the students reviewed and signed an Informed Consent Form. This alerted the students as to the nature of the study and explained their full rights as participants (bParticipation in this study is voluntary. . . . You may discontinue participation at any time.Q). The use of this form is mandatory for any experiments dealing with human subjects at the University according to the Office of Research Administration’s guidelines. These signed forms were then collected. Next, the students were given a flyer stating that if they did not receive the method of instruction they believed they deserved, they could arrange for a special BI session for a later date. This step was added so no student felt denied a certain level of service, especially the members of the control group who would not be receiving any instruction on the day of the study. In the third step of the study, pretests were administered to each student (see Appendix A). The form of the pretest was one page, single sided. The title of the study was at the top. There was also an admonition to bnot place your name on this form.Q The pretest contained five questions, which centered on basic knowledge of the documents collection. Both BI sessions were designed to teach participants a basic knowledge of the Government Documents Collection. The questions were each multiple choice, with three choices for each question. The questions were designed to be challenging, but not impossible to answer
566
J.M. Wilhite / Journal of Government Information 30 (2004) 561–574
Table 1 Control group, pretest versus posttest Group
Number
Mean
T distribution
Significance
Control (pretest) Control (posttest)
13 11
7.40 6.20
1.05
0.323
with a basic knowledge of the documents collection. Ten minutes were allowed for the students to answer and return this pretest. Pencils were provided to the students. The fourth step was the instruction itself. The Internet group was instructed on how to access the Internet video and how to manipulate it (stop, pause, rewind, and volume). Each session had 45 minutes of training. The control group received no training. Finally, each group was given a posttest/evaluation (see Appendix B). The form of the posttest/evaluation was one sheet of paper, double sided. The only difference in the front page of the posttest was the title at the top of the page and a note on the bottom to bOver Please.Q Due to the complex nature of government document collections, the questions on the pre- and posttest/evaluation were the same. Personal information and the evaluation questions were on the reverse side of the page. The personal information section asked three basic demographic questions (college level, gender, and age). The evaluation section contained four questions designed to determine the usefulness of the mode of instruction each group received. The fifth point asked for personal comments. The page ended with the statement, bThank you for your assistance with this study.Q
4. Participant information On the day of the study, 44 students willingly participated. The subjects were mainly sophomores (53%), with juniors (30%), and seniors (17%) also participating. A slight majority of the subjects were male (53%). The subjects’ ages ranged from the late teens to the early twenties: 19 (18%), 20 (45%), 21 (18%), 22 (10%), and other (9%). Thus, the average subject for this study was a 20-year-old male sophomore.
5. Results The three groups were compared on the basis of pretests scores to posttests scores. The control group’s score was not statistically significant at the 0.05 level (see Table 1). The Table 2 Internet group, pretest versus posttest Group
Number
Mean
Internet (pretest) Internet (posttest)
18 16
11.6 12.2
T distribution 0.283
Significance 0.784
J.M. Wilhite / Journal of Government Information 30 (2004) 561–574
567
Table 3 Live group, pretest versus posttest Group
Number
Mean
Live (pretest) Live (posttest)
13 13
7.60 11.6
T distribution 2.30
Significance 0.050
Internet group’s pre- and posttests were not significantly different (see Table 2). This group did have an unexpectedly high mean in their posttest, something the other two groups did not show. There was a statistically significant difference in the live group’s pre- and posttest (see Table 3). The posttest mean results were unexpected. The Internet group’s posttest score was significantly higher than the posttest of the control group (see Table 4). This was also the case with the live group’s posttest score, as it was also significantly higher than the posttest of the control group (see Table 5). There was no statistically significant difference between the posttest of the Internet group and the posttest of the live group (see Table 6).
6. Evaluation The evaluation section of the posttest contained four general questions and a section for comments. Unlike the statistical data produced, the evaluations and comments varied considerably from the live group to the Internet group. Question 1 (see Table 7) asked, bDid you find this library instruction session helpful? Q In the live group (n = 13), 93% of respondents answered byesQ to this question, while only 7% agreed it was bsomewhatQ helpful. In the Internet group (n = 16), 50% of respondents answered bsomewhatQ to this question, while only 25% answered byes,Q and 25% answered bno.Q According to these responses, the live group was more satisfied with their instruction session than the Internet group. Question 2 (see Table 8) asked, bWas the information presented to you in a clear and understandable fashion?Q One hundred percent of the live group answered byesQ to this question. In the Internet group, only 33% answered byesQ while 59% answered bsomewhatQ and 8% answered bno.Q The live group responded more positively with regard to their BI session than did the Internet group. Question 3 (see Table 9) asked, bHow did you find the length of this session?Q In the live group, 93% of respondents answered that the session was the bright length,Q whereas 7% thought it was btoo long.Q In the Internet group, 83% of respondents thought the session was Table 4 Comparison of control and Internet group posttest Group
Number
Mean
Control (posttest) Internet (posttest)
11 16
6.20 12.2
T distribution 3.36
Significance 0.010
568
J.M. Wilhite / Journal of Government Information 30 (2004) 561–574
Table 5 Comparison of control and live group posttest Group
Number
Mean
Control (posttest) Live (posttest)
11 13
6.20 11.6
T distribution 5.69
Significance 0.000
btoo long,Q with only 17% answering that the session was the bright length.Q In a third instance, the live group was again more satisfied with their BI session when compared to the Internet group. Question 4 (see Table 10) asked, bWhich method of library instruction session would you have preferred to take for this session?Q In the live group, 93% responded blive classQ while 7% answered bindependent study.Q In the Internet group, 42% of respondents noted bInternetQ as their preferred method of BI, while 33% noted bindependent studyQ and 25% noted blive class.Q The live group clearly approved of their live BI class and chose it over any other form of instruction. The Internet group was scattered in its choices for BI, with each form of BI receiving a smattering of approval. There was no outstanding statistical preference.
7. Analysis The last section of the posttest/evaluation (see Table 11) asked for comments to questions such as, bWhat was the best or worst part of this library instruction session; What did you wish you had been presented that you did not see,Q etc. A majority of freshmen at OU participate in a live general library training session, so most participants of this study would have had previous experience with a library BI session on the University level. Interestingly, the comments were perfectly divided between the two groups, with 100% of the live group’s comments (n = 5) being positive and 100% of the Internet group’s comments (n = 6) being negative. The positive comments of the live group included bpresentation was good,Q bvery informative and helpful,Q and bI think a live class is better so that you are able to ask questions.Q These questions reflected the live group’s general satisfaction with their BI session as exemplified in the previous evaluation questions. Conversely, the Internet groups’ comments were entirely negative. Whereas most comments focused on issues of technology, two comments did focus on the material presented, bWorst: too long,Q and bToo much detail. . . . If the student wants the extra details then they will ask.Q It is notable that comments about the length were registered from the Table 6 Comparison of Internet and live group posttest Group
Number
Mean
T distribution
Significance
Internet (posttest) Live (posttest)
16 13
12.2 11.6
0.325
0.753
J.M. Wilhite / Journal of Government Information 30 (2004) 561–574
569
Table 7 Question 1: Did you find this library instruction session helpful? Answer
Live group (%)
Internet group (%)
Yes Somewhat No
93 7 –
25 50 25
Internet group and not the live group, as both sessions were timed to be the same length. The Internet group also got to sit through their entire session, while the live group had to walk through the Documents Collection. This level of physical output on the subject’s part might contribute to a negative reaction about the length of the session; however, the nonphysically exerting Internet group responded negatively about the length of their session. The interaction of a live speaker may have offset the negativism of the live group’s necessary physical output, whereas the lack of a live speaker may have contributed to the negative responses of the nonphysically asserting Internet group. A majority of the negative responses from the Internet group centered on the technology with which they were faced. Comments included, bIt was really hard to watch this over the Internet in a classroom with everyone else watching the same thing, it made it hard to focus,Q bCouldn’t understand because of all the other computers,Q and bToo many computers showing movie at once.Q These comments focused on issues of concern when preparing for the Internet BI session. The first concern was that the Electronic BI Room had only 15 PCs. This was about two-thirds the number of computers devised for the study to allow individual student assessment of the BI Internet video. As this was the only Electronic BI Room available, it was anticipated that some students would need to double-up to accommodate the lack of equipment. The second concern of the room was having 15 BI sessions playing at the same time. Headphones were dismissed as an option after testing all 15 PCs synchronously playing the Internet video; the noise was not deemed sufficient to interrupt normal concentration. What had not been tested was the noise level of 18 college students in the room, most of whom were 20-year-old male sophomores. The live group was a quiet and attentive audience. The Internet group was loud and talkative. This certainly needs to be factored into the study results. In replicating this study, headphones should be provided to each Internet subject. Overall, the entire Internet group’s comments were negative, be it because of the material presented or problems with the technology involved in the study. This, compared with the entirely positive comments of the Table 8 Question 2: Was the information presented to you in a clear and understandable fashion? Answer
Live group (%)
Internet group (%)
Yes Somewhat No
100 – –
33 59 8
570
J.M. Wilhite / Journal of Government Information 30 (2004) 561–574
Table 9 Question 3: How did you find the length of this session? Answer
Live group (%)
Internet group (%)
Too short Right length Too long
– 93 7
– 17 83
live group, again draws a stark comparison between the two. Even with the added information from the Internet group, the live group still appears to be more completely satisfied with the form of BI they received.
8. Discussion The statistical results of this study essentially duplicated the findings of the previous Internet versus live reference studies. The Internet and live groups both improved after instruction, and both scored significantly higher than the control group. The live group improved significantly, whereas the Internet group did not. Given this finding, it was surprising that the live versus Internet group, when compared with one another, did not show more statistical significance. The results of this study showed little difference between offering live or electronic BI for government documents. This result was surprising, as the author assumed government documents BI, unlike general reference BI, is complicated and intensive enough to require a live BI session. According to the statistical results, government documents appeared to fare as well as reference in the electronic BI medium. When the evaluation and comments are factored into this, the final conclusion changes slightly. Government documents BI may be able to be presented to subjects in a live or Internet format and produce essentially the same results, but the human response to each of these BI methods may be dramatically different. According to the evaluation and comments, live government documents BI rated more positively than Internet government documents BI. This result needs to be tempered with the added technological concern of the Internet group, specifically the noise level in the room where their section of the study was conducted. On the whole, while it does appears that the subjects fared statistically as well in an Internet BI environment as in a live BI environment, the subjects expressed greater satisfaction with a live BI class than an Internet class for government documents instruction.
Table 10 Question 4: Which method of library instruction session would you have preferred to take for this session? Answer
Live group (%)
Internet group (%)
Internet Live class Independent study
– 93 7
42 25 33
J.M. Wilhite / Journal of Government Information 30 (2004) 561–574
571
Table 11 Comments Answer
Live group (%)
Internet group (%)
Positive N/A Negative
100 – –
– – 100
Government documents librarians should heed the results of this study when considering a decision to present live BI or electronic BI. Both librarians and library administrators need to consider the results of this study before funds are allotted and staff time delegated to create an electronic BI program. The students may learn as much in electronic BI as they do in live BI, but they certainly do not appear to respond as positively to it as to an in-person presentation explaining government documents. 9. Conclusion The University of Oklahoma Government Documents Collection continues to use the Internet BI session in a limited capacity. The link to the Internet BI session on the Web page remains for general patron use and for some student employee training, but the author provides live BI sessions for all classes coming into the documents collection. Live BI classes are informed about the Internet BI session so they can utilize it for review after their live session. In this age of dropping library statistics and 60% of current government publications going directly to the Internet,18 a decision was made to give the patrons what they apparently preferred: human contact and experience of a librarian leading a bibliographic instruction session introducing the Government Documents Collection. Appendix A. Pretest For the study: Internet versus live: Assessment of government documents bibliographic instruction March 15, 2001 Circle the most appropriate answer DO NOT PUT YOUR NAME ON THIS FORM 1. The Government Documents Collection is what percentage of a federal depository? a. Almost 80% b. Almost 90% c. 100% 2. The Government Documents Collection contains items in the following formats: a. Paper only b. Paper and electronic c. Paper, electronics, and others
572
J.M. Wilhite / Journal of Government Information 30 (2004) 561–574
3. Government Document items are not accessible on the Main Library Electronic Catalog Web page: a. Yes b. Some are available, others are not c. No 4. Congressional Universe is an Internet database that allows indexing access and some full-text items to: a. Information about Congress, 1790s to present b. Information about Congress, 1970s to present c. Information about Congress, 1990s to present 5. The Government Documents Collection contains what level of government information: a. Federal, State of Oklahoma, and some International b. Federal and State of Oklahoma c. Federal and some International
Appendix B. Posttest and evaluation For the study: Internet versus live: Assessment of government documents bibliographic instruction March 15, 2001 Circle the most appropriate answer DO NOT PUT YOUR NAME ON THIS FORM 1. The Government Documents Collection is what percentage of a federal depository? a. Almost 80% b. Almost 90% c. 100% 2. The Government Documents Collection contains items in the following formats: a. Paper only b. Paper and electronic c. Paper, electronics, and others 3. Government Document items are not accessible on the Main Library Electronic Catalog Web page: a. Yes b. Some are available, others are not c. No 4. Congressional Universe is an Internet database that allows indexing access and some full-text items to: a. Information about Congress, 1790s to present b. Information about Congress, 1970s to present c. Information about Congress, 1990s to present 5. The Government Documents Collection contains what level of government information: a. Federal, State of Oklahoma, and some International b. Federal and State of Oklahoma c. Federal and some International
PERSONAL INFORMATION (optional) COLLEGE LEVEL: GENDER: AGE:
Freshman
Sophomore
Male ________ (please note)
Female
Junior
Senior
J.M. Wilhite / Journal of Government Information 30 (2004) 561–574
573
EVALUATION 1. Did you find this library instruction session helpful? YES SOMEWHAT NO 2. Was the information presented to you in a clear and understandable fashion? YES SOMEWHAT NO 3. How did you find the length of this session: TOO SHORT RIGHT LENGTH TOO LONG 4. Which method of library instruction session would you have preferred to take for this session? INTERNET LIVE CLASS INDEPENDENT STUDY 5. COMMENTS (e.g., What was the best or worst part of this library instruction session; What did you wish you had been presented that you did not see, etc. . .) Thank you for your assistance with this study.
Notes 1. Tobin, T. and Kesselman, Martin A. (2000). Evaluation of Web-based library instruction programs. INSPEL, 34(2), 67. 2. As defined by the ranking system of U.S. News and World Report Online. (2001, September). Available: http://www.usnews.com/usnews/edu/college/directory/drglance_3184.htm 3. University of Oklahoma Government Documents Web Page: http://libraries.ou.edu/depts/govdocs/ 4. Crowther, Karmen N.T., & Wallace, Alan (2001). Delivering video-stream library orientation on the Web. C&RL News, 62(3), 280. 5. Ardis, Susan (1998). Creating Internet-based tutorials. Information Outlook, 2(10), 17. 6. Kocour, Bruce G. (2000). Using Web-based tutorials to enhance library instruction. College and Undergraduate Libraries, 7(1), 50. 7. Getty, Nancy, Burd, Barbara, & Burns, K. Sarah (2000). Using courseware to deliver library instruction via the Web: Four examples. Reference Services Review, 28(4), 352. 8. Fowler, Clara S., & Dupis, Elizabeth A. (2000). What have we done? TILT’s impact on our instruction program. Reference Services Review, 28(4), 343. 9. Dewald, Nancy H. (1999). Web-based library instruction: What is good pedagogy? Information Technologies and Libraries, 18(1), 26. 10. Murphy, Deborah A. (1998). Building the library starter kit: Developing new technologies for instruction. Reference Services Review, 26(3–4), 111. 11. Dewald, Nancy H. (1999). Web-based library instruction: What is good pedagogy? Information Technologies and Libraries, 18(1). 12. Ardis, pp. 19–20. 13. Johnson, Anna Marie (1998). Too many students, too little time: Creating and implementing a self-paced, interactive computer tutorial for the libraries’ online catalog. Research Strategies, 16(4), 274 – 276. 14. Johnson, p. 279. 15. Lederer, Naomi (2000). New form(at): Using the Web to teach research and critical thinking skills. Reference Services Journal, 28(2), 130. 16. Samson, Sue (2000). What and when do they know? Web-based assessment. Reference Services Review, 28(4). 17. Holman, Lucy (2000). A comparison of computer-assisted instruction and classroom bibliographic instruction. Reference and User Services Quarterly, 40(1), 53. 18. U.S. Government Printing Office (2001). Library programs service update, spring 2001 depository library council meeting. Administrative Notes: Newsletter of the Federal Depository Library Program, 22(6), 9.
574
J.M. Wilhite / Journal of Government Information 30 (2004) 561–574
Jeffrey M. Wilhite is an Associate Professor of Bibliography, Government Documents Reference Librarian, for the University of Oklahoma. He has held his current position since 1995 and was tenured in June 2001. He manages a collection of 2.6 million U.S. Federal, State of Oklahoma, and International documents. The OU documents collection was established in 1893 and is one of the oldest and largest government depository libraries in the state of Oklahoma. He has served as an adjunct professor in the University of Oklahoma School of Library and Information Studies, teaching the Government Publications class. He is currently working on the book, A Chronology of Librarianship, 1960–2000, for Scarecrow Press.