Reference service to international students: A field stimulation research study

Reference Service to International Students: A Field Stimulation Research Study by Ann Curry and Deborah Copeman Available online 15 July 2005

This study analyzed checklist and narrative reports from twenty reference interactions initiated in eleven Greater Vancouver colleges/ universities by a proxy with heavily accented English. The results showed that librarians were generally sensitive to the language barrier, but they failed to ask enough questions and rushed the interaction to premature closure.

Ann Curry is Associate Professor and Chair of Doctoral Studies, School of Library, Archival, and Information Studies, The University of British Columbia, Vancouver, BC, Canada V6T 1Z3 [email protected]; Deborah Copeman is Information Specialist, British Columbia Securities Commission, Vancouver, BC, Canada V7Y 1L2 [email protected].

The Journal of Academic Librarianship, Volume 31, Number 5, pages 409–420

INTRODUCTION Scenario: After the international student wastes several hours trying to decipher the mysteries of an unfamiliar library system and another hour gathering her courage, she approaches the college library reference desk and asks her question in the best English she can muster. What will happen? How will she feel about the encounter? Can the librarian satisfy the information needs of this desperate user who very much needs the librarian’s help if she is to succeed? This research study was prompted by international students in the University of British Columbia MLIS program relating personal scenarios, similar to the one above, that occurred while they were undergraduates, and their questioning of the reference service they received— good and bad. The encounter that occurs when a library user with a query approaches a reference librarian is a multi-faceted process during which a complex interaction between user and librarian occurs.1 Through the possible obstacles of vague or uncertain questions, half-remembered or partially understood academic assignments, and misconceptions about the type of sources that would provide needed information, a reference librarian applies professional communication skills to determine the nature of the user’s information needs. Determining any library user’s information needs can be difficult; the problem is compounded with international students, who bring to the reference encounter their own cultural background and social values that influence the students’ expectation of library services, and who engage in the reference process using a language in which they often lack fluency. The number of international students attending institutions of higher education in North America has increased in the last twenty-five years and the number will continue to rise in the future, bringing new challenges to academic reference librarians. Specifically, the number of international students in the United States rose from 311,880 in 1980 to 407,530 in 1990, and to 547,867 in the 2000–2001 academic year, according to the National Center for Education Statistics.2 Prior to 1970, international students studying in the United States came primarily from European countries where the language, culture, and economy may be similar to that of the United States. Now international students come from more than 180 countries, where for most, English is neither an official language nor a medium of instruction.3 As a result, international students face

September 2005 409

potentially difficult adjustments to new cultural, linguistic, and academic environments, including that of the academic library. Canadian universities and colleges also enroll significant numbers of international and non-native English-speaking students for whom effective library service is just as important as for their counterparts in the United States. Foreign student enrollment in Canadian universities and colleges has risen dramatically, particularly in the last ten years: from 24,194 in 1980 to 31,005 in 1990, more than doubling to 76,210 in 2001.4 British Columbia has become a particularly important destination for international students, attracting 30 percent of Canada’s total in 2001, second only to Ontario (with 36 percent) as a location choice. Three-quarters of those bound for a university or college in British Columbia during 2001 studied at an institution in the Vancouver metro area; the majority of these students were from South Korea, China, Japan, and Taiwan.5 In addition to students, Vancouver is also a destination of choice for many immigrants to Canada, particularly from East and South Asia, resulting in the phenomenon that more than 56 percent of children in Vancouver schools speak a language other than English at home.6 Although these young new Canadians benefit from several years in public school system ESL (English as a Second Language) classes and have become tangentially familiar with the cultural life of Canada as they work diligently toward high-school graduation or college entrance, many still lack fluency in English and cultural integration into Canadian society. These non-native Englishspeaking Canadian students represent another group facing barriers to effective use of library resources and services, including reference services. Despite increasing numbers of international and non-native English-speaking students in both Canada and the United States, particularly through the 1990s, it appears that improving library service to these special groups through such activities as providing training in cross-cultural communication or establishing an international student liaison librarian position has not been a high priority.7,8

LITERATURE REVIEW Research-Based Articles While the literature is rich in articles that discuss the barriers affecting international students’ successful use of the academic library, very few (and no Canadian studies) focus on international students’ interactions at the reference desk and the quality of reference service they experience. Most researchers have focused on library orientation and bibliographic instruction.9 Four dissertations that investigated aspects of international students’ experiences using American libraries during the period 1988 to 1992 were identified, but none focused primarily on reference services.10 Only one Canadian study, which focused on orientation programs for international students at Canadian academic libraries, appeared to be relevant to the current study as it included frequency of staff intercultural training and presence or absence of a liaison librarian for international students as variables.11 Research done by Anthony J. Onwuegbuzie and Qun G. Jiao (1997), Sara Baron and Alexia Strout-Dapaz (2001), and Ziming Liu (1993) confirms that international students face challenges when they use American academic libraries.12 In their comparative study of library usage patterns and anxiety

410 The Journal of Academic Librarianship

levels of 552 native and non-native English-speaking university students, Onwuegbuzie and Jiao used two data-gathering instruments: the Demographic Information Form (DIF) and the Library Anxiety Scale (LAS). The study found that non-native English speaking students reported higher levels of anxiety due to barriers associated with library staff interactions, along with affective barriers (e.g., feelings of inadequacy and/or incompetence) and mechanical barriers (e.g., difficulties using library technology, such as computers, fiche readers, photocopiers), even though these students had taken more library instruction and computer skills courses than the American students. Baron and Strout-Dapaz mailed surveys to the head of reference services and to the director of the international student office of 300 academic member libraries of TexShare, a Texas library consortium, and Amigos, a multi-state consortium (600 surveys in total). Their research reveals that three major challenges international students face are language and communication problems, adjusting to a new educational and library system, and general cultural adjustments. The research also recommends that reference and instructional services to help international students be made more effective through staff diversity training and collaboration with campus departments and international students themselves. In her study of international students’ difficulties and coping mechanisms in American libraries, Ziming Liu interviewed fifty-four foreign students at the University of California, Berkeley. Liu concluded that international students have problems using American academic libraries due to insufficient English proficiency, unfamiliarity with American library systems, and confusion when too many results are retrieved. Descriptive Articles: Barriers to Effective Reference Service The non-research literature is comprised mainly of descriptive studies, written by reference librarians, that discuss barriers that impede effective delivery of reference services and suggest guidelines or solutions to overcome these obstacles and provide effective reference services. There is general agreement that the barriers to effective reference service in the academic libraries include language, conceptual awareness of library services, and culture. Language Language barriers–the major obstacle for international students–usually result from the reference librarian’s inability to understand students because of variations in pronunciation, intonation, speech patterns, and enunciation. The barrier may also result from international students’ inability to express their information need correctly.13 Mengxiong Liu, in a study of ethnicity and information seeking, notes that insufficient English proficiency and vocabulary hamper international students when they try to understand library terminologies and use libraries effectively.14 Christopher C. Brown explored the same area when he considered the language barriers that impede effective delivery of reference service from six aspects: accent, lexicon, syntax, sociolinguistics, language as nationalism, and transcription systems.15 Conceptual Awareness of Library Services Several authors discuss international students’ lack of conceptual awareness of library services. Kumar and Suresh

found that the reference librarian role is unclear to most international students.16 Because they do not know that a reference librarian’s primary responsibility is to answer questions, international students feel they are troubling the librarian with their queries. A lack of awareness regarding self-service procedures is another major obstacle when international students attempt to navigate through North American information systems. Several authors point out that commonplace features of North American libraries such as open stacks, interlibrary loan, online service, and even reference service may not exist in foreign libraries.17 In addition, students from other countries may be unfamiliar with North American classification systems and subject headings. Cultural Barriers The information needs and information-seeking behaviors of international students also differ from those of North American-born library users due to different cultural backgrounds. Body language, eye contact, and even the meanings of specific words vary among cultures. The personal space that white North Americans defend when they converse is not the norm in many other cultures, and the influence of gender and social status on communication between individuals differs widely among cultures. Liestman points out that these differences and others may leave an interaction in tatters although it has appeared that both parties have good English communication skills and solid cultural understanding.18 Brown also examines cultural barriers. He focuses on two aspects: general cultural problems—including such elements as preference for formal or informal and high-context or low-context communication styles, and library-specific problems—such as different attitudes toward intellectual property rights and proper user/ library staff relationship.19 Other areas of cultural difference that may limit reference interview success are learning styles and behaviors. Both the American and Canadian educational environments reward students for participation, questioning, and assertiveness in pursuit of independent research and original, creative work. In contrast, Wayman notes that the students in other educational systems, such as those of Japan, China, and the Middle East, are usually verbally passive, as they learn by observation and imitation and may respond only to direct questions.20 These dissimilar learning styles may hinder communication between librarians and international students when the librarian attempts to facilitate learning. Overcoming Barriers to Effective Communication Designing strategies to provide effective reference service for international students requires a considerable investment of thought and effort. Sarkodie-Mensah suggests that reference librarians become more aware of international students’ speech patterns, listen to their words rather than intonation, and that librarians be cautiously tactful in what they ask and say.21 To elicit more complete information about international students’ needs, de Souza advises reference librarians to use neutral questions that are open-ended yet structured.22 Mengxiong Liu cites the guidelines offered by Joan Ormondroyd for effective communication with international students, which include maintaining simple sentence structure and vocabulary, speaking clearly and slowly, rephrasing, and honing listening skills and sensitivity.23

Effective Reference Service Although librarians can formulate criteria that characterize effective reference service, evaluation or measurement of that effectiveness is difficult. In 1996, Reference and User Services Association (RUSA), a division of the American Library Association, proposed broad guidelines for effective reference service that recommended five evaluation criteria: approachability, interest, listening/inquiring, searching, and follow-up.24 The RUSA guidelines emphasize that effective service is not measured entirely by an answer’s degree of accuracy, but through the interaction between the librarian and user. This point of view regarding effective service is supported by Mendelsohn who discovered that quality reference involves a relationship between the user and librarian within a ‘‘Cycle of Dimension of Service’’: willingness (to assist user); knowledge (how to assist user); assessment (of user’s need), and action (physically moving with the user, getting up from behind the desk).25 According to Mendelsohn, this Cycle assists in meeting the user’s needs, teaches a research process, facilitates user independence, and ultimately results in quality reference service. Researchers employing unobtrusive testing to evaluate reference services have incorporated measures focusing on the librarian/user interaction. In their user-centered, unobtrusive study of reference services, Dewdney and Ross tested willingness of the inquirer to return to the same staff member at a later date as a key measure of reference effectiveness.26 In their study, student proxies formulated an individual reference question important to them and posed the question at an academic or public library desk. The students then rated service effectiveness in terms of friendliness, understanding, helpfulness of answer, and overall satisfaction. Dewdney and Ross analyzed the proxies’ questionnaires and their descriptive accounts to uncover most and least helpful aspects of service, and occurrence and effectiveness of the reference interview, referral, and followup. In their study, and in a follow-up study carried out five years later, the problems identified in unsuccessful reference interactions were the same: failure to conduct a reference interview, failure to ask follow-up questions, and unmonitored referrals.27 Two additional problematic librarian behaviors identified in Dewdney and Ross’ studies were using systems-based questions rather than questions designed to clarify the user’s information need, and resorting to ‘‘negative closures’’—various strategies the librarian engages in simply to get rid of the user.28

PURPOSE The purpose of this research project was to investigate and evaluate the quality of reference service that international students experience at academic libraries in the Lower Mainland of British Columbia, a geographic area roughly equal to Greater Vancouver with an overall population of 2.5 million, and a very large concentration of international and Canadian students whose first language is not English. With this limited data set and geographic area, and the limitations (but also substantial advantages) of the methodology chosen, it is acknowledged that the results cannot be generalized to the academic library community in Greater Vancouver, nor in Canada. However, it was hoped that the research would uncover data that would inform librarians working with

September 2005 411

international students and provide the foundation for a larger study. Research Questions: 1. In light of the language, conceptual, and cultural barriers revealed in the literature, did the reference librarians in this study employ effective communication strategies such as rephrasing the user’s question, avoiding or explaining library jargon, and speaking slowly and clearly? 2. Since non-native English speakers had been found to experience greater library anxiety associated with barriers created or exacerbated by staff members, did the reference librarians in the study listen carefully and show patience when an international student had difficulty expressing herself in English? 3. Were the reference librarians in the study perceived as approachable, and did they engage in practices recommended for any and all reference interactions: clarifying the information need through questioning, explaining processes, accompanying the user, and following up to ensure the user’s question was answered? 4. Are there barriers to international student/reference librarian communication additional to those identified in the literature? 5. What strategies can be recommended for overcoming communication barriers?

METHODOLOGY Field Stimulation The methodology used was field stimulation—a form of structured observation recommended for data gathering in situations where other techniques such as questionnaires or scheduled interviews will likely yield biased results.29 Projects using this methodology must be carried out under strict ethical conditions because the subjects do not give personal informed consent. In Research Methods in the Social and Behavioral Sciences, Jones sets out the conditions under which field stimulations (sometimes called unobtrusive testing) should be carried out: (1) the research occurs in a public place; (2) no possibility of identifying information is recorded; (3) the interactions take little time or effort by the subject; and (4) tasks required of the subject are within the range of the subject’s normal experience.30 The current project met all four of Jones’ conditions and was approved by the University of British Columbia’s Behavioural Research Ethics Board. In this field stimulation, an MLIS student who belongs to a visible minority, whose first language is not English, and who speaks strongly accented English with some articulation difficulties asked the same reference question on two separate occasions at reference desks in eleven postsecondary academic libraries in the Lower Mainland area of British Columbia (twenty-two reference interactions in total).

412 The Journal of Academic Librarianship

The student’s verbal abilities were such that clarification of the information need, patience, and adjustment of language used by library staff members would be highly desirable features of the reference interaction. The institutions included both universities and colleges, ranging in size from The University of British Columbia with 40,000 students to Columbia College with 600 students. Each institution supported at minimum a two-year general undergraduate university transfer program. The chief librarians of all the institutions supported the project and granted permission for the student to ask the question. They were given the option of informing or not informing their staff in a very general manner about the research project, and three decided to do so. They told their reference staff that a ‘‘mystery’’ student might approach the desk sometime in the next two months and ask a ‘‘normal’’ question. No staff member voiced objections. The researchers felt that the MLIS background of this female international volunteer proxy was an advantage, as it made her more confident and comfortable visiting the various libraries, heightened her powers of observation regarding the librarians’ reactions, and increased her sensitivity to jargon used in conversation. Her pseudonym for this article will be Chin Ping. The same person, Chin Ping, was used for all the field stimulations, due to the many variables introduced by using multiple international students as proxies—differences in physical appearance, manner, accent, and level of English proficiency would tend to influence the outcome of the interaction. In addition, it was preferable to use a library and information studies student who would not be unnerved by the variety of library settings and therefore could concentrate more intensely on the interview dynamics, and who would fully comprehend what we wished to evaluate. It is acknowledged, however, that the limited amount of data collected through this single-proxy approach must be seen as preliminary and suggestive, and that a larger study employing numerous proxies is a desirable next step. The study was carried out over a three-week period in November of 2003. Chin Ping visited every library between 9:00 am and 5:00 pm, when experience had shown that a professional librarian would most likely be staffing the reference desk. Although the reference interaction was relatively brief, each visit was scheduled for a half-day, due to travel time from UBC (over one hour for most institutions), the time Chin Ping required to record data immediately after the interaction, and the need for ‘‘downtime’’ so that Chin Ping could be intensely alert for each encounter. To achieve greater breadth of results, each library was visited twice, on different weekdays, and at different times of day, so that Chin Ping would encounter different staff members. She asked the same initial question of each staff member: ‘‘I am going to write a paper on immigration. Can you help me to find some information on the topic?’’ If prompted to provide more information, Chin Ping said, ‘‘I’d like to focus on the immigration policy related to Canada’s economics.’’ She was interested in finding journal articles and books on the topic, and if asked, stated that the paper was for a Political Science class. Chin Ping approached the person who appeared (in her best judgment) to be a professional librarian, but if a nonprofessional staff dealt with the query, Chin Ping still recorded

the interaction, as it fulfilled the spirit of the research and reflected the ‘‘real life’’ service an international student would receive. All reference desks in the libraries visited are staffed with professional librarians, and Chin Ping believed that all her interactions were indeed with professionals, but this could not be verified without breaking the anonymity requirement of the project. Three of the chief librarians had warned us that it was standard for their library staff to ask if a client was a student of that institution, and that if the proxy was discovered not to be a student, the proxy might receive less extensive help or be refused service by the librarian. This did occur during two visits: Chin Ping was refused service in accordance with library policy (a policy developed due to an exceedingly high usage of reference services by students of nearby institutions, which taxed library resources). She was prepared for this possibility, so the refusal did not come as an unpleasant surprise. As a result, we were able to analyze data from twenty interactions. Data Gathering Instrument The data collection instrument (the Reference Interaction Record) which Chin Ping filled out immediately following her reference desk visit consisted of three sections (see Appendix A). The first section was a checklist that contained seventeen questions in which Chin Ping noted the presence or absence of positive interpersonal behaviors, of recommended reference interview practices, and of communication strategies that demonstrated an awareness of possible language barriers. Many sources informed the development of these questions: studies investigating ethnicity, information seeking, and barriers to effective communication experienced by international students;31 questionnaires and checklists of positive and negative staff behaviors resulting from previous field stimulation studies of reference service;32 and the Reference and User Services Association (RUSA) Guidelines for effective reference staff performance.33 In the second section, Chin Ping described the entire reference interaction in a detailed, step-by-step narrative, a technique used successfully in similar previous studies.34 In the last section, Chin Ping recorded the amount of time the encounter had taken, rated her satisfaction with the answer, and rated her willingness to return to the same staff member, variables also used in previous projects.35

CHECKLIST RESULTS Overall, the librarians in this study performed well: Chin Ping indicated that in 75 percent (15/20) of the cases she was satisfied or very satisfied with the answer received and was likely to or would definitely return to that staff member in the future. Based on these variables of satisfaction and likeliness to return, the cases were separated into three groups labeled as Category 1 (seven cases), Category 2 (eight cases), and Category 3 (five cases) because the reference interviews in each group appeared to have many common characteristics. In nineteen cases, level of satisfaction dovetailed exactly with level of willingness to return to the same librarian, a finding reported in previous studies.36 In only one case did Chin Ping indicate satisfaction with the answer received but note she was not likely to return to the same librarian in the future (Table 1). The reference interactions ranged in length from just a few minutes to almost half an hour, with most of them lasting from

Table 1 User Satisfaction and Willingness to Return to the Same Librarian Number of Interactions

Category

Very satisfied/definitely would return

7

1

Satisfied/likely to return

8

2

Satisfied/not likely to return

1

3

Dissatisfied/not likely to return

3

3

Very dissatisfied/definitely would not return

1

3

Chin Ping’s Evaluation

five to fifteen minutes. No clear correlation could be established between length of interaction and satisfaction/ willingness to return, as the most highly rated interactions ranged from five to more than twenty minutes. However, all three interactions that lasted less than five minutes were rated as Category 3 (low satisfaction/would not return). Reference Service Actions In a checklist of ten questions, Chin Ping recorded the positive presence or negative absence of actions or behaviors recommended in every reference encounter: approachability, asking questions, explaining his/her actions, providing instruction, avoiding information overload, respecting the question and questioner, providing direction to sources, referring, asking if sufficient information was found, and giving an invitation to return for more help. In most cases, Chin Ping rated the librarian as approachable (15/20) and reported that the librarian asked questions (20/20), explained his/her actions (17/20), provided instruction (18/20), respected her and her question (16/20), and avoided overwhelming her with information (19/20). In the one instance where Chin Ping felt overwhelmed, the information overload did not have a negative impact on her overall rating of the experience: she was very satisfied with the encounter. An important part of the reference interview is the ‘‘closing,’’ which should include an invitation to return for additional help. However, in only six instances did the librarians invite Chin Ping to return. The importance of this element, as noted in previous research,37 is demonstrated by the fact that Chin Ping gave high ‘‘satisfaction’’ and ‘‘willingness to return’’ scores to all six of the librarians who extended this invitation (Table 2). Awareness of Language Barriers In a checklist of six questions, Chin Ping recorded whether the librarian exhibited actions or behaviors specifically recommended for a reference encounter with a non-native English speaking library user–listening, patience, rephrasing, speaking clearly, and avoiding library jargon–and her overall perception of the librarian’s awareness of language barriers (Table 3). Chin Ping recorded that the librarians listened carefully, spoke clearly, and showed patience in almost all cases. In seventeen of twenty cases, however, she marked ‘‘not applicable’’ for the action—‘‘staff member rephrased to clarify

September 2005 413

Table 2 Reference Service Actions Actions According to Chin Ping, the Reference Librarian:

Number of Interactions Yes

No

N/A

15

2

3

Asked questions for clarification

20

0

0

Explained what he/she was doing at every stage

17

3

0

Provided instruction on how to use information sources

18

2

0

1

19

0

Had a respectful attitude toward Chin Ping and her question

! advice and suggestions

a

Looked approachable

Overwhelmed Chin Ping with information

average of 500 words in length. Seventeen themes emerged from the data, the majority of them concepts or behaviors noted previously in the research literature as important factors that determine reference interview quality:

! approachability ! follow-up—invitation to return, asking if question answered ! asking questions ! attentiveness ! awareness of language barriers ! early termination of interview

16

4

0

! explanation/instruction

Accompanied Chin Ping to information sources (to a print source or public access computer terminal)

9

9

2b

! library jargon

Referred Chin Ping to someone else

2

18

0

! non-verbal communication

Asked if Chin Ping had found what she needed

4

16

0

! patience

Invited Chin Ping to return if she needed more help

6

14

0

Notes:

a

! listening skills

Librarian was roving/helping users; reference desk was empty. problems prevented librarian from demonstrating database usage.

b Server

! referral ! repeating and rephrasing ! resourcefulness ! showing interest in proxy’s query ! speaking clearly

a misunderstanding.’’ When asked to shed light on this result, Chin Ping said that the librarians had listened to her so carefully that they had understood what she said in most cases. The two instances in which the librarian did rephrase as a clarification strategy were from the Category 1 (high satisfaction) group, while the one instance in which Chin Ping felt misunderstanding occurred without any attempt by the librarian to clarify through rephrasing came from the Category 3 group. Twelve of the twenty librarians used library jargon in their reference interviews, but this transgression did not affect Chin Ping’s satisfaction with the interaction nor her willingness to return. This lack of a correlation between satisfaction level and avoidance of jargon may suggest that Chin Ping, due to her MLIS training, had a higher tolerance for library jargon and must be acknowledged as a weakness of choosing a librarysavvy proxy. In 80 percent of the cases (16/20), Chin Ping reported that the librarian demonstrated awareness of language barriers and had modified his/her behavior accordingly. The four cases in which Chin Ping recorded that the librarian did not exhibit awareness of language barriers were all from Category 3 (low satisfaction/would not return).

NARRATIVE RESULTS Chin Ping’s ‘‘stream of memory’’ recording of everything she could remember about the interaction provided a wealth of relevant and usable data that the researchers described and analyzed using an open coding process derived from grounded theory.38 The narratives for each library interaction were an

414 The Journal of Academic Librarianship

Correlation between the number of times that nine of these themes appeared within the narrative description of an interaction and the ultimate category into which the interactions fell (1, 2, or 3) revealed a strong relationship between the themes and Chin Ping’s overall satisfaction and willingness to return. The nine key themes were approachability, awareness of language barriers, asking questions, rephrasing, explanation, library jargon, instruction, early termination of interview,

Table 3 Awareness of Language Problems Actions According to Chin Ping, the Reference Librarian:

Number of Interactions Yes

No

N/A

Listened carefully to what she said

19

1

0

Was patient if she paused to reformulate the question

18

2

0

2

1

17a

20

0

0

8

12

0

16

4

0

Rephrased to clarify a misunderstanding Spoke clearly Avoided library jargon Overall, showed awareness of language barriers Note:

a

Indicates from proxy’s perspective the librarian understood what she said in these cases, making rephrasing unnecessary.

patience, and follow-up. Analysis of these themes in Chin Ping’s narrative accounts also furnished us with a richer picture of the reference behaviors and communication skills that could be fitted into the typologies of ‘‘best’’ and ‘‘worst’’ practices developed in the extant research literature. For the remaining themes, either no relationship regarding satisfaction/willing to return could be determined, or the data were not rich enough for analysis. Approachability Reference librarians demonstrated approachability through actions and characteristics that made them appear welcoming, friendly, and immediately ready to begin an exchange when Chin Ping walked up to the reference desk. Narrative related to this theme is divided into three levels, indicating a low, average, or high degree of approachability. Examples: High: ‘‘When I went to the desk, the staff member with a smile and cheerful voice said to me, ‘‘Hi, how can I help?’’ Average: ‘‘When I approached to the [desk], the staff member just finished helping one student and she said dHiT to me first.’’ Low: ‘‘[The staff members behind the desk] saw me, but still continued the discussing and didn’t acknowledge me. I waited for about 1 minute before she said dHiT to me.’’

With only one exception, the interactions in which Chin Ping related high or medium approachability encounters, she also rated highly for overall satisfaction and willingness to return. In ‘‘Flying a Light Aircraft,’’ Dewdney and Ross described aspects of reference service that are most helpful and least helpful, with most helpful behaviors demonstrating a welcoming and respectful attitude to users and their questions. Highly rated librarian actions in the Dewdney/Ross study include positive body language such as making eye contact, smiling and nodding to acknowledge the user’s presence, and overcoming the physical barrier of the reference desk by moving out from behind it. These behaviors are even more critical when providing service to international students as studies have shown international students lack understanding of the role of reference librarian and may be much more hesitant to approach.39 Analysis of Chin Ping’s narrative prose categorized within the theme of ‘‘approachability’’ supports the literature in that librarians who were ready to respond to Chin Ping’s question with a simple greeting such as ‘‘Hi, can I help you?’’ or ‘‘What can I do for you?’’ and who made an effort to move out from behind the desk to accompany her to a computer terminal were perceived in a more positive light than those who failed to do so. In those cases where the librarian remained at the reference desk, the librarians’ use of and failure to disengage from the computer terminal had a negative impact on the development of rapport between the librarian and Chin Ping. This was evident in one of the four Category 3 narratives: when Chin Ping asked her question, the librarian ‘‘continued to look at her screen and said, dWhat kind of . . . just a second. . .T’’ After several moments and after no greeting of any kind, the librarian eventually tore her attention away from her computer terminal and asked, ‘‘What do you want on immigration?’’ This inattentiveness and lack of a general greeting made Chin Ping

feel that she was being pushed away before the reference started. Such a reception would likely exacerbate an international student’s already high levels of library visit-related anxiety. Awareness of Language Barriers Chin Ping concluded while marking her checklist that most of librarians encountered naturally possessed an adequate level of speaking and listening skills, and that in most cases, they were not changing their interaction style because of her communication difficulties. Information from Chin Ping’s narrative, however, revealed some evidence that appeared to indicate the librarian’s awareness or ignorance of a language barrier. Chin Ping wrote that she felt some librarians were willing to spend time over and above the norm, going slowly over the search strategy and information sources found; in contrast, she wrote that other librarians spoke very rapidly or talked ‘‘over’’ her, rather than to her. Awareness of language barriers was one of the more difficult themes to code as it may be based on a perception that the librarian is adjusting his/her listening, speaking, and non-verbal cues to ensure successful communication with the user. These adjustments include strategies such as slowing speech, enunciating, choosing simple words and constructions over complex ones, and using gestures. Although Chin Ping selected ‘‘yes’’ for whether she felt the staff member showed awareness of potential language barriers in 16/20 of the questionnaires, there were comparatively few actual examples in the narratives that described this ‘‘awareness’’ in more detail. However, notwithstanding the lack of rich data, each of the Category 3 (low satisfaction/unlikely to return) visits generated more comments regarding lack of awareness of language barriers than did visits in the other categories. Chin Ping recounted the fast pace of one librarian from a Category 3 encounter: ‘‘I spoke slowly, dI am going to write a paper on immigration. Can you help me find some information on the topic?T She listened to me and asked what specific topic I wanted. She talked very fast and I couldn’t recall exactly what she said. I felt that I was slow to answer, but she kept going quickly.’’ It is encouraging to note, however, that in most interactions the librarians did follow two of Ormondroyd’s guidelines for effective communication with international students—‘‘to articulate clearly and use a slow pace.’’40 Asking Questions The narrative data revealed that librarians asked Chin Ping two types of questions. Librarians used the first type to clarify the question’s context. They used the second type, which Ross41 identifies as ‘‘systems-based’’ questions, to ascertain Chin Ping’s level of familiarity with the library catalog or databases and whether she had home access to the Internet. In a follow-up to her 1994 reference study, Ross finds that one of the main culprits in causing a failed, unsatisfactory reference interview is neglect of the interview process itself, the foundation of which is asking questions.42 In a common scenario within Ross’ study, the librarian silently begins doing some ‘‘mysterious activity’’ such as typing on the keyboard without first asking the user any questions to ascertain and clarify the information need. It appears from this study that the presence and depth of the reference interview did influence

September 2005 415

Chin Ping’s level of satisfaction and willingness to return. In the Category 3 (low satisfaction) narratives, the librarians asked few or no clarifying questions—with the greatest number being three questions during one interaction. This contrasts to the more involved interviews in Category 1 interactions during which the librarians engaged in a more thorough process to clarify Chin Ping’s information need. They asked Chin Ping three to five questions, including subject probes such as ‘‘What specific topic do you want?’’; context probes such as ‘‘What course is this for?’’ ‘‘Have you done any papers like this before?’’ and ‘‘How many articles you need?’’; and systems-based probes such as ‘‘Have you used our databases before?’’ Repeating and Rephrasing The researchers were disappointed to learn from the narrative how few librarians–only six–repeated or rephrased Chin Ping’s question in their own words, with five of these six incidents occurring in Category 1 or 2 interactions. When librarians did rephrase her question with slightly different words, Chin Ping related that they did so carefully and slowly, deliberately applying the technique. Both the RUSA and Ormondroyd guidelines highly recommend that librarians adopt this strategy: RUSA advocates it to confirm understanding of a user’s question in every reference interview, while Ormondroyd, whose guidelines focus on international students, says the technique helps to circumvent misunderstandings caused by difficulties with expression, articulation, or lack of vocabulary. Included in the six incidents is one of written rephrasing in which a librarian wrote down words after Chin Ping asked her question and deliberately showed them to her. This may be considered under the theme of ‘‘Instruction,’’ but Chin Ping felt that the librarian was writing the words just in case she might not understand the librarian’s verbal response: ‘‘She wrote down some keywords on a piece of paper, like dCanada,T dimmigration policy,T deconomy justifications,T and tried to figure out the best search strategy.’’ Overall, Chin Ping rated very highly these acts of rephrasing, repeating, or writing down key words and showing them to her. It should be noted that Chin Ping indicated in the Checklist only three incidents of rephrasing due to her strict interpretation of the category that required rephrasing due to obvious librarian misinterpretation. Explanation and Instruction Chin Ping related instances in which the librarian explained or failed to explain a library service such as inter-library loan, an area of the collection such as reference, or the general content of an information source such as a database. The narrative accounts reveal twenty-three instances of explanation: fourteen of these emerged from the narratives of the Category 1 interactions, while only one came from Category 3 interactions. Incidents of instruction occurred when the librarian demonstrated to Chin Ping or told Chin Ping in a detailed, step-by-step manner (with Chin Ping carrying out the action) how to accomplish a task. Instruction differs from mere explanation in its underlying purpose of helping the user to become more self-reliant with some aspect of library use such as using the catalog or saving, e-mailing, or printing articles. Chin Ping related sixty-six instances of instruction within her

416 The Journal of Academic Librarianship

narratives: fifty-five of those occurred in Category 1 or 2 interactions. Analysis of Chin Ping’s narrative reveals that all librarians in the Category 3 interactions neglected to include Chin Ping in the search process and failed to explain their search actions. Research has shown that users appreciate being included in the reference process through discussion, explanation, and demonstration,43 and Chin Ping’s twenty narratives support this view. Librarians who maintained a running commentary about the steps within and reasons for their actions were rated highly: Chin Ping felt that these librarians expended extra effort to bridge her communication problems. In the best interactions, the librarians explained to Chin Ping what search strategy they were using and why. Four examples from Category 1 or 2 interactions: The librarian said to me dAt this stage we really don’t know how the material is ranged, so I’ll use keyword search.T Then he walked me to the public workstation in front of the desk and instructed me in the subject alphabetical search (it is subject heading search) step-by-step. She explained how to display the full-text and how to find the items without full-text. He showed me how to find the journal article from the library catalogue and newspaper articles from the other databases. He said that’s called navigation from one database to another database for finding the fulltext articles (No librarian had showed this method so far). I thought it was a very good way to find FT articles.

Such dialogue may seem ‘‘routine,’’ but it is noticeably absent in the Category 3 narratives and consistently present in those of Category 1. The narratives also reveal that Chin Ping was particularly appreciative of librarians who offered guidance on how to evaluate and select relevant materials from the many hits that a search produced. In one case the librarian reviewed a list of journal articles, then offered advice on which journals were highly recommended and which articles might best answer Chin Ping’s question. Another librarian in a Category 1 interaction went out of her way to evaluate the journal references in consultation with Chin Ping and then print out several full-text articles for her. In contrast, she often felt lost in the Category 3 interactions where commentary was minimal or non-existent. Examples: She made searches and wrote down the call numbers, she didn’t show me and or explain to me how to do the search. She handed the paper [with call numbers] to me and didn’t tell me where to find these books. She then continued her previous work.

One interesting finding of the analysis was that certain negative aspects of an interview such as a lack of respect, low approachability, and even tactics at early termination in three Category 2 interactions appeared to have been offset by Chin Ping’s appreciation of the search strategy demonstration and the information sources found or recommended. Chin Ping’s personal preferences and the small data set must be acknowledged in this (and all other) observations, but her narrative regarding this theme highlights the importance of providing the user with some level of instruction.

Library Jargon Chin Ping noted thirty-two instances of library jargon (such as ‘‘peer reviewed,’’ ‘‘call number,’’ and ‘‘full-text’’) in her narrative: in seventeen instances the librarian explained the jargon (unasked), while in fifteen cases the word or phrase was unexplained. As noted in section E-2, Chin Ping did not feel that use of library jargon lessened her satisfaction with service, but a student without her knowledge of library operations would likely have been more critical and dissatisfied. While librarians from all three interaction categories used library jargon, Chin Ping’s narratives revealed a noticeable difference between Categories 1 and 3 in frequency of jargon explanation. Although librarians in Category 1 interactions used as much or more jargon, they were far more aware of the need to provide definitions or catalog demonstrations to clarify a term such as ‘‘subject heading,’’ or ‘‘Boolean search.’’ Patience In Chin Ping’s narrative, the librarians demonstrated patience by their willingness to take time to explain a process or resource through step-by-step instruction, and to wait for Chin Ping to ‘‘digest’’ information and give a response. Two examples from her narratives illustrate the presence and absence of patience: Patience present: ‘‘The librarian said dYou can use the same keyword search that I showed you to search. Is that OK?T She waited [for] me to say something.’’ Patience present: ‘‘Then he was very patient and gave me step-by-step instruction on how to use subject search (not keyword search) in the library catalog.’’ Patience absent: ‘‘She asked me, dWhich one you want?T I looked at the list and before I answered her, she wrote down 2 call numbers for me. But those 2 were not the related [relevant] books.’’

dismiss her before fully answering her question. These behaviors were usually coupled with a total lack of concern to check whether Chin Ping found some appropriate sources. None of the eleven instances in which Chin Ping felt rushed or dismissed appeared in the Category 1 interactions (high satisfaction/very willing to return): all of them were part of the Category 2 or 3 narratives. Catherine Ross describes this early termination behavior as ‘‘negative closure.’’45 Negative closure in Chin Ping’s narratives was characterized in eight of the eleven interactions by librarians handing Chin Ping a list of call numbers and telling her to check the book shelves, neglecting to provide further direction or help to find the journal articles she had requested. Chin Ping described one early termination encounter as follows: She said, dCheck those 2 books upstairs and look for these number areas.T I said, dIs there any other information I can have?T She answered, dLook at these books first, OK? For journal articles, it needs in-depth search.T She went to help the next person behind me.

This lack of clarity and user involvement resulted in Chin Ping reluctantly leaving the reference desk with a clear signal from the librarian that the transaction had ended, and that a further effort to search for the requested journal articles was somehow too complex to be explained. According to Ross, negative closure/early termination is likely the single most persistent and remembered aspect that influences a user’s impression of the reference interview.46 As a composite of several negative behaviors, including failure to welcome the user, ask questions, provide sufficient explanation or instruction, or, in the end, fully answer the user’s question, early termination strongly discourages a user from returning to the same librarian. Chin Ping’s narratives on this theme, in which early termination incidents appear only in Category 2 or 3 interactions, certainly support Ross’ findings and highlight this theme as a crucial one in reference service for international students.

CONCLUSION Examples of lack of patience appear in all four Category 3 interactions where Chin Ping said she was unlikely to return, whereas five of the six examples in which the librarian exhibited patience appear in the Category 1 interactions. Follow-up Chin Ping’s narratives revealed fourteen instances (from ten interactions) in which the librarian asked her to come back if she needed more help, but only three instances (from three interactions) in which the librarian asked Chin Ping if her question had been answered. According to the RUSA Guidelines for Behavioral Performance of Reference and Information Services Professionals, both these follow-up procedures are crucial to reference interview success,44 and since these procedures would be of greater importance to an international students, it appears that improvement is needed in this area. Early Termination Chin Ping related eleven instances in which she felt that the librarians rushed through the reference process or attempted to

As noted in the literature review, most research in this area focuses on library orientation and bibliographic instruction, not on reference interactions, so this study is a first step in filling this gap. With the data restricted to twenty interactions by one proxy, the results cannot be extrapolated to the experiences of all international students. As well, the limited replicability of field stimulation studies such as this and the unavoidable influence and bias of the data gatherer (Chin Ping) must be acknowledged. The results must be viewed as exploratory and the emergent themes as topics for discussion by practitioners and future researchers. Overall, the librarians in the study did very well. According to Chin Ping’s checklist, she was satisfied or very satisfied and would likely or definitely return after fifteen of twenty library interactions. Chin Ping felt that three-quarters or more of the librarians were approachable, explained their actions while providing instruction, and showed her respect during the interaction. The librarians excelled in their listening skills: Chin Ping felt that in most cases the librarians gave her their full and undivided attention, and that this concentration facilitated their understanding of her heavily accented English. All of the librarians used some library

September 2005 417

jargon, but in all but a few cases, they also explained the ‘‘mystery’’ terms they used. The librarians also excelled in speaking slowly and clearly: in only one instance did Chin Ping feel overwhelmed by the librarian’s avalanche of verbiage. The importance of speaking slowly and clearly is emphasized in the previously discussed research of Mengxiong Liu and appears to be of paramount importance in this research as well.47 Surprisingly, very few (6/20) librarians rephrased or repeated Chin Ping’s question, an action highly recommended in all reference interviews irrespective of the questioner’s language abilities. Although Chin Ping generally rated the librarians as approachable on her checklist, her narratives revealed that some librarian’s initial contact methods caused her less anxiety than others. The simple act of the librarian speaking first with a friendly greeting such as ‘‘Hi, how can I help you?’’ engendered a much higher comfort level than the librarian just looking up (though with a smile) and waiting for Chin Ping to speak. It is likely that a verbal invitation reassured Chin Ping that she was indeed in the right place to ask a question, and that she had permission to do so, worries that trouble most international students. These results support and amplify the research findings of Onwuegbuzie and Jiao48 regarding library anxiety levels among international students, and the conclusions of Wayman regarding the hesitation of international students to initiate conversation.49 In most instances, Chin Ping noted that the librarians exhibited patience while dealing with her question, but this quality appeared to diminish as the interaction progressed. In over 50 percent of interactions (11/20), Chin Ping felt that she was being rushed toward an early conclusion of the interview, without the full extent of her inquiry being revealed and without her receiving sufficient guidance. In most of these cases, it appears that the librarians were acting like reference racehorses, eager to shoot out the starting gate and finish the race, but in too many cases leaving Chin Ping (the jockey?) behind, and therefore not winning the race. Analysis of Chin Ping’s narratives appears to indicate that most of these librarians who conducted cursory reference interviews expected her to interrupt or interject with more clarifying information about her question as they charged ahead. This ‘‘aggressiveness’’ would be difficult for many international students, as cultural differences might characterize such forwardness as rude, and hesitation with spoken English might make articulation of a revised question difficult. Cultural sensitivity is noted as a key element in Liestman’s work, and his conclusions are supported in this research.50 Lack of sufficient communication was also evident in many librarians’ closing procedures as only three of twenty librarians asked Chin Ping if they had given her sufficient tools or assistance for her to answer her question. In many cases, had she been asked, her answer would have been ‘‘no.’’ Overall, tactics that librarians used in the highest-rated interactions that were absent in the lowest were offering a warm, friendly greeting, spoken first; relating a search-strategy story to explain what the librarian was doing; writing down search terms or key words; asking enough questions to ‘‘tease out’’ the full extent of Chin Ping’s question; and conducting follow-up—ensuring that the piece of paper with call numbers or article titles on it had actually started Chin Ping on the right track.

418 The Journal of Academic Librarianship

Prevalent in the lowest-rated interactions was a disturbing lack of sensitivity and awareness that the librarians had left Chin Ping at the ‘‘starting gate’’ of the reference interaction, either with their rapidly paced English, their use of jargon, their premature assumptions about the scope of the question, or their eagerness to conclude the interaction. It is hoped that others can learn from the strengths and weaknesses exhibited by the college and university librarians in this study, and that all librarians take advantage of general multi-cultural diversity training or specialized international student reference workshops to heighten their awareness of recommended strategies. Our goal should be that all students like the young woman described in the Introduction of this article – frightened, hesitant, and uncertain of her ability to express her question in English – receive the best help we can provide. Acknowledgments: The contributions of research collaborators Diana Wu and Mei Li to this project are gratefully acknowledged. As well, we would like to express our appreciation to the chief librarians and reference librarians from the colleges and universities in the study.

APPENDIX A Reference Interaction Record Interaction #________ Time of day: _____________ Day of week: _____________ How many people are waiting, not engaged with a librarian, at the reference desk? o0

o 1–2

o 3–4

o 5+

A.1 For questions 1 to 16, check the one best answer Reference Service 1. The staff member looked approachable. ___yes

___no

___n/a

2. The staff member asked questions for clarification of my information need. ___yes

___ no

___ n/a

3. The staff member explained what he/she was doing at every stage. ___ yes

___ no

___ n/a

4. The staff member instructed me in how to use information sources. ___ yes

___ no

___ n/a

5. The staff member did not overwhelm me with information. ___ yes

___ no

6. The staff member had a respectful attitude toward me and my question. ___ yes

___ no

7. The staff member went with me to direct me to useful sources. ___ yes

___ no

___ n/a

8. The staff member referred me to someone else. ___ yes

___ no

___ n/a

9. The staff member asked me if I had found what I needed. ___ yes

___ no

___ no

___ n/a

14. The staff member spoke clearly.

___ no

16. Overall, the staff member showed awareness of potential language barriers. ___ yes

3. Given a choice, would you return to the same staff member for help? ___ not likely to return ___ likely to return ___ definitely would return

___ no

15. The staff member avoided using library jargon, using instead simple sentences and vocabulary. ___ yes

___ satisfied

___ definitely would not return

13. The staff member rephrased to clarify a misunderstanding.

___ yes

____ more than 20 minutes

___ very satisfied

___ no ___ no

____ from 15 to 20 minutes

___ dissatisfied

12. The staff member was patient if I paused to formulate my query.

___ yes

____ from 5 to 10 minutes

___ very dissatisfied

11. The staff member listened carefully to what I said.

___ yes

____ less than 5 minutes

2. How satisfied are you with the answer you received?

Awareness of Language Barriers ___ yes

1. How much time did the librarian spend with you on your question?

____ from 10 to 15 minutes

___ no

10. The staff member told me to come back if I needed more help. ___ yes

A.3 For questions 1 to 3, check the one best answer.

___ no

A.2 1. Write an account of what happened, step by step, when you went to the library to ask your question. Include everything you did, said and thought, as well as everything that others did and said. You can use the points below to help you write as complete an account of the experience as possible. ! what the librarian was doing as you approached ! whether the librarian initiated the communication ! any non-verbal signals you received (for example, facial expression) ! whether the librarian moved out from behind the desk ! whether the librarian made sure you had found what you needed __________________________________________________ __________________________________________________ __________________________________________________ **Note: Three pages of blank lined paper were included here for Chin Ping’s narrative response.

NOTES

AND

REFERENCES

1. RUSA Reference Guidelines, Guidelines for Behavioral Performance of Reference and Information Services Professionals (Chicago: American Library Association, 2004), [Online]. Available: http://www.ala.org/ala/rusa/rusaprotools/referenceguide/ guidelinesbehavioral.htm (accessed September 24, 2004). 2. National Center for Education Statistics, Digest of Education Statistics (Washington, DC: National Center for Education Statistics, 2002), [Online]. Available: http://nces.ed.gov/programs/ digest/d02/tables/dt415.asp (accessed September 24, 2004). 3. Anthony J. Onwuegbuzie and Qun G. Jiao, ‘‘Academic Library Usage: A Comparison of Native and Non-native Englishspeaking Students,’’ Australian Library Journal 46 (3) (1997): 258 – 269. 4. Citizenship and Immigration Canada, Priorities, Planning and Research Branch, Foreign Students in Canada 1980–2001 (January 2003), [Online]. Available: http://www.cic.gc.ca/english/ research/papers/foreignstudents/students-toc.html (accessed September 24, 2004). 5. Ibid. 6. Rick Ouston, ‘‘English Now a Minority Language in Vancouver: Long-time Residents Are Moving to the Suburbs, Adding to the Language Change,’’ The Vancouver Sun 2 (November 1996): 1 (sec. A). 7. Yvonne de Souza, ‘‘Reference Work with International Students: Making the Most of the Neutral Question,’’ Reference Services Review 24 (4) (1996): 41 – 48. 8. Ziming Liu, ‘‘Difficulties and Characteristics of Students from Developing Countries in Using American Libraries,’’ College & Research Libraries 54 (1) (1993): 25 – 31. 9. Liu, p. 26. 10. Dania M. Bilal, ‘‘Library Knowledge of International Students from Developing Countries: A Comparison of their Perceptions

September 2005 419

with those of Reference Librarians’’ (Ph.D dissertation, Florida State University, 1988); Felicia Suila Kimo Lafon, ‘‘A Comparative Study and Analysis of the Library Skills of American and Foreign Students at the University of Michigan (Michigan)’’ (Ph.D dissertation, University of Michigan, 1992); Ting Ming Lai, ‘‘A Comparative Study of the Use of Academic Libraries by Undergraduates in the United States and Taiwan’’ (Ph.D dissertation, University of Wisconsin-Madison, 1990); Kwasi Sarkodie-Mensah, ‘‘Foreign Students and U.S. Academic Libraries: A Case Study of Foreign Students and Libraries in Two Universities in New Orleans, Louisiana (United States)’’ (Ph.D dissertation, University of Illinois at Urbana-Champaign, 1988). 11. Vivian Francis Howard, ‘‘Orientation Programs for International Students at Canadian Academic Libraries’’ (Master’s thesis, Dalhousie University, 1995). 12. Sara Baron and Alexia Strout-Dapaz, ‘‘Communicating with and Empowering International Students with a Library Skills Set’’, Reference Services Review 29 (4) (2001): 314 – 326; Ziming Liu, ‘‘Difficulties and Characteristics of Students from Developing Countries in Using American Libraries,’’ College & Research Libraries 54 (1) (1993): 25 – 31; Anthony J. Onwuegbuzie and Qun G. Jiao, ‘‘Academic Library Usage: A Comparison of Native and Non-native Englishspeaking Students,’’ Australian Library Journal 46 (3) (1997): 258 – 269. 13. Suhasini L. Kumar and Raghini S. Suresh, ‘‘Strategies for Providing Effective Reference Services for International Adult Learners,’’ Reference Librarian 69/70 (2000): 327 – 336. 14. Mengxiong Liu, ‘‘Ethnicity and Information Seeking,’’ Reference Librarian 49/50 (1995): 123 – 134. 15. Christopher C. Brown, ‘‘Reference Services to the International Adult Learner: Understanding the Barriers,’’ Reference Librarian 69/70 (2000): 337 – 348. 16. Kumar and Suresh, p. 334. 17. Kumar and Suresh, 2000; Brown, 2000; Daniel Liestman, ‘‘Reference Services and the International Adult Learner,’’ Reference Librarian 69/70 (2000): 363 – 377; Mengxiong Lui, 1995. 18. Liestman, p. 368. 19. Brown, pp. 343–344. 20. Sally G. Wayman, ‘‘The International Student in the Academic Library,’’ Journal of Academic Librarianship 9 (6) (1984): 336 – 341. 21. Kwasi Sarkodie-Mensah, ‘‘Dealing with International Students in a Multicultural Era,’’ Journal of Academic Librarianship 92 (4) (1992): 214 – 216. 22. de Souza, p. 44.

420 The Journal of Academic Librarianship

23. Mengxiong Liu, p. 130. 24. RUSA Reference Guidelines. 25. Jennifer Mendelsohn, ‘‘Perspectives on Quality of Reference Service in an Academic Library: A Qualitative Study,’’ RQ 36 (4) (1997): 544 – 557. 26. Patricia Dewdney and Catherine Sheldrick Ross, ‘‘Flying a Light Aircraft: Reference Service Evaluation from a User’s Perspective,’’ RQ 34 (2) (1994): 217 – 230. 27. Dewdney and Ross, 1994; Catherine Sheldrick Ross and Kirsti Nilsen, ‘‘Has the Internet Changed Anything in Reference?: The Library Visit Study, Phase 2,’’ Reference & User Services Quarterly 40 (2) (2000): 147 – 155. 28. Catherine Sheldrick Ross, ‘‘The Reference Interview: Why It Needs to Be Used in Every (Well, Almost Every) Reference Transaction,’’ Reference & User Services Quarterly 43 (1) (2003): 38 – 42. 29. Alan Bryman, Social Research Methods (Oxford: Oxford University Press, 2001). 30. Russell Jones, Research Methods in the Social and Behavioral Sciences (Sunderland, Maine: Sinauer Associates, 1996). 31. Brown, 2000; Kumer and Suresh, 2000; Mengxiong Liu, 1995; Ziming Liu, 1993. 32. Dewdney and Ross, 1994; Joan C. Durrance, ‘‘Reference Success: Does the 55 Percent Rule Tell the Whole Story?’’ Library Journal 114 (1989): 31 – 36. 33. RUSA Reference Guidelines. 34. Dewdney and Ross, 1994; Durrance, 1989. 35. Ibid. 36. Ibid. 37. Dewdney and Ross, 1994. 38. Bryman, pp. 390–399. 39. Kumar and Suresh, 2000; Mengxiong Liu, 1995; Liestman, 2000. 40. Mengxiong Liu, p. 130. 41. Ross, p. 40. 42. Ibid., p. 40. 43. Dewdney and Ross, p. 225; RUSA Reference Guidelines, p. 4.1. 44. RUSA Reference Guidelines, 5.0. 45. Ross, p. 41. 46. Dewdney and Ross, p. 225. 47. Mengxiong Liu, p. 130. 48. Onwuegbuzie and Jiao, pp. 258–269. 49. Wayman, pp. 336–341. 50. Liestman, pp. 363–377.