- Email: [email protected]
The measurement and evaluation of library services
Book Reviews
450
tamed, the subset loses its value. Another problem is that some of the terms may be out of date because the source of this list, the 13th edition of LCSH, has been replaced by the 15th edition by the time this book was published. There is a strong need for guidance in providing subject access to films and videos. But Subject Access to Films and Videos fails to meet the need. Nonprint materials present special problems for catalogers, and unpublished nonprint materials double the challenge because the Anglo-American Cataloguing Rules, second edition, 1988 revision (AACR 2), provides no specific rules on how to handle them. Cataloging Unpublished Nonprint Materials meets this special need. It is the result of an effort to apply AACR 2 rules systematically to unpublished nonprint materials, including sound recordings, motion pictures and videorecordings, graphic materials, computer files, three-dimensional artefacts and realia, and kits. Designed as a book for reference, CUNM closely parallels AACR 2 in structure and contents. A “General Provisions” chapter repeats general cataloging rules from Chapter 1 of AACR 2, following each rule with a discussion on how it can be applied to unpublished nonprint materials. For each medium a separate chapter lists relevant AACR 2 rules for that medium and discusses their applicability to nonpublished materials in such a medium. Catalogers are advised to consult both the “general provisions” and the specific chapter for the medium they are dealing with. The authors discuss typical cataloging problems carefully and provide many examples for specific rules. Pertinent LC rule interpretations are included when necessary. A very helpful feature is the inclusion of cataloging examples, ranging from 5 to 9, for each medium at the end of each chapter. Each unpublished item is described, followed by a bibliographic record, and a discussion of how the record is created. These examples help to illustrate the problems and the solutions for cataloging such materials. Another strength of CUNM is its chapter on the cataloging of kits. AACR 2 does not have specific rules for kits, and its definition for kits, judging from the confusion among catalogers, is insufficient, to say the least. CUNM offers a good operational definition of kits and recommends LCRI 1.10 for solving cataloging problems of unpublished kits. Several examples are provided for important cataloging areas, and the general section is followed by five examples. The book has a fairly complete bibliography and an index. Librarians in school, public, and special libraries often deal with nonprint materials, many of them locally produced or informally recorded. These librarians will find helpful suggestions in CUNM for cataloging unpublished nonprint items. Archivists and librarians in research institutions with special media collections will also appreciate the book. The information is clearly presented, and excellent examples are selected to illustrate the various applications of AACR 2 rules. CUNM shows that unpublished nonprint materials can be adequately analyzed and described for effective retrieval. REFERENCES Frost, C.O. (1989). Media access and organizarion:
A cataloging and reference sources guide for nonbook materials. Englewood, CO: Libraries Unlimited. Maillet, L. (1991). Subject control offibn and video: A comparison of three methods. Chicago: American Library Association.
Catholic University of America School of Library & Information Washington, D.C.
INGRID HSIEH-YEE
Science
The Measurement and Evaluation of Library Services (2nd edition). S. L. BAKER and F. W. LANCASTER. Information Resources Press, Arlington, VA (1991). xviii + 411 pp., ISBN o-87815-061-7. The first edition of The Management and Evaluation of Library Services presented an excellent review of evaluative literature and was an instant success. The second edition is in many ways a valuable update. Going beyond the first edition’s focus on academic libraries, the second edition covers evaluative studies in public, school, and special libraries, and provides an excellent synthesis of studies conducted in the last five decades. Baker and Lancaster not only describe research techniques and findings of these studies, but also advise readers on research design and alert them to important issues involved in evaluating particular services. The result is both impressive and overwhelming. Researchers, library science students, and librarians interested in evaluation will enjoy the literature survey and synthesis. But administrators and those who need the know-how on evaluation quickly may find some studies remotely related to their concerns and difficult to wade through. Such audience may do better in consulting Lancaster’s If You Want to Evaluate Your Library. . . (1988) for practical advice on evaluation techniques.
Book Reviews
451
The second edition focuses on evaluation of selected public services, including collection evaluation, collection use and material availability, catalog use, question answering, and database searching. Evaluation by the range and scope of library services is also included. The role of standards in evaluating library services and how they should be developed are thoughtfully debated. Common methods for user studies, questionnaires and interviews in particular, are carefully analyzed. The authors devote nearly 30% of the book to studies of collection and collection use, and group them into “materials-centered approaches, ” “use-centered approaches, ” “in-house use,” and “material availability” chapters to make them more accessible to readers. Many evaluation methods are described in great detail to illustrate their strengths and limitations, and the use of data for collection assessment and improvement is discussed. Collection development librarians will find these sections helpful, though somewhat technical. For a quick overview, readers may first want to consult the American Library Association’s Guide to the Evaluation of Library Collections (1989), which succinctly describes and assesses many of the evaluation methods discussed in these chapters. The question-answering part of reference services is treated in Chapter 8 competently. After contrasting macroevaluation and microevaluation in reference services, the authors present studies on how reference services are used, how many and what type of reference questions are received, reference accuracy rates, possible causes for low reference accuracy rates, and studies on patron satisfaction. Findings are well synthesized, data is effectively presented (see for example the summary table of reference accuracy rate on page 246), and sound advice is provided (for instance, on how to ensure data reliability and how to test for reference accuracy). Unlike the first edition, which contrasted obtrusive and unobtrusive evaluation and discussed major studies on the latter in detail, the distinction is not stressed in the second edition. The authors, instead, present a brief but careful critique of unobtrusive tests and point interested readers to the authorities in this area. The authors are also successful in drawing on findings on reference interview, search strategy, and other areas to help readers understand the complexity of evaluating this service. The chapter on database searching has a structure similar to that of the chapter on question answering, but the result is less satisfactory. User awareness of service, volume of requests and types of answers, patron satisfaction, and important measures of search results, such as precision, recall, and novelty, are carefully described. But the authors’ advice on evaluating the search process could be questioned. For instance, evaluators are advised to determine if the right databases are used, if the appropriate search terms are used, if the search logic is appropriate, and how long a search takes. Logical as these questions may seem, such advice presumes that the evaluators possess much technical knowledge of online searching and subject knowledge of the search request, and have the right answers for these questions. But the fact is that all of these measures can be affected by a number of factors, such as the searcher’s experience with a particular database and its thesaurus, personal knowledge of the search topic, the amount of information requested, the presence or absence of the patron at the time of searching, telecommunication traffic jam, and so on. There is more than one way to conduct a search correctly. The closest indicator of a good search is probably the combination of patron’s satisfaction and precision and novelty (recall is not feasible, mainly for economic and technical reasons). The authors’ recommended approach could probably work on evaluating junior online searchers, but for experienced searchers, a different approach will be necessary. In spite of the authors’ best effort to bring important issues to readers’ attention, there are a few that could be removed to avoid information overload. For instance, studies on database failures, the retrieval performance of keyword searching, full text searching, abstract and title searching, and the “quorum function search” are not of immediate interest to typical evaluators (though researchers may find them intriguing) and could be left out. The authors, however, present a series of good questions to evaluate the presearch interview (p. 291-294). These are questions that ought to be covered in online searching courses. Catalog use studies span four decades, but the emphasis here is on more recent studies. Unlike the first edition in which major catalog use studies were discussed in detail one by one, followed by a summary of minor studies, the second edition has a thematic approach. It begins by summarizing studies on the volume of catalog use, then discusses common approaches to measuring the effectiveness of catalog use-surveys, controlled experiment, focus group interviews, and transaction logs. This is followed by findings on the types of searches conducted, the success of known-item searches, that of subject searches, and user problems with the catalog. The authors handle the challenge of literature synthesis well. What is missing is the summary of important catalog findings that appeared in the first edition. It would have been useful to have a similar summary on studies conducted since 1977. The second edition contains much updated information. The chapters on standards and evaluation of a library’s range and scope of services are expanded, and many studies conducted between
452
Book Reviews
1977 and 1988 are included. Perhaps due to the time lag in publication, research done after 1988 is not included, even though the book was published in 1991. This is most unfortunate, because the book was not as current as it could have been. The second edition, nonetheless, has many good features, such as fairly comprehensive bibIiographies on evaluative literature, inclusion of excellent evaluation forms developed by ALA (e.g., the reference transaction assessment instrument and the search evaluation questionnaire to assess patron satisfaction), and an adequate index. The authors state in the preface that the new edition “seeks to combine the best of” the 1977 edition and Lancaster’s 1988 work on evaluation. Essentially the challenge for the authors is to provide readers with skills and knowledge on how to evaluate effectively and efficiently. The authors chose to accomplish this goal by illustrating various evaluative techniques with many studies. Conceptually sound as it may be, the result is mixed. The book contains basically detailed literature reviews, but the authors strive to make it an evaluation guidebook. In some cases, such as the chapters on collection evaluation and reference questioning and answering they succeed. But in others, such as “evaluation of catalog use,” the flavor of literature review is definitely stronger. Depending on their orientation, readers will enjoy this book to varying degrees. Researchers will admire the authors’ ability to synthesize major findings, doctoral students will find the book a helpful tool in familiarizing themsdves with research literature, librarians and master’s degree students will appreciate the exposure to important issues involved in evaluating particular public service areas, and faculty can selectively use this book for courses on research methodology, management, and evaluation. But administrators and novice evaluators may find it difficult to deal with issues and studies not quite related to their immediate concerns. Nonetheless, the second edition as a whole is a success. Serious readers would want to have a personal copy. REFERENCES Guide
to the Evaluation of Library Collection (1989). BARBARA LOCKETT (Ed.). Chicago: American Libraries Association. Lancaster, F.W. (1988). If You Want to Evaluate Your Library . . . Champaign, IL: University of Illinois, Graduate School of Library and Information Science.
The Common Market for Information: Proceedings of the Annual Conference of the Institute of Information Scientists, June 1992, Bedford. M. BLAKE (Ed.). Taylor Graham, London (1992). 121 pp_,
ISBN O-947568-56-5. This is an avowedly incomplete record of the ISi’s (institute of Information Scientists) 1992 annual conference. The 14 more or less substantive contributions are written as spoken. Some are so brief (e.g., Patricia Stoat’s gossamer piece on information provision for small firms), some so general (e.g., Martin White’s overview of telecommunications trends), and some so enigmatic (e.g., Diana Clegg’s workshop summary) that their inclusion is a trifle perplexing. Others are we~I-meaning, but seemingly out of place (e.g., Andrzej Dziubecki’s short introduction to Polish library and information services), and yet others have simply been overtaken by political events (e.g., Maria Hughes’ minimalist account of information management issues in Czechoslovakia). But this curate’s egg, with its exclusively European focus and feel, has its goods parts. And the conscientious North American reader would do well to check out those parts, for one very good reason: Nafta. What has been happening in Europe, following the advent of the Single Market, not to mention the demise of communism, could we11be a harbinger of events in the Americas. On the assumption that trade flows follow information flows, eventual pan-American market liberalization could constitute a veritable bonanza for the information professions (Cronin, 1992). Four papers, in particular, convey a sense of the organizational challenges, legal intricacies, market opportunities, and bureaucratic inanities that the European common market for information goods and services has spawned. Alan Blick describes how, post merger, SmithKlein Beecham developed a transnational information sciences department for its globally distributed R&D operations, which includes a strategic intelligence unit with a staff of eight analysts. One outcome of the re-engineering exercise was Blick’s belief that he had “spent 25 years headed in the wrong direction of trying to meet user need” when he should have been “focussing on those operations in the organisation on which information had the most impact” (p. IO). What Blick has to say should be of interest to information scientists both inside and outside the pharmaceuticals sector.
450
tamed, the subset loses its value. Another problem is that some of the terms may be out of date because the source of this list, the 13th edition of LCSH, has been replaced by the 15th edition by the time this book was published. There is a strong need for guidance in providing subject access to films and videos. But Subject Access to Films and Videos fails to meet the need. Nonprint materials present special problems for catalogers, and unpublished nonprint materials double the challenge because the Anglo-American Cataloguing Rules, second edition, 1988 revision (AACR 2), provides no specific rules on how to handle them. Cataloging Unpublished Nonprint Materials meets this special need. It is the result of an effort to apply AACR 2 rules systematically to unpublished nonprint materials, including sound recordings, motion pictures and videorecordings, graphic materials, computer files, three-dimensional artefacts and realia, and kits. Designed as a book for reference, CUNM closely parallels AACR 2 in structure and contents. A “General Provisions” chapter repeats general cataloging rules from Chapter 1 of AACR 2, following each rule with a discussion on how it can be applied to unpublished nonprint materials. For each medium a separate chapter lists relevant AACR 2 rules for that medium and discusses their applicability to nonpublished materials in such a medium. Catalogers are advised to consult both the “general provisions” and the specific chapter for the medium they are dealing with. The authors discuss typical cataloging problems carefully and provide many examples for specific rules. Pertinent LC rule interpretations are included when necessary. A very helpful feature is the inclusion of cataloging examples, ranging from 5 to 9, for each medium at the end of each chapter. Each unpublished item is described, followed by a bibliographic record, and a discussion of how the record is created. These examples help to illustrate the problems and the solutions for cataloging such materials. Another strength of CUNM is its chapter on the cataloging of kits. AACR 2 does not have specific rules for kits, and its definition for kits, judging from the confusion among catalogers, is insufficient, to say the least. CUNM offers a good operational definition of kits and recommends LCRI 1.10 for solving cataloging problems of unpublished kits. Several examples are provided for important cataloging areas, and the general section is followed by five examples. The book has a fairly complete bibliography and an index. Librarians in school, public, and special libraries often deal with nonprint materials, many of them locally produced or informally recorded. These librarians will find helpful suggestions in CUNM for cataloging unpublished nonprint items. Archivists and librarians in research institutions with special media collections will also appreciate the book. The information is clearly presented, and excellent examples are selected to illustrate the various applications of AACR 2 rules. CUNM shows that unpublished nonprint materials can be adequately analyzed and described for effective retrieval. REFERENCES Frost, C.O. (1989). Media access and organizarion:
A cataloging and reference sources guide for nonbook materials. Englewood, CO: Libraries Unlimited. Maillet, L. (1991). Subject control offibn and video: A comparison of three methods. Chicago: American Library Association.
Catholic University of America School of Library & Information Washington, D.C.
INGRID HSIEH-YEE
Science
The Measurement and Evaluation of Library Services (2nd edition). S. L. BAKER and F. W. LANCASTER. Information Resources Press, Arlington, VA (1991). xviii + 411 pp., ISBN o-87815-061-7. The first edition of The Management and Evaluation of Library Services presented an excellent review of evaluative literature and was an instant success. The second edition is in many ways a valuable update. Going beyond the first edition’s focus on academic libraries, the second edition covers evaluative studies in public, school, and special libraries, and provides an excellent synthesis of studies conducted in the last five decades. Baker and Lancaster not only describe research techniques and findings of these studies, but also advise readers on research design and alert them to important issues involved in evaluating particular services. The result is both impressive and overwhelming. Researchers, library science students, and librarians interested in evaluation will enjoy the literature survey and synthesis. But administrators and those who need the know-how on evaluation quickly may find some studies remotely related to their concerns and difficult to wade through. Such audience may do better in consulting Lancaster’s If You Want to Evaluate Your Library. . . (1988) for practical advice on evaluation techniques.
Book Reviews
451
The second edition focuses on evaluation of selected public services, including collection evaluation, collection use and material availability, catalog use, question answering, and database searching. Evaluation by the range and scope of library services is also included. The role of standards in evaluating library services and how they should be developed are thoughtfully debated. Common methods for user studies, questionnaires and interviews in particular, are carefully analyzed. The authors devote nearly 30% of the book to studies of collection and collection use, and group them into “materials-centered approaches, ” “use-centered approaches, ” “in-house use,” and “material availability” chapters to make them more accessible to readers. Many evaluation methods are described in great detail to illustrate their strengths and limitations, and the use of data for collection assessment and improvement is discussed. Collection development librarians will find these sections helpful, though somewhat technical. For a quick overview, readers may first want to consult the American Library Association’s Guide to the Evaluation of Library Collections (1989), which succinctly describes and assesses many of the evaluation methods discussed in these chapters. The question-answering part of reference services is treated in Chapter 8 competently. After contrasting macroevaluation and microevaluation in reference services, the authors present studies on how reference services are used, how many and what type of reference questions are received, reference accuracy rates, possible causes for low reference accuracy rates, and studies on patron satisfaction. Findings are well synthesized, data is effectively presented (see for example the summary table of reference accuracy rate on page 246), and sound advice is provided (for instance, on how to ensure data reliability and how to test for reference accuracy). Unlike the first edition, which contrasted obtrusive and unobtrusive evaluation and discussed major studies on the latter in detail, the distinction is not stressed in the second edition. The authors, instead, present a brief but careful critique of unobtrusive tests and point interested readers to the authorities in this area. The authors are also successful in drawing on findings on reference interview, search strategy, and other areas to help readers understand the complexity of evaluating this service. The chapter on database searching has a structure similar to that of the chapter on question answering, but the result is less satisfactory. User awareness of service, volume of requests and types of answers, patron satisfaction, and important measures of search results, such as precision, recall, and novelty, are carefully described. But the authors’ advice on evaluating the search process could be questioned. For instance, evaluators are advised to determine if the right databases are used, if the appropriate search terms are used, if the search logic is appropriate, and how long a search takes. Logical as these questions may seem, such advice presumes that the evaluators possess much technical knowledge of online searching and subject knowledge of the search request, and have the right answers for these questions. But the fact is that all of these measures can be affected by a number of factors, such as the searcher’s experience with a particular database and its thesaurus, personal knowledge of the search topic, the amount of information requested, the presence or absence of the patron at the time of searching, telecommunication traffic jam, and so on. There is more than one way to conduct a search correctly. The closest indicator of a good search is probably the combination of patron’s satisfaction and precision and novelty (recall is not feasible, mainly for economic and technical reasons). The authors’ recommended approach could probably work on evaluating junior online searchers, but for experienced searchers, a different approach will be necessary. In spite of the authors’ best effort to bring important issues to readers’ attention, there are a few that could be removed to avoid information overload. For instance, studies on database failures, the retrieval performance of keyword searching, full text searching, abstract and title searching, and the “quorum function search” are not of immediate interest to typical evaluators (though researchers may find them intriguing) and could be left out. The authors, however, present a series of good questions to evaluate the presearch interview (p. 291-294). These are questions that ought to be covered in online searching courses. Catalog use studies span four decades, but the emphasis here is on more recent studies. Unlike the first edition in which major catalog use studies were discussed in detail one by one, followed by a summary of minor studies, the second edition has a thematic approach. It begins by summarizing studies on the volume of catalog use, then discusses common approaches to measuring the effectiveness of catalog use-surveys, controlled experiment, focus group interviews, and transaction logs. This is followed by findings on the types of searches conducted, the success of known-item searches, that of subject searches, and user problems with the catalog. The authors handle the challenge of literature synthesis well. What is missing is the summary of important catalog findings that appeared in the first edition. It would have been useful to have a similar summary on studies conducted since 1977. The second edition contains much updated information. The chapters on standards and evaluation of a library’s range and scope of services are expanded, and many studies conducted between
452
Book Reviews
1977 and 1988 are included. Perhaps due to the time lag in publication, research done after 1988 is not included, even though the book was published in 1991. This is most unfortunate, because the book was not as current as it could have been. The second edition, nonetheless, has many good features, such as fairly comprehensive bibIiographies on evaluative literature, inclusion of excellent evaluation forms developed by ALA (e.g., the reference transaction assessment instrument and the search evaluation questionnaire to assess patron satisfaction), and an adequate index. The authors state in the preface that the new edition “seeks to combine the best of” the 1977 edition and Lancaster’s 1988 work on evaluation. Essentially the challenge for the authors is to provide readers with skills and knowledge on how to evaluate effectively and efficiently. The authors chose to accomplish this goal by illustrating various evaluative techniques with many studies. Conceptually sound as it may be, the result is mixed. The book contains basically detailed literature reviews, but the authors strive to make it an evaluation guidebook. In some cases, such as the chapters on collection evaluation and reference questioning and answering they succeed. But in others, such as “evaluation of catalog use,” the flavor of literature review is definitely stronger. Depending on their orientation, readers will enjoy this book to varying degrees. Researchers will admire the authors’ ability to synthesize major findings, doctoral students will find the book a helpful tool in familiarizing themsdves with research literature, librarians and master’s degree students will appreciate the exposure to important issues involved in evaluating particular public service areas, and faculty can selectively use this book for courses on research methodology, management, and evaluation. But administrators and novice evaluators may find it difficult to deal with issues and studies not quite related to their immediate concerns. Nonetheless, the second edition as a whole is a success. Serious readers would want to have a personal copy. REFERENCES Guide
to the Evaluation of Library Collection (1989). BARBARA LOCKETT (Ed.). Chicago: American Libraries Association. Lancaster, F.W. (1988). If You Want to Evaluate Your Library . . . Champaign, IL: University of Illinois, Graduate School of Library and Information Science.
The Common Market for Information: Proceedings of the Annual Conference of the Institute of Information Scientists, June 1992, Bedford. M. BLAKE (Ed.). Taylor Graham, London (1992). 121 pp_,
ISBN O-947568-56-5. This is an avowedly incomplete record of the ISi’s (institute of Information Scientists) 1992 annual conference. The 14 more or less substantive contributions are written as spoken. Some are so brief (e.g., Patricia Stoat’s gossamer piece on information provision for small firms), some so general (e.g., Martin White’s overview of telecommunications trends), and some so enigmatic (e.g., Diana Clegg’s workshop summary) that their inclusion is a trifle perplexing. Others are we~I-meaning, but seemingly out of place (e.g., Andrzej Dziubecki’s short introduction to Polish library and information services), and yet others have simply been overtaken by political events (e.g., Maria Hughes’ minimalist account of information management issues in Czechoslovakia). But this curate’s egg, with its exclusively European focus and feel, has its goods parts. And the conscientious North American reader would do well to check out those parts, for one very good reason: Nafta. What has been happening in Europe, following the advent of the Single Market, not to mention the demise of communism, could we11be a harbinger of events in the Americas. On the assumption that trade flows follow information flows, eventual pan-American market liberalization could constitute a veritable bonanza for the information professions (Cronin, 1992). Four papers, in particular, convey a sense of the organizational challenges, legal intricacies, market opportunities, and bureaucratic inanities that the European common market for information goods and services has spawned. Alan Blick describes how, post merger, SmithKlein Beecham developed a transnational information sciences department for its globally distributed R&D operations, which includes a strategic intelligence unit with a staff of eight analysts. One outcome of the re-engineering exercise was Blick’s belief that he had “spent 25 years headed in the wrong direction of trying to meet user need” when he should have been “focussing on those operations in the organisation on which information had the most impact” (p. IO). What Blick has to say should be of interest to information scientists both inside and outside the pharmaceuticals sector.