- Email: [email protected]
Reviews and Analysis of Special Reports
KEEPING CURRENT · Reviews and Analysis of Special Reports by Leslie Stebbins
Available online 21 April 2012
T
his column provides in-depth analysis of recently issued white papers, research studies, presentations, and special reports that the editor feels would be of greatest interest to academic librarians. This is a highly selective compilation, and the editor welcomes suggestions from readers. [email protected]. “Users, Narcissism and Control—Tracking the Impact of Scholarly Publications in the 21st century.” SURF Foundation (February 2012). Available: http://www.surffoundation.nl/nl/publicaties/Documents/ Users%20narcissism%20and%20control.pdf. Overview Researchers and funders have always been interested in evaluating the scientific and social impact of particular research publications. With the advent of the Internet, many tracking tools have been developed to monitor in real time how new research is being read, cited, used, and transformed. This report investigates sixteen of these new tools and reviews the advantages, disadvantages, and risks associated with each tool. Key Points • The report investigated and reported on the following 16 tools: Faculty of 1000 (F1000), Peer Evaluation, PaperCritic, Google Scholar, Google Citations, Microsoft Academic Search (MAS), ArnetMiner, JISC Open Citations, Mendeley, CiteUlike, Zotero, Readermeter, Total-Impact, ScienceCard, PLoS One Altmetrics, and SURE. • Most of the tools were found to have friendly interfaces; provided metrics at the level of the article, manuscript or book; and were free. • These tools can be useful to the individual researcher to get a sense of the impact of his or her own work, but they do not adhere to strict enough protocols or data quality and indicator reliability and validity to be used in a more formal manner. • Many of these tools lack transparency in the way indicators are computed. • Limitations of these new tools need to be addressed before they can be useful for measuring publication impact. Analysis The report concludes that there are now a variety of new information filters that can enable researchers to make some limited self-assessments about their own work, but most of these new tools do not have the reliability and validity needed to provide objective assessments. The report encourages the development of a new program to help foster and advise the future development of some of these new tools, in particular F1000, MAS, Research PlosOne, Altmetrics and Google Scholar.
Leslie Stebbins, Lexington, MA 02421, USA [email protected].
While these new tools have done a great deal toward capturing data that moves beyond traditional frameworks for evaluation, they are still in the early stages and need to make significant strides forward before researchers and tenure and promotion committees use them for any formal evaluation process. As the authors point out in this report, many of these tools are limited in scope, some have very little transparency and fail to filter out less useful documents, and reviews and ratings and their relationship to other systems are difficult to ascertain. MAS seems to do the best job of being accessible by promising APIs soon for data access, but MAS is still small in scope in comparison to tools like GoogleScholar. Altmetrics and some of the other new tools are beginning to provide metrics on the number of readers, tags used, bookmarks, comments, blogging, and tweets, but there is not enough transparency in these systems to determine accuracy or bias. While more traditional tools such as Web of Science and Scopus focus on traditional scholarly sources, their transparency and ease of use provide users with a fairly accurate idea of what is being measured and how it is being measured. While the authors call on the developers of these new systems to improve their tools, this same request needs to go out to our more traditional database providers. These traditional databases provide us with comprehensive coverage of scholarly articles, but it is time for these vendors to investigate adding access to less traditional markers of scholarly success such as tweets, blogs, and reviews. The approach will need to be uniform, and it is certain that different disciplines will embrace these alternative measures more quickly than others, but a cautious step or perhaps collaborative endeavor in this direction would be welcomed by many. “The Future of University Libraries: 2012 Midwinter Report,” by Michelle Dunaway for the Committee on the Future of University Libraries (January 2012). Available: http://connect.ala.org/node/168863. Overview The ACRL/ULS Committee on the Future of University Libraries has been investigating emerging trends in order to develop ways to better support librarians and help them to increase their value to their constituents. As part of their work, the committee reviewed publications from the past year relating to the evolving role of the academic library. This document summarizes their key findings and provides analysis based on three broad themes: mission, money, and people. Key Points • Librarians are well aware of the need to shift their central mission and core services in light of changing needs. Themes emerging from recent articles relating to the role of the research library focus on demonstrating value, rethinking library services, reconfiguring library spaces, and preparing psychologically for the future.
The Journal of Academic Librarianship, Volume 38, Number 3, pages 183–185
May 2012 183
• Budget cuts have become the “new normal.” Reducing costs and finding ways to generate funds from non-traditional sources are the focus for this area. • Entrepreneurship within the profession, and finding ways to attract and increase retention of talented library leaders are seen as essential to helping libraries be successful in the rapidly evolving higher education environment. Analysis This review of the literature from the past year on planning for the future of academic libraries provides broadly defined ideas for moving forward into an uncertain future where budgets are decreasing, physical space is shifting away from traditional library activities, and the very definition of the purpose of an academic library is being called into question. The large number of publications that have come out in the past few years on the future of the academic library has been staggering and draws attention to significant concerns that many in the profession are experiencing. Especially useful in this report is the analysis of the recent focus on libraries becoming entrepreneurial institutions. Heather MonroeBlum of McGill University and James Neal at Columbia University have pointed out that entrepreneurship is about innovation and problem solving. The role of the entrepreneur, Monroe-Blum says, is to “identify new problems and crystallize the benefits of solving them.” This is an exciting new lens from which to view the role of the library. What are the current challenges in higher education that might relate to the new work of libraries? What is broken now and how can we help fix it? How can we identify these needs and figure out our new roles? Monroe-Blum argues convincingly that an entrepreneurial approach should be combined with building internal and external connections while at the same time balancing this work with our more traditional focus on learning, teaching, and knowledge. (For more details see: www.mcgill.ca/principal/speeches/lectures/ahead-ofthe-storm/.) One area in the world of information that is becoming broken is the ability of Google and the Internet to provide reliable, high quality and easy to find information. The commercial web is now dominating search results and valuable information, except in the case of open access journals, is starting to retreat behind pay walls. Though this was not touched on in the report, the need for college students to learn to find and evaluate information is higher than ever, and the library is already positioned and providing this kind of instruction. Better marketing and developing creative ways to effectively deliver this instruction through cutting edge instructional design and online learning tools should be near the top of the list in academic library planning documents. This review also touches on new roles successfully undertaken by librarians in the past few years that reposition these libraries to better serve a rapidly changing higher education environment. The rise of the virtual library and the embedded librarian; e-research support services to better support learning, teaching and research; and a focus on content delivered will help provide more valued services to the community. Marketing and formative evaluation need to go hand-inhand with the rollout of these new services. “The NMC Horizon Report: 2012 Higher Education Edition,” by Johnson, L., Adams, S., and Cummins, M. Austin, Texas: The New Media Consortium (2012). Overview For the past ten years the NMC Horizon Report: Higher Education Edition has outlined emerging trends in colleges and universities that are “on the horizon,” and can be expected to reach a tipping point in the next five years. The process of compiling the report involves engaging diverse groups of experts in conversations, and drawing
184 The Journal of Academic Librarianship
the findings of recent research. Each year the report highlights six emerging technologies or practices and identifies related trends and challenges. Key Points • Recent trends include an expectation of anywhere and anytime working and learning; decentralized IT support in the cloud; an increase in collaborative student projects to reflect changes in the workplace; the rising importance of helping students navigate and evaluate vast amounts of information and social networks, an increase in online and hybrid learning; and an increase and call for active challenge-based learning. • Challenges include traditional education models that are increasingly competing with new models of education; the metrics of evaluation that are lagging behind new forms of scholarly communication; digital media literacy becoming a crucial skill; institutional barriers that are significant in implementing new technologies; and new modes of scholarship that are presenting challenges to libraries and universities. • The six technologies to currently watch are mobile apps, tablet computing, game-based learning, learning analytics, gesture-based computing, and the “Internet of Things,” meaning methods that enable an object to be assigned a unique identifier with data attached to it in order to relay information about that object. Analysis All of these trends, challenges and new technologies are already having a significant impact on the day-to-day work of academic librarians, with the exception of gesture based computing and the “Internet of Things.” Mobile Apps, for example, have challenged and changed the way libraries interact with users including apps such as iPrinceton, which includes the ability to browse the library catalog easily on a mobile device. At Stanford, lectures and slides can be accessed through iTunes U for their iPhone and iPad application development course. This course has now broken records by being downloaded one million times within its first seven weeks online. What role will Learning Management Systems have in relationship to these new trends? What role will librarians play? Tablets, and primarily the iPad, now account for 47% of all mobile web traffic. Relevant to libraries, tablets are particularly good for reading magazines, newspapers, and e-books. Tablets are now especially popular in the K-12 learning environment, which means that college students of tomorrow will enter campuses with tablet in hand. A number of institutions are currently providing their students with tablets that come pre-loaded with course materials, digital textbooks, and other resources. Though tablets will not replace mobile devices, they are becoming devices in their own right and apart from laptops and smart phones. While game-based learning seems a little less immediately relevant to libraries, serious games do lend themselves to helping engage students in critical thinking and digital literacy skills, skills that are on the front burner for current library instruction programs. Games also impact the academic library as a collection development issue. University of Michigan now has a Computer and Video Game Archive and California University at Santa Cruz has a substantial collection of video games available for checkout. These new developments raise questions relating to access, storage, maintenance, and scope for libraries entering this area. Learning Analytics, the interpretation of data about students for formative and summative assessment, is becoming more prominent as data-driven education and formative assessment practices become widespread. Institutions are still scrambling with this issue, and librarians would be a natural to take the lead on this.
“7 Things You Should Know about Flipped Classrooms.” EDUCAUSE Learning Initiative (ELI) (February 2012). Available: http://net. educause.edu/ir/library/pdf/ELI7081.pdf. Overview This report provides an overview and examples of the flipped classroom. The Flipped Classroom is a pedagogical model in which the in-class lecture and the homework are reversed. Students watch short videos at home prior to a class session, and in-class time is then devoted to working on projects, exercises, or collaborative activities facilitated by the instructor. Key Points • There is no one model, but emphasis is placed on students viewing short, 5–7 min videos. Online quizzes, interactive worksheets, blog postings or other activities can be combined with the video to assess student learning and compliance. • The flipped model puts more responsibility for learning on the students and provides them with activities that help them master the material rather than just passively listen to information being delivered in a lecture format. • While simple in concept, the flipped model is not without implementation challenges. Analysis The Flipped Classroom is really an idea whose time has been trying to come for several decades. Its roots can be found in active and collaborative learning, “guide on the side,” and other learnercentered and constructivist pedagogies. With the current ease in lecture capture tools, instructors can finally rest assured that students will be able to access course lecture content while the classroom experiences can focus on building learning communities and providing students with activities that will help them connect to their current level of knowledge.
The beauty of learner-centered and collaborative instruction is that it pushes students to enter into a conversation about a subject at the level that they are comfortable with, and by this very conversation they can gradually build their own knowledge about a subject by tying it into what they already understand. The Flipped Classroom is a brilliant model that leverages innovative new technologies to provide students with an improved learning experience. Though simple in concept, the Flipped Classroom can be challenging to implement. Lecture capture is not simply a matter of filming a talking head. To be effective, lecture capture must be concise, include slides or visuals, and provide an interactive “check-in” component to reinforce learning. The classroom side of the equation can present challenges for faculty not comfortable with new learning pedagogies. Initially developing and working out the kinks in in-class learning activities requires time and perseverance, not to mention motivation on the part of faculty. The Flipped Classroom does not mean just letting students sit in groups and discuss content, but instead requires active participation in meaningful activities that are shepherded by the instructor. Ideally this model can include challenge-based or problembased learning activities. A number of faculty members around the country are using the Flipped Classroom in innovative ways. One physics professor at Harvard uses the flipped model and has also developed a correlative site, “Learning Catalytics,” that provides instructors with free interactive software that enables students to discuss, apply, and receive formative feedback on what they hear during the home-based lecture video. The Flipped Classroom would be conducive to library instruction in that initial learning objects or videos could be embedded in courses and used prior to a library session while the session itself could use collaborative learner-centered strategies to help students understand content. Librarians could serve as models for helping faculty understand the value of a carefully constructed Flipped Classroom design. For readers not familiar with the “7 Things” series created by ELI these deceptively short and simple descriptions of new trends in technology and education are a must read. See: http://www.educause. edu/ELI7Things.
May 2012 185
Available online 21 April 2012
T
his column provides in-depth analysis of recently issued white papers, research studies, presentations, and special reports that the editor feels would be of greatest interest to academic librarians. This is a highly selective compilation, and the editor welcomes suggestions from readers. [email protected]. “Users, Narcissism and Control—Tracking the Impact of Scholarly Publications in the 21st century.” SURF Foundation (February 2012). Available: http://www.surffoundation.nl/nl/publicaties/Documents/ Users%20narcissism%20and%20control.pdf. Overview Researchers and funders have always been interested in evaluating the scientific and social impact of particular research publications. With the advent of the Internet, many tracking tools have been developed to monitor in real time how new research is being read, cited, used, and transformed. This report investigates sixteen of these new tools and reviews the advantages, disadvantages, and risks associated with each tool. Key Points • The report investigated and reported on the following 16 tools: Faculty of 1000 (F1000), Peer Evaluation, PaperCritic, Google Scholar, Google Citations, Microsoft Academic Search (MAS), ArnetMiner, JISC Open Citations, Mendeley, CiteUlike, Zotero, Readermeter, Total-Impact, ScienceCard, PLoS One Altmetrics, and SURE. • Most of the tools were found to have friendly interfaces; provided metrics at the level of the article, manuscript or book; and were free. • These tools can be useful to the individual researcher to get a sense of the impact of his or her own work, but they do not adhere to strict enough protocols or data quality and indicator reliability and validity to be used in a more formal manner. • Many of these tools lack transparency in the way indicators are computed. • Limitations of these new tools need to be addressed before they can be useful for measuring publication impact. Analysis The report concludes that there are now a variety of new information filters that can enable researchers to make some limited self-assessments about their own work, but most of these new tools do not have the reliability and validity needed to provide objective assessments. The report encourages the development of a new program to help foster and advise the future development of some of these new tools, in particular F1000, MAS, Research PlosOne, Altmetrics and Google Scholar.
Leslie Stebbins, Lexington, MA 02421, USA [email protected].
While these new tools have done a great deal toward capturing data that moves beyond traditional frameworks for evaluation, they are still in the early stages and need to make significant strides forward before researchers and tenure and promotion committees use them for any formal evaluation process. As the authors point out in this report, many of these tools are limited in scope, some have very little transparency and fail to filter out less useful documents, and reviews and ratings and their relationship to other systems are difficult to ascertain. MAS seems to do the best job of being accessible by promising APIs soon for data access, but MAS is still small in scope in comparison to tools like GoogleScholar. Altmetrics and some of the other new tools are beginning to provide metrics on the number of readers, tags used, bookmarks, comments, blogging, and tweets, but there is not enough transparency in these systems to determine accuracy or bias. While more traditional tools such as Web of Science and Scopus focus on traditional scholarly sources, their transparency and ease of use provide users with a fairly accurate idea of what is being measured and how it is being measured. While the authors call on the developers of these new systems to improve their tools, this same request needs to go out to our more traditional database providers. These traditional databases provide us with comprehensive coverage of scholarly articles, but it is time for these vendors to investigate adding access to less traditional markers of scholarly success such as tweets, blogs, and reviews. The approach will need to be uniform, and it is certain that different disciplines will embrace these alternative measures more quickly than others, but a cautious step or perhaps collaborative endeavor in this direction would be welcomed by many. “The Future of University Libraries: 2012 Midwinter Report,” by Michelle Dunaway for the Committee on the Future of University Libraries (January 2012). Available: http://connect.ala.org/node/168863. Overview The ACRL/ULS Committee on the Future of University Libraries has been investigating emerging trends in order to develop ways to better support librarians and help them to increase their value to their constituents. As part of their work, the committee reviewed publications from the past year relating to the evolving role of the academic library. This document summarizes their key findings and provides analysis based on three broad themes: mission, money, and people. Key Points • Librarians are well aware of the need to shift their central mission and core services in light of changing needs. Themes emerging from recent articles relating to the role of the research library focus on demonstrating value, rethinking library services, reconfiguring library spaces, and preparing psychologically for the future.
The Journal of Academic Librarianship, Volume 38, Number 3, pages 183–185
May 2012 183
• Budget cuts have become the “new normal.” Reducing costs and finding ways to generate funds from non-traditional sources are the focus for this area. • Entrepreneurship within the profession, and finding ways to attract and increase retention of talented library leaders are seen as essential to helping libraries be successful in the rapidly evolving higher education environment. Analysis This review of the literature from the past year on planning for the future of academic libraries provides broadly defined ideas for moving forward into an uncertain future where budgets are decreasing, physical space is shifting away from traditional library activities, and the very definition of the purpose of an academic library is being called into question. The large number of publications that have come out in the past few years on the future of the academic library has been staggering and draws attention to significant concerns that many in the profession are experiencing. Especially useful in this report is the analysis of the recent focus on libraries becoming entrepreneurial institutions. Heather MonroeBlum of McGill University and James Neal at Columbia University have pointed out that entrepreneurship is about innovation and problem solving. The role of the entrepreneur, Monroe-Blum says, is to “identify new problems and crystallize the benefits of solving them.” This is an exciting new lens from which to view the role of the library. What are the current challenges in higher education that might relate to the new work of libraries? What is broken now and how can we help fix it? How can we identify these needs and figure out our new roles? Monroe-Blum argues convincingly that an entrepreneurial approach should be combined with building internal and external connections while at the same time balancing this work with our more traditional focus on learning, teaching, and knowledge. (For more details see: www.mcgill.ca/principal/speeches/lectures/ahead-ofthe-storm/.) One area in the world of information that is becoming broken is the ability of Google and the Internet to provide reliable, high quality and easy to find information. The commercial web is now dominating search results and valuable information, except in the case of open access journals, is starting to retreat behind pay walls. Though this was not touched on in the report, the need for college students to learn to find and evaluate information is higher than ever, and the library is already positioned and providing this kind of instruction. Better marketing and developing creative ways to effectively deliver this instruction through cutting edge instructional design and online learning tools should be near the top of the list in academic library planning documents. This review also touches on new roles successfully undertaken by librarians in the past few years that reposition these libraries to better serve a rapidly changing higher education environment. The rise of the virtual library and the embedded librarian; e-research support services to better support learning, teaching and research; and a focus on content delivered will help provide more valued services to the community. Marketing and formative evaluation need to go hand-inhand with the rollout of these new services. “The NMC Horizon Report: 2012 Higher Education Edition,” by Johnson, L., Adams, S., and Cummins, M. Austin, Texas: The New Media Consortium (2012). Overview For the past ten years the NMC Horizon Report: Higher Education Edition has outlined emerging trends in colleges and universities that are “on the horizon,” and can be expected to reach a tipping point in the next five years. The process of compiling the report involves engaging diverse groups of experts in conversations, and drawing
184 The Journal of Academic Librarianship
the findings of recent research. Each year the report highlights six emerging technologies or practices and identifies related trends and challenges. Key Points • Recent trends include an expectation of anywhere and anytime working and learning; decentralized IT support in the cloud; an increase in collaborative student projects to reflect changes in the workplace; the rising importance of helping students navigate and evaluate vast amounts of information and social networks, an increase in online and hybrid learning; and an increase and call for active challenge-based learning. • Challenges include traditional education models that are increasingly competing with new models of education; the metrics of evaluation that are lagging behind new forms of scholarly communication; digital media literacy becoming a crucial skill; institutional barriers that are significant in implementing new technologies; and new modes of scholarship that are presenting challenges to libraries and universities. • The six technologies to currently watch are mobile apps, tablet computing, game-based learning, learning analytics, gesture-based computing, and the “Internet of Things,” meaning methods that enable an object to be assigned a unique identifier with data attached to it in order to relay information about that object. Analysis All of these trends, challenges and new technologies are already having a significant impact on the day-to-day work of academic librarians, with the exception of gesture based computing and the “Internet of Things.” Mobile Apps, for example, have challenged and changed the way libraries interact with users including apps such as iPrinceton, which includes the ability to browse the library catalog easily on a mobile device. At Stanford, lectures and slides can be accessed through iTunes U for their iPhone and iPad application development course. This course has now broken records by being downloaded one million times within its first seven weeks online. What role will Learning Management Systems have in relationship to these new trends? What role will librarians play? Tablets, and primarily the iPad, now account for 47% of all mobile web traffic. Relevant to libraries, tablets are particularly good for reading magazines, newspapers, and e-books. Tablets are now especially popular in the K-12 learning environment, which means that college students of tomorrow will enter campuses with tablet in hand. A number of institutions are currently providing their students with tablets that come pre-loaded with course materials, digital textbooks, and other resources. Though tablets will not replace mobile devices, they are becoming devices in their own right and apart from laptops and smart phones. While game-based learning seems a little less immediately relevant to libraries, serious games do lend themselves to helping engage students in critical thinking and digital literacy skills, skills that are on the front burner for current library instruction programs. Games also impact the academic library as a collection development issue. University of Michigan now has a Computer and Video Game Archive and California University at Santa Cruz has a substantial collection of video games available for checkout. These new developments raise questions relating to access, storage, maintenance, and scope for libraries entering this area. Learning Analytics, the interpretation of data about students for formative and summative assessment, is becoming more prominent as data-driven education and formative assessment practices become widespread. Institutions are still scrambling with this issue, and librarians would be a natural to take the lead on this.
“7 Things You Should Know about Flipped Classrooms.” EDUCAUSE Learning Initiative (ELI) (February 2012). Available: http://net. educause.edu/ir/library/pdf/ELI7081.pdf. Overview This report provides an overview and examples of the flipped classroom. The Flipped Classroom is a pedagogical model in which the in-class lecture and the homework are reversed. Students watch short videos at home prior to a class session, and in-class time is then devoted to working on projects, exercises, or collaborative activities facilitated by the instructor. Key Points • There is no one model, but emphasis is placed on students viewing short, 5–7 min videos. Online quizzes, interactive worksheets, blog postings or other activities can be combined with the video to assess student learning and compliance. • The flipped model puts more responsibility for learning on the students and provides them with activities that help them master the material rather than just passively listen to information being delivered in a lecture format. • While simple in concept, the flipped model is not without implementation challenges. Analysis The Flipped Classroom is really an idea whose time has been trying to come for several decades. Its roots can be found in active and collaborative learning, “guide on the side,” and other learnercentered and constructivist pedagogies. With the current ease in lecture capture tools, instructors can finally rest assured that students will be able to access course lecture content while the classroom experiences can focus on building learning communities and providing students with activities that will help them connect to their current level of knowledge.
The beauty of learner-centered and collaborative instruction is that it pushes students to enter into a conversation about a subject at the level that they are comfortable with, and by this very conversation they can gradually build their own knowledge about a subject by tying it into what they already understand. The Flipped Classroom is a brilliant model that leverages innovative new technologies to provide students with an improved learning experience. Though simple in concept, the Flipped Classroom can be challenging to implement. Lecture capture is not simply a matter of filming a talking head. To be effective, lecture capture must be concise, include slides or visuals, and provide an interactive “check-in” component to reinforce learning. The classroom side of the equation can present challenges for faculty not comfortable with new learning pedagogies. Initially developing and working out the kinks in in-class learning activities requires time and perseverance, not to mention motivation on the part of faculty. The Flipped Classroom does not mean just letting students sit in groups and discuss content, but instead requires active participation in meaningful activities that are shepherded by the instructor. Ideally this model can include challenge-based or problembased learning activities. A number of faculty members around the country are using the Flipped Classroom in innovative ways. One physics professor at Harvard uses the flipped model and has also developed a correlative site, “Learning Catalytics,” that provides instructors with free interactive software that enables students to discuss, apply, and receive formative feedback on what they hear during the home-based lecture video. The Flipped Classroom would be conducive to library instruction in that initial learning objects or videos could be embedded in courses and used prior to a library session while the session itself could use collaborative learner-centered strategies to help students understand content. Librarians could serve as models for helping faculty understand the value of a carefully constructed Flipped Classroom design. For readers not familiar with the “7 Things” series created by ELI these deceptively short and simple descriptions of new trends in technology and education are a must read. See: http://www.educause. edu/ELI7Things.
May 2012 185