- Email: [email protected]
Broken-Link Reports from SFX Users: How Publishers, Vendors and Libraries Can Do Better
Serials Review 38 (2012) 222–227
Contents lists available at SciVerse ScienceDirect
Serials Review journal homepage: www.elsevier.com/locate/serrev
Broken-Link Reports from SFX Users: How Publishers, Vendors and Libraries Can Do Better☆ Xiaotian Chen ⁎ Electronic Services Librarian and Associate Professor, Cullom-Davis Library, Bradley University, 1501W. Bradley Ave, Peoria, IL 61625, USA
a r t i c l e
i n f o
Available online 16 October 2012 Keywords: OpenURL link resolver SFX dead links
a b s t r a c t This study analyzes more than 400 SFX broken-link reports sent by users of an academic library. It raises technical issues regarding OpenURL linking in the handling of special journal volume and issue numbers, journal supplemental issues, embargo release dates, book reviews, DOIs, and other areas. It reports on full-text resources with the most broken links, causes of broken links, and the library's responses to users. It also explores how journal publishers, database vendors, and OpenURL vendors can improve the quality of their products and how librarians can better serve users. © 2012 Elsevier Inc. All rights reserved.
1. Introduction On December 16, 2011, a library user looking for the article “On Being Mindful, Emotionally Aware, and More Resilient” from Australian Psychologist, vol. 45, no. 4 (2010) hit a dead link. The article was supposed to be on EBSCO's Academic Search Premier (ASP), according to SFX, Ex Libris's OpenURL link resolver, but there was no full text. It turns out that the entire issue of vol. 45, no. 4 (2010) of this journal had not yet been loaded into EBSCO's databases. Both EBSCO and SFX acknowledged that Australian Psychologist has a 12-month embargo on Academic Search Premier and indicated that issue no. 4 of 2010 should have been available. However, full text from issue 4 of 2010 was still unavailable on December 16, 2011, even though SFX thought the embargo of one year had elapsed. Users of academic libraries encounter and report dead links from OpenURL link resolvers every week, if not every day. This study examines more than 400 SFX dead-link reports collected during fall 2007 and fall 2011 from a mid-size university in the United States. 2. Methodology When users of SFX encounter a dead link, they have the ability to report the problem to their home library through an SFX feedback form. Article citation and OpenURL are automatically populated into the feedback, so users need only to enter their name and email and submit the form. At this mid-size university, more than 400 SFX dead link reports plus responses from library staff to users have been recorded in a Microsoft Excel file. These records include the ☆ The author is grateful to Ellie Hansen of Bradley University (Emeritus) for editing this paper. ⁎ Tel.: +1 309 677 2839; fax: +1 309 677 2558. E-mail address: [email protected]. 0098-7913/$ – see front matter © 2012 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.serrev.2012.09.002
day, date, and time when the dead link reports were sent, when the library responded to the users, the citation of the articles, the databases in which users began searches (SFX sources), the full-text databases the users were referred to (SFX targets), the causes of the dead link, and whether the library managed to find the full text needed for the users. The library is a mid-sized academic library from a private institution in the Midwestern United States. The library implemented SFX in the summer of 2005 and started to record SFX dead link reports in the fall 2007. Two librarians were dedicated to receive SFX feedback from email and reply to users with email. During the office hours, they reply instantly. After office hours, users get a response most of time on the same day if the feedback is sent before 9:00 pm, including evenings, weekends, and holidays. When feedback is sent late at night, the responses are sent to users the next day. With the records in an Excel file, it is easy to filter and sort them for statistical analysis. This study tries to determine and statistically summarize the SFX targets with the most dead-link reports, the causes of dead links, and the library's average response time to users. It also reports whether the library could get the full text needed instantly and send the full text to users through email. The author hopes the paper will provide information to help full-text vendors and publishers who provide data to OpenURL link resolver vendors, and link resolver vendors themselves, to provide more accurate and timely exchange of data. 3. Literature Review To date, no literature has been found that studies SFX users' dead-link reports to summarize OpenURL dead-link causes. In their general study of SFX perceptions from end-users and librarians, Wakimoto et al. found through survey and focus-group studies that users expect more full-text availability while librarians expect more link accuracy. They estimated that SFX link accuracy overall is about
X. Chen / Serials Review 38 (2012) 222–227
80 percent (Wakimoto, Walker, & Dabbour, 2006). Trainor and Price (2010a) used the Wakimoto study data as a base to perform a historical comparison, and they found that full-text linking accuracy has not improved from 2004 to 2010. Trainor and Price used keyword searches from popular databases (SFX sources) and tested 351 full-text links. They summarized the causes of dead links as inaccurate source URL data, incomplete source URL data, inaccurate resolver knowledge base, resolver translation error, incomplete resolver target URL, no article-level linking, provider target URL translation error, incomplete provider content, and other miscellaneous reasons. Chen assessed data accuracy provided by full-text aggregators to OpenURL link resolver vendors during the early years of OpenURL linking and found these factors as causes of dead links: outdated data provided by full-text aggregators, wrong data (wrong coverage dates or indexedonly titles mistakenly considered as full-text titles), and no embargo information or incomplete embargo information (Chen, 2004). Wiley and Thomas (2009) reported similar issues and called for metadata improvement. To pressure full-text database vendors to supply more accurate title lists to link resolver vendors, Munsona advocated that libraries start using the accuracy of the vendors' full-text source lists as a criterion when they select full-text databases (Munsona, 2006). Chandler, Wiley, and LeBlanc (2011) used metadata from L'Année Philologique as examples to demonstrate that OpenURL quality needs to improve. Culling reported on a UKSG-funded research study. UKSG (originally United Kingdom Serials Group) now simply calls itself UKSG after it expanded beyond its original geography and scope. The study explored the data supply chain that has developed in recent years to facilitate the creation of link resolver knowledge bases. Through interviews and surveys, the study identified twelve issues in link resolvers and the serials supply chain. The issues related to link resolvers included inaccurate and incomplete data, unclear responsibility for data quality, lack of data standards, inbound linking issues, and OpenURL issues (Culling, 2007). Trainor and Price (2010b) called for better data in library linking and reported two industry initiatives in addressing OpenURL issues. Imler and Tomaszewski reported students' usage of SFX through observing search behaviors based on volunteer participants. One of the findings of the study was the difficulty of using the SFX software; patience and persistence are required to get to the full text of an article. This is exemplified by Imler and Tomaszewski's (2011) finding that almost one quarter of participants did not follow through and click on “Go” to get to the full text of the article. Walker gave an overview and general introduction to OpenURL and SFX linking. She pointed out that the metadata for linking included ISSN, year, volume, issue, and start page, and that OpenURL linking is also integrated with DOI/CrossRef linking (Walker, 2003). Overall, the existing literature has reported on issues of accuracy or quality of metadata used in OpenURL linking, but the literature has not yet focused on technical issues of OpenURL linking when links fail even with accurate metadata. 4. Findings from SFX Feedback Analysis Four hundred thirty-two SFX dead-link reports received via email during fall 2007 through fall 2011 were recorded in an Excel file. Of the 432 records, librarians who responded to the emails found forty-six instances where the problem could not be replicated—the links worked for the librarians. There may be various reasons why users reported problems for links that did work. One reason may be that the SFX menu does not look straightforward to some users, as Imler and Tomaszewski (2011) found in their study. Another reason may be users' browser settings and other computer settings that make viewing or printing full text difficult or impossible. Since there were no full-text link problems that could be replicated by library staff for these forty-six records, they were not used for further study of dead-link causes. The remaining 386 records were further filtered and sorted into various categories in the Excel file for dead-link analysis.
223
4.1. SFX Targets with Most Dead Links Subscriptions to full-text resources from various aggregators and publishers are activated in the SFX knowledge base in groups called targets. Major SFX targets activated for the library in this study include ABI/INFORM (ProQuest), American Chemical Society journals (full package), BioOne (full package), Cambridge University Press journals (full package), EBSCO's Academic Search Premier, EBSCO Business Source Elite, Elsevier ScienceDirect (partial), Emerald (partial), Factiva, IEEE All Society Periodicals Package (partial), Institute of Physics journal (full package), JSTOR (full package), LexisNexis Academic, OVID journals (partial), Oxford University Press journals (partial), Project MUSE (full package), Sage Journals Online (full package), Springer journals (partial), Taylor and Francis journals (partial), Wiley Online Library (full package), and Wilson Omnifile Full Text Select. One major open-access journal package that has been activated in SFX is DOAJ. During the four years of SFX feedback recorded, one major SFX target was deactivated due to cancellation, Gale's Expanded Academic ASAP. Given this list of active targets, Table 1 lists SFX targets that received the most dead-link reports. About 62 percent of the 386 dead links reported were from these eight full-text databases or journal packages, and about 27 percent of those were from the top two trouble makers (EBSCO's Academic Search Premier and Wiley Online Library). 4.2. Most Common Reasons of Dead-Link Reports The greatest reason for dead links (123 cases out of 368) was that the SFX targets actually do not have the full-text articles SFX says they have. Table 2 summarizes the four most common reasons for dead links. The top four reasons (articles not available or not accessible on SFX targets, index error, no article-level link, and articles being book reviews or other special items) account for 53.4 percent of the causes of dead links. Three of these causes are relatively simple to summarize. Index errors include incorrect data for in the following areas: ISSN, year of publication, volume number, issue number, and page numbers. For example, the article “Pynchon's World and Its Legendary Past” written by Elaine Safer is in vol. 31, no. 2 of the journal Critique. The MLA International Bibliography incorrectly lists a volume number of vol. 32. As a result, the user who clicked on the SFX target received a dead-link page. No article-level linking (linking to journal level only) is a major issue for several SFX targets, with DOAJ being the target receiving the most complaints. DOAJ is a collection of open-access journals, but many of these journals do not allow article-level linking. Users are typically taken to a journal home page, where they may get confused and think the article is not available. Book reviews and other special items in journals pose a major headache for OpenURL linking. They account for 22 (5.7 percent) of the dead links in this study. An example is the review of “Muddy Waters Rural Responses to Industrialization” by Robert Bate in the Journal of African History. The full text of the review is in JSTOR, but SFX cannot lead users to the full text, as is the case for the other twenty-one problems related to book reviews and other special items in journals. Table 1 Full-text targets with the most dead links. SFX target
# of dead link reports
Academic Search Premier Wiley Online Library DOAJ OmniFile Full Text Select LexisNexis Academic Sage Journals Online Factiva ABI/INFORM Total
53 50 29 27 24 24 17 15 239
224
X. Chen / Serials Review 38 (2012) 222–227
Management, vol. 22, no. 9. SFX interpreted the page numbers as “1, 16–1, 19” and displayed “1, 16–1, 19” on the SFX menu.
Table 2 Most common reasons for reported dead links. Causes of dead link reports
# of reports
Article not available or not accessible Index error No article-level link Articles requested were book reviews or other special items Total
123 32 29 22 206
On the other hand, the top reason for dead link reports (articles not available or not accessible in SFX targets) has various causes. In most cases, the articles are simply not available in the databases and neither are the journals. Inaccurate metadata from full-text vendors or publishers is a major cause. But other kinds of technical issues can cause problems, too. The story at the very beginning of this article, a target with a journal that has a 12-month full-text embargo without access to the 4th quarter issue of 2010 on December 16, 2011, is one technical issue SFX needs to resolve. There are numerous dead-link reports of this kind, in which SFX indicates that the 12-month embargo has passed for articles published 12 months ago, but the full-text vendors had not yet loaded these articles. Another similar technical issue is that sometimes the first issue of certain journals in some full-text databases is issue 3 or issue 4 instead of issue 1, and SFX tells users the database has all the issues for a particular year. For example, the first issue of Nephrology Nursing Journal on Academic Search Premier in 2001 was vol. 28, no. 4, and a user looking for an article in vol. 28, no. 3 was told by SFX that the article was available. The users received a bad link due to only the year, but not the volume and issue being defined for Nephrology Nursing Journal. 4.3. Other Dead Link Reasons That Deserve Attention 4.3.1. Articles from supplemental issues When articles are included in supplemental issues, dead links often occur. There are different scenarios that might cause this problem. Abnormal issue numbers or page numbers for supplemental issues often cause dead links. Some publishers add the letter S in page numbers for supplemental issues. ScienceDirect journals have S before the number, and Sage journals have S after the number. An example can be found in the article “The Impact of Nursing Interventions” in the supplemental issue of the journal Medical Care Research and Review. The following are how different databases display the volume, issue, and page numbers for the same article: • CINAHL: 64 (2): Supplement: 123S-43S • PubMed: 64 (2 Suppl): 123S-43S • CAT.INIST: vol. 64, no2, SUP, [Note(s): 123S-143S]. The unusual issue information and pagination can cause a breakdown or mismatch in linking from SFX sources to SFX targets. Another scenario is that full-text aggregators and publishers may not include supplemental issues online. An example is from Sage; this publisher did not include online the supplemental issue of volume 15 of the journal Asia-Pacific Journal of Public Health. As a result, the user looking for the article “Longevity and Diet in Okinawa, Japan” in that supplemental issue received a dead link. 4.3.2. Abnormal Volume, Issue, and Page Numbers Abnormal volume, issue and page number data, such as vol. XII, no. 1–2, and no. 2–3, also caused links to fail. One example is the Journal of Clinical Nursing, which has vol. 19, no. 15–16. The two issue numbers (no. 15–16) caused a dead link reported by a user. Some unusual page numbers may also cause a breakdown or mismatch in linking from SFX sources to SFX targets, such as the page numbers of the article “The Evolution of Medically-based Fitness Centers.” Its page numbers include pages 1 and 16–19 of the journal Health Care Strategic
4.3.3. Some Wiley journal articles have two publication dates Wiley journals may have two dates on the Wiley Web site: one is the date of online publishing and the other is the official publication date of the issue. Most indexes use the official date instead of the online publishing date, but Google Scholar sometimes uses the online publishing date. As a result, users of Google Scholar sometimes get dead links when trying to get Wiley articles. Fig. 1 illustrates the two dates listed for a Wiley journal article. 4.3.4. The intuitiveness (or the lack) of full-text link The librarians who responded to users' dead-link reports found no problem in forty-six cases; the links worked just fine for librarians in those forty-six reports. Of these forty-six cases, thirteen (28.3 percent) had Wiley Online Library and ten (21.7 percent) had Sage Journals Online as the SFX target. These two targets accounted for 50 percent of all complaints in this category. Why did users complain when the link worked just fine? One possible reason is that users may get confused or find it difficult to retrieve the full text after they were taken from the SFX menu to the SFX target Web site. On the article home page of the Wiley site, the link to the full text is identified as “Get PDF.” Is the wording “Get PDF” intuitive to users who are looking for the full text? The answer may be negative, since some users chose to report that full text was not there even if their librarians could not find anything wrong. On the other hand, the article page of Sage journals sometimes has full-text links to articles that cite the article. All the links are named “Full Text (PDF).” Fig. 2 is a screen shot of a Sage article. A faculty member clicked the “Full Text (PDF)” at the bottom of the screen and received a wrong article. The professor did not know what to do and used the SFX feedback form to report the issue. 4.3.5. DOI errors The integration of OpenURL linking with DOI/CrossRef linking is certainly a positive factor leading to article-level linking. But when journals change platform or when DOIs are wrong, it just becomes another cause for dead links. The following is an example where a publisher (Sage) registered a wrong DOI for an article: Ruth Page, “Feminist Narratology? Literary and Linguistic Perspectives on Gender and Varrativity,” Language and Literature 12, no. 1, (2003): 43–56. The DOI listed on the Sage Web site for this article is http://dx.doi. org/10.1177/096394700301200103, but its DOI on the CrossRef site is http://dx.doi.org/10.1177/0963947003012001078. SFX uses the correct DOI and therefore cannot take users to the Sage Web site that includes an incorrect DOI if CrossRef is turned on for Sage in the library's SFX administration site. A more complicated example is Journal of the American Psychiatric Nurses Association, which changed publisher from Elsevier to Sage. Elsevier's ScienceDirect keeps some older years of the journal, and Sage Journals Online has newer years, but 2002 is on both sites. The library in this study has a subscription to the journal only through Sage. A user looking for an article in a 2002 issue got the SFX menu saying the article is available on Sage, but a click on the Sage link (DOI) took the user to ScienceDirect. The title of the article is “Becoming Responsible Teens: Promoting the Health of Adolescents in Foster Care.” Of course, the user could not get the full text on ScienceDirect and reported the issue through the SFX feedback form. 4.3.6. Missing volumes and articles Some aggregators have missing volumes and articles in their digital collections. ABI/INFORM (ProQuest), Factiva, and Wilson Omnifile
X. Chen / Serials Review 38 (2012) 222–227
225
Figure 1. This Wiley article has 23 June 2006 as the online publishing date and April 2005 as the print publishing date.
Full Text Select are among these aggregated databases. For example, the vol. 1, no. 4 issue of Journal of the Knowledge Economy had not yet been indexed by ABI/INFORM by the time vol. 2, no. 4 was already indexed by ABI/INFORM in January 2012. Similarly, the entire April 1999 issue of Archives of Sexual Behavior was missing on Factiva as of January 2012, but Factiva had other 1999 issues of the journal available at the same time. Consequently, when users try to access an issue whose data are missing from the provider, they are unable to access the content.
Librarians received and responded to only 169 (39 percent) emails when they were in their offices (7:30 am–4:30 pm) and received and responded to 263 (61 percent) requests for assistance when they were out of the office. The two librarians in this study checked emails every evening, weekend day, and holiday. With a proxy server or VPN, the librarians were able to use library databases to help users, even when they were on vacation out of the country.
5. Librarians' Response to Users Feedback
There are various reasons why users could not access the full text of an article after seeing the SFX menu. The quality of the title lists supplied from full-text providers to OpenURL link resolver vendors is still a major concern as has been pointed out by existing literature. Inaccurate title lists are primarily from unstable aggregated databases such as EBSCO's Academic Search Premier, Wilson's Omnifile Full Text Select, LexisNexis Academic, Factiva, and ABI/INFORM. Aggregated databases with a stable title set such as JSTOR do not have this kind of issue. Most publishers' packages do not have this kind of issue, either, with the exception of Wiley and Sage. Inaccurate title lists lead to a simple dead-link result: the articles are simply not in the full-text databases where the OpenURL says they are. Full-text vendors and OpenURL vendors should certainly continue to strive to improve the quality of title lists. However, this study would like to call attention to the technical issues causing dead links. Abnormal volume and issue numbers (such as vol. XII and no. 1–2) are a cause of dead links. The problem can be masked if DOI is used, but DOI linking only works for targets that are publisher Web sites (CrossRef/DOI enabled targets). DOI linking does not work for a typical aggregator's database, such as EBSCO's Academic Search Premier, JSTOR, and LexisNexis Academic. The burden of this improvement is probably on the OpenURL vendors, since there is not much the index vendors and aggregated vendors can do, if for instance, publishers choose to combine two issues of a certain volume into no. 1–2.
Librarians responded to all the 432 recorded SFX dead-link emails, and managed to find 276 (out of 432, or 63.9 percent) available through full text and sent those 276 articles to users either in email attachments or by a full-text URL. Additionally, the librarians also notified users of six additional articles: five held by the library in paper or microfilm format and one free online full-text article requiring user registration. The free-but-required-registration article was from a journal in SFX's “Miscellaneous Free” target. In the remaining 150 cases, the librarians had to make apologies to the users and give them an option to request the articles through interlibrary loan (ILL). Of those 276 full-text articles or links sent to users, 231 were found from databases that were supposed to have these articles. Users could not get to the full text for various reasons, but librarians were able to retrieve them either manually or following the OpenURL. The remaining 45 articles were not available in the databases they were supposed to be in, but librarians were able to find a free online copy with Google or Google Scholar for users. 6. When were Library Users more Active Online? The SFX feedback records also find that users were more active during evenings, weekends, and holidays than during office hours.
7. Analyses and Discussions
226
X. Chen / Serials Review 38 (2012) 222–227
Figure 2. Confusing full-text links on Sage site. The correct one is on the right, and the bottom link takes users to a different article.
Similarly, an important technical issue is linking to articles in supplemental issues. Again, DOI may solve the problem of linking to the publishers' Web sites, but cannot help with aggregated databases. At present, it seems that there is no standard in indexing issue number and page numbers for articles in supplemental issues, as indicated in the examples of this study. Page numbers sometimes prefixed with an S and sometimes suffixed with an S, and various words or abbreviations of supplement such as “sup,” “suppl,” and “supplement” may be another cause of breakdown in either generating a correct OpenURL for an article or matching the article in the target. Embargo release is also a technical challenge that OpenURL linking faces. The journal Australian Psychologist has a 12-month embargo on EBSCO's Academic Search Premier and SFX acknowledged that. On December 16, 2011, SFX thought the embargo was past for issue 4 of 2010 and told users that full text from that issue was available. However, the most recent issue available on that date was issue 3 of 2010. Clearly, SFX's trigger to release embargo is not based on the actual availability of an embargoed issue but rather based on estimated month of the release of the embargoed issue. That kind of “estimate” is often not accurate and therefore causes dead links and frustrations. OpenURL link resolver vendors should find a better way to accurately reflect when an embargoed volume or issue becomes available. Book reviews and other special items in journals are another challenge to OpenURL linking. These special items normally have DOIs, so CrossRef/DOI enabled targets are probably fine, but book reviews
from aggregator's databases such as Academic Search Premier do not have DOIs and have to count on regular metadata to link. The possible cause of breakdown for a link to a book review is the complicated metadata. The author, title, and other fields are more complicated than in a typical journal article. For example, there can be two sets of authors: one is the author of the book and the other the author of the review. The burden of improvement is probably more on OpenURL link resolver vendors than on index vendors and full-text aggregators. Publishers' improvement of Web sites is an area that deserves attention. The most noticeable issue is the confusing publication dates (one online publication date and one paper publication date) for some Wiley journals. Traditional indexes normally use the paper publication date as the official date, but modern indexes such as Google Scholar can pick the online publication date as the official publication date, and users of Google Scholar sometimes get dead links when Wiley is the target. Wiley should either remove the online publication date, which is meaningless to users, or work with Google so that Google Scholar would know which date to use. Another technical issue found in this study is DOI linking breakdown due to changes in journal ownership and the publishers' Web platform. In theory, the DOI should direct users to the new/current Web site for the articles after ownership and platform changes. In reality things can get complicated and users may not get what they are looking for. Both OpenURL link resolver vendors and publishers should closely follow these changes and work together to resolve
X. Chen / Serials Review 38 (2012) 222–227
potential issues. It is always a sign of bad library service when users are the first to find out that links are not working. In addition, DOIs and other index metadata can have errors due to human or computer mistakes. Errors of this kind are not new. There is probably no error-free paper and electronic index, but index and DOI errors can more likely cause failure as well as frustration in the age of OpenURL linking. Before OpenURL came into being, wrong page numbers in paper or electronic indexes could also cause headaches, but when OpenURL linking does not work, incorrect metadata and DOIs cause bigger headaches for users who were not trained in how to retrieve articles manually. On the library service side, it is imperative for librarians to respond to users' reports in the evenings, weekends, and holidays, since that is when the users are more active online. After all, in almost two-thirds of the cases reported in this study, librarians were able to email users the full text of the desired content when they responded to users. Dead links are frustrating, but a quick email reply from a librarian with a PDF full-text attachment will certainly ease frustration, especially when the users are working under the pressure of approaching deadlines.
8. Conclusions The OpenURL link resolver is one of the greatest inventions during the past 10 or 15 years in the information industry. It has transformed the full-text searching and retrieving process. But the dead links in OpenURL linking have remained an issue. Based on user reports, this study examines dead links and divides them into different categories. In addition to full-text title list errors or quality issues, this
227
study raises technical issues that previously have not been discussed in the existing literature. These technical issues include OpenURL's handling of special journal volume and issue numbers, journal supplemental issues, embargo release dates, book reviews, DOI problems, and publishers' presentation of publication dates and full-text links on their Web sites. This study also finds that users are more active online outside of the regular office hours. The author of this study hopes that full-text vendors and publishers, OpenURL link resolver vendors, and libraries can strive to make improvements or adjustments to better serve their customers or patrons. References Chandler, A., Wiley, G, & LeBlanc, J. (2011). Towards transparent and scalable OpenURL quality metrics. D-Lib Magazine. Retrieved from http://dlib.org/dlib/march11/ chandler/03chandler.html Chen, X. (2004). Assessment of full-text sources used by serials management systems, OpenURL link resolvers, and imported E-journal MARC records. Online Information Review, 28, 428–434. Culling, J. (2007). Link resolvers and the serials supply chain: A research project sponsored by UKSG. Serials, 20(2), 111–116. Imler, B., & Tomaszewski, M. (2011). Do they “Get it”? Student usage of SFX citation linking software. College and Research Libraries, 72, 454–463. Munsona, D. (2006). Link resolvers: An overview for reference librarians. Internet Reference Services Quarterly, 11(1), 17–28. Trainor, C., & Price, J. (2010a). Chapter 3: Digging into the data: Exposing the causes of resolver failure. Library Technology Reports, 46(7), 15–26. Trainor, C., & Price, J. (2010b). Rethinking library linking: Making OpenURL better with data, data, and more data. American Libraries, 41(10), 34-34. Wakimoto, J., Walker, D., & Dabbour, K. (2006). The myths and realities of SFX in academic libraries. The Journal of Academic Librarianship, 32, 127–136. Walker, J. (2003). OpenURL and SFX linking. The Serials Librarian, 45(3), 87–100. Wiley, G., & Thomas, W. M. (2009). Improving OpenURL metadata. The Serials Librarian, 56, 282–286.
Contents lists available at SciVerse ScienceDirect
Serials Review journal homepage: www.elsevier.com/locate/serrev
Broken-Link Reports from SFX Users: How Publishers, Vendors and Libraries Can Do Better☆ Xiaotian Chen ⁎ Electronic Services Librarian and Associate Professor, Cullom-Davis Library, Bradley University, 1501W. Bradley Ave, Peoria, IL 61625, USA
a r t i c l e
i n f o
Available online 16 October 2012 Keywords: OpenURL link resolver SFX dead links
a b s t r a c t This study analyzes more than 400 SFX broken-link reports sent by users of an academic library. It raises technical issues regarding OpenURL linking in the handling of special journal volume and issue numbers, journal supplemental issues, embargo release dates, book reviews, DOIs, and other areas. It reports on full-text resources with the most broken links, causes of broken links, and the library's responses to users. It also explores how journal publishers, database vendors, and OpenURL vendors can improve the quality of their products and how librarians can better serve users. © 2012 Elsevier Inc. All rights reserved.
1. Introduction On December 16, 2011, a library user looking for the article “On Being Mindful, Emotionally Aware, and More Resilient” from Australian Psychologist, vol. 45, no. 4 (2010) hit a dead link. The article was supposed to be on EBSCO's Academic Search Premier (ASP), according to SFX, Ex Libris's OpenURL link resolver, but there was no full text. It turns out that the entire issue of vol. 45, no. 4 (2010) of this journal had not yet been loaded into EBSCO's databases. Both EBSCO and SFX acknowledged that Australian Psychologist has a 12-month embargo on Academic Search Premier and indicated that issue no. 4 of 2010 should have been available. However, full text from issue 4 of 2010 was still unavailable on December 16, 2011, even though SFX thought the embargo of one year had elapsed. Users of academic libraries encounter and report dead links from OpenURL link resolvers every week, if not every day. This study examines more than 400 SFX dead-link reports collected during fall 2007 and fall 2011 from a mid-size university in the United States. 2. Methodology When users of SFX encounter a dead link, they have the ability to report the problem to their home library through an SFX feedback form. Article citation and OpenURL are automatically populated into the feedback, so users need only to enter their name and email and submit the form. At this mid-size university, more than 400 SFX dead link reports plus responses from library staff to users have been recorded in a Microsoft Excel file. These records include the ☆ The author is grateful to Ellie Hansen of Bradley University (Emeritus) for editing this paper. ⁎ Tel.: +1 309 677 2839; fax: +1 309 677 2558. E-mail address: [email protected]. 0098-7913/$ – see front matter © 2012 Elsevier Inc. All rights reserved. http://dx.doi.org/10.1016/j.serrev.2012.09.002
day, date, and time when the dead link reports were sent, when the library responded to the users, the citation of the articles, the databases in which users began searches (SFX sources), the full-text databases the users were referred to (SFX targets), the causes of the dead link, and whether the library managed to find the full text needed for the users. The library is a mid-sized academic library from a private institution in the Midwestern United States. The library implemented SFX in the summer of 2005 and started to record SFX dead link reports in the fall 2007. Two librarians were dedicated to receive SFX feedback from email and reply to users with email. During the office hours, they reply instantly. After office hours, users get a response most of time on the same day if the feedback is sent before 9:00 pm, including evenings, weekends, and holidays. When feedback is sent late at night, the responses are sent to users the next day. With the records in an Excel file, it is easy to filter and sort them for statistical analysis. This study tries to determine and statistically summarize the SFX targets with the most dead-link reports, the causes of dead links, and the library's average response time to users. It also reports whether the library could get the full text needed instantly and send the full text to users through email. The author hopes the paper will provide information to help full-text vendors and publishers who provide data to OpenURL link resolver vendors, and link resolver vendors themselves, to provide more accurate and timely exchange of data. 3. Literature Review To date, no literature has been found that studies SFX users' dead-link reports to summarize OpenURL dead-link causes. In their general study of SFX perceptions from end-users and librarians, Wakimoto et al. found through survey and focus-group studies that users expect more full-text availability while librarians expect more link accuracy. They estimated that SFX link accuracy overall is about
X. Chen / Serials Review 38 (2012) 222–227
80 percent (Wakimoto, Walker, & Dabbour, 2006). Trainor and Price (2010a) used the Wakimoto study data as a base to perform a historical comparison, and they found that full-text linking accuracy has not improved from 2004 to 2010. Trainor and Price used keyword searches from popular databases (SFX sources) and tested 351 full-text links. They summarized the causes of dead links as inaccurate source URL data, incomplete source URL data, inaccurate resolver knowledge base, resolver translation error, incomplete resolver target URL, no article-level linking, provider target URL translation error, incomplete provider content, and other miscellaneous reasons. Chen assessed data accuracy provided by full-text aggregators to OpenURL link resolver vendors during the early years of OpenURL linking and found these factors as causes of dead links: outdated data provided by full-text aggregators, wrong data (wrong coverage dates or indexedonly titles mistakenly considered as full-text titles), and no embargo information or incomplete embargo information (Chen, 2004). Wiley and Thomas (2009) reported similar issues and called for metadata improvement. To pressure full-text database vendors to supply more accurate title lists to link resolver vendors, Munsona advocated that libraries start using the accuracy of the vendors' full-text source lists as a criterion when they select full-text databases (Munsona, 2006). Chandler, Wiley, and LeBlanc (2011) used metadata from L'Année Philologique as examples to demonstrate that OpenURL quality needs to improve. Culling reported on a UKSG-funded research study. UKSG (originally United Kingdom Serials Group) now simply calls itself UKSG after it expanded beyond its original geography and scope. The study explored the data supply chain that has developed in recent years to facilitate the creation of link resolver knowledge bases. Through interviews and surveys, the study identified twelve issues in link resolvers and the serials supply chain. The issues related to link resolvers included inaccurate and incomplete data, unclear responsibility for data quality, lack of data standards, inbound linking issues, and OpenURL issues (Culling, 2007). Trainor and Price (2010b) called for better data in library linking and reported two industry initiatives in addressing OpenURL issues. Imler and Tomaszewski reported students' usage of SFX through observing search behaviors based on volunteer participants. One of the findings of the study was the difficulty of using the SFX software; patience and persistence are required to get to the full text of an article. This is exemplified by Imler and Tomaszewski's (2011) finding that almost one quarter of participants did not follow through and click on “Go” to get to the full text of the article. Walker gave an overview and general introduction to OpenURL and SFX linking. She pointed out that the metadata for linking included ISSN, year, volume, issue, and start page, and that OpenURL linking is also integrated with DOI/CrossRef linking (Walker, 2003). Overall, the existing literature has reported on issues of accuracy or quality of metadata used in OpenURL linking, but the literature has not yet focused on technical issues of OpenURL linking when links fail even with accurate metadata. 4. Findings from SFX Feedback Analysis Four hundred thirty-two SFX dead-link reports received via email during fall 2007 through fall 2011 were recorded in an Excel file. Of the 432 records, librarians who responded to the emails found forty-six instances where the problem could not be replicated—the links worked for the librarians. There may be various reasons why users reported problems for links that did work. One reason may be that the SFX menu does not look straightforward to some users, as Imler and Tomaszewski (2011) found in their study. Another reason may be users' browser settings and other computer settings that make viewing or printing full text difficult or impossible. Since there were no full-text link problems that could be replicated by library staff for these forty-six records, they were not used for further study of dead-link causes. The remaining 386 records were further filtered and sorted into various categories in the Excel file for dead-link analysis.
223
4.1. SFX Targets with Most Dead Links Subscriptions to full-text resources from various aggregators and publishers are activated in the SFX knowledge base in groups called targets. Major SFX targets activated for the library in this study include ABI/INFORM (ProQuest), American Chemical Society journals (full package), BioOne (full package), Cambridge University Press journals (full package), EBSCO's Academic Search Premier, EBSCO Business Source Elite, Elsevier ScienceDirect (partial), Emerald (partial), Factiva, IEEE All Society Periodicals Package (partial), Institute of Physics journal (full package), JSTOR (full package), LexisNexis Academic, OVID journals (partial), Oxford University Press journals (partial), Project MUSE (full package), Sage Journals Online (full package), Springer journals (partial), Taylor and Francis journals (partial), Wiley Online Library (full package), and Wilson Omnifile Full Text Select. One major open-access journal package that has been activated in SFX is DOAJ. During the four years of SFX feedback recorded, one major SFX target was deactivated due to cancellation, Gale's Expanded Academic ASAP. Given this list of active targets, Table 1 lists SFX targets that received the most dead-link reports. About 62 percent of the 386 dead links reported were from these eight full-text databases or journal packages, and about 27 percent of those were from the top two trouble makers (EBSCO's Academic Search Premier and Wiley Online Library). 4.2. Most Common Reasons of Dead-Link Reports The greatest reason for dead links (123 cases out of 368) was that the SFX targets actually do not have the full-text articles SFX says they have. Table 2 summarizes the four most common reasons for dead links. The top four reasons (articles not available or not accessible on SFX targets, index error, no article-level link, and articles being book reviews or other special items) account for 53.4 percent of the causes of dead links. Three of these causes are relatively simple to summarize. Index errors include incorrect data for in the following areas: ISSN, year of publication, volume number, issue number, and page numbers. For example, the article “Pynchon's World and Its Legendary Past” written by Elaine Safer is in vol. 31, no. 2 of the journal Critique. The MLA International Bibliography incorrectly lists a volume number of vol. 32. As a result, the user who clicked on the SFX target received a dead-link page. No article-level linking (linking to journal level only) is a major issue for several SFX targets, with DOAJ being the target receiving the most complaints. DOAJ is a collection of open-access journals, but many of these journals do not allow article-level linking. Users are typically taken to a journal home page, where they may get confused and think the article is not available. Book reviews and other special items in journals pose a major headache for OpenURL linking. They account for 22 (5.7 percent) of the dead links in this study. An example is the review of “Muddy Waters Rural Responses to Industrialization” by Robert Bate in the Journal of African History. The full text of the review is in JSTOR, but SFX cannot lead users to the full text, as is the case for the other twenty-one problems related to book reviews and other special items in journals. Table 1 Full-text targets with the most dead links. SFX target
# of dead link reports
Academic Search Premier Wiley Online Library DOAJ OmniFile Full Text Select LexisNexis Academic Sage Journals Online Factiva ABI/INFORM Total
53 50 29 27 24 24 17 15 239
224
X. Chen / Serials Review 38 (2012) 222–227
Management, vol. 22, no. 9. SFX interpreted the page numbers as “1, 16–1, 19” and displayed “1, 16–1, 19” on the SFX menu.
Table 2 Most common reasons for reported dead links. Causes of dead link reports
# of reports
Article not available or not accessible Index error No article-level link Articles requested were book reviews or other special items Total
123 32 29 22 206
On the other hand, the top reason for dead link reports (articles not available or not accessible in SFX targets) has various causes. In most cases, the articles are simply not available in the databases and neither are the journals. Inaccurate metadata from full-text vendors or publishers is a major cause. But other kinds of technical issues can cause problems, too. The story at the very beginning of this article, a target with a journal that has a 12-month full-text embargo without access to the 4th quarter issue of 2010 on December 16, 2011, is one technical issue SFX needs to resolve. There are numerous dead-link reports of this kind, in which SFX indicates that the 12-month embargo has passed for articles published 12 months ago, but the full-text vendors had not yet loaded these articles. Another similar technical issue is that sometimes the first issue of certain journals in some full-text databases is issue 3 or issue 4 instead of issue 1, and SFX tells users the database has all the issues for a particular year. For example, the first issue of Nephrology Nursing Journal on Academic Search Premier in 2001 was vol. 28, no. 4, and a user looking for an article in vol. 28, no. 3 was told by SFX that the article was available. The users received a bad link due to only the year, but not the volume and issue being defined for Nephrology Nursing Journal. 4.3. Other Dead Link Reasons That Deserve Attention 4.3.1. Articles from supplemental issues When articles are included in supplemental issues, dead links often occur. There are different scenarios that might cause this problem. Abnormal issue numbers or page numbers for supplemental issues often cause dead links. Some publishers add the letter S in page numbers for supplemental issues. ScienceDirect journals have S before the number, and Sage journals have S after the number. An example can be found in the article “The Impact of Nursing Interventions” in the supplemental issue of the journal Medical Care Research and Review. The following are how different databases display the volume, issue, and page numbers for the same article: • CINAHL: 64 (2): Supplement: 123S-43S • PubMed: 64 (2 Suppl): 123S-43S • CAT.INIST: vol. 64, no2, SUP, [Note(s): 123S-143S]. The unusual issue information and pagination can cause a breakdown or mismatch in linking from SFX sources to SFX targets. Another scenario is that full-text aggregators and publishers may not include supplemental issues online. An example is from Sage; this publisher did not include online the supplemental issue of volume 15 of the journal Asia-Pacific Journal of Public Health. As a result, the user looking for the article “Longevity and Diet in Okinawa, Japan” in that supplemental issue received a dead link. 4.3.2. Abnormal Volume, Issue, and Page Numbers Abnormal volume, issue and page number data, such as vol. XII, no. 1–2, and no. 2–3, also caused links to fail. One example is the Journal of Clinical Nursing, which has vol. 19, no. 15–16. The two issue numbers (no. 15–16) caused a dead link reported by a user. Some unusual page numbers may also cause a breakdown or mismatch in linking from SFX sources to SFX targets, such as the page numbers of the article “The Evolution of Medically-based Fitness Centers.” Its page numbers include pages 1 and 16–19 of the journal Health Care Strategic
4.3.3. Some Wiley journal articles have two publication dates Wiley journals may have two dates on the Wiley Web site: one is the date of online publishing and the other is the official publication date of the issue. Most indexes use the official date instead of the online publishing date, but Google Scholar sometimes uses the online publishing date. As a result, users of Google Scholar sometimes get dead links when trying to get Wiley articles. Fig. 1 illustrates the two dates listed for a Wiley journal article. 4.3.4. The intuitiveness (or the lack) of full-text link The librarians who responded to users' dead-link reports found no problem in forty-six cases; the links worked just fine for librarians in those forty-six reports. Of these forty-six cases, thirteen (28.3 percent) had Wiley Online Library and ten (21.7 percent) had Sage Journals Online as the SFX target. These two targets accounted for 50 percent of all complaints in this category. Why did users complain when the link worked just fine? One possible reason is that users may get confused or find it difficult to retrieve the full text after they were taken from the SFX menu to the SFX target Web site. On the article home page of the Wiley site, the link to the full text is identified as “Get PDF.” Is the wording “Get PDF” intuitive to users who are looking for the full text? The answer may be negative, since some users chose to report that full text was not there even if their librarians could not find anything wrong. On the other hand, the article page of Sage journals sometimes has full-text links to articles that cite the article. All the links are named “Full Text (PDF).” Fig. 2 is a screen shot of a Sage article. A faculty member clicked the “Full Text (PDF)” at the bottom of the screen and received a wrong article. The professor did not know what to do and used the SFX feedback form to report the issue. 4.3.5. DOI errors The integration of OpenURL linking with DOI/CrossRef linking is certainly a positive factor leading to article-level linking. But when journals change platform or when DOIs are wrong, it just becomes another cause for dead links. The following is an example where a publisher (Sage) registered a wrong DOI for an article: Ruth Page, “Feminist Narratology? Literary and Linguistic Perspectives on Gender and Varrativity,” Language and Literature 12, no. 1, (2003): 43–56. The DOI listed on the Sage Web site for this article is http://dx.doi. org/10.1177/096394700301200103, but its DOI on the CrossRef site is http://dx.doi.org/10.1177/0963947003012001078. SFX uses the correct DOI and therefore cannot take users to the Sage Web site that includes an incorrect DOI if CrossRef is turned on for Sage in the library's SFX administration site. A more complicated example is Journal of the American Psychiatric Nurses Association, which changed publisher from Elsevier to Sage. Elsevier's ScienceDirect keeps some older years of the journal, and Sage Journals Online has newer years, but 2002 is on both sites. The library in this study has a subscription to the journal only through Sage. A user looking for an article in a 2002 issue got the SFX menu saying the article is available on Sage, but a click on the Sage link (DOI) took the user to ScienceDirect. The title of the article is “Becoming Responsible Teens: Promoting the Health of Adolescents in Foster Care.” Of course, the user could not get the full text on ScienceDirect and reported the issue through the SFX feedback form. 4.3.6. Missing volumes and articles Some aggregators have missing volumes and articles in their digital collections. ABI/INFORM (ProQuest), Factiva, and Wilson Omnifile
X. Chen / Serials Review 38 (2012) 222–227
225
Figure 1. This Wiley article has 23 June 2006 as the online publishing date and April 2005 as the print publishing date.
Full Text Select are among these aggregated databases. For example, the vol. 1, no. 4 issue of Journal of the Knowledge Economy had not yet been indexed by ABI/INFORM by the time vol. 2, no. 4 was already indexed by ABI/INFORM in January 2012. Similarly, the entire April 1999 issue of Archives of Sexual Behavior was missing on Factiva as of January 2012, but Factiva had other 1999 issues of the journal available at the same time. Consequently, when users try to access an issue whose data are missing from the provider, they are unable to access the content.
Librarians received and responded to only 169 (39 percent) emails when they were in their offices (7:30 am–4:30 pm) and received and responded to 263 (61 percent) requests for assistance when they were out of the office. The two librarians in this study checked emails every evening, weekend day, and holiday. With a proxy server or VPN, the librarians were able to use library databases to help users, even when they were on vacation out of the country.
5. Librarians' Response to Users Feedback
There are various reasons why users could not access the full text of an article after seeing the SFX menu. The quality of the title lists supplied from full-text providers to OpenURL link resolver vendors is still a major concern as has been pointed out by existing literature. Inaccurate title lists are primarily from unstable aggregated databases such as EBSCO's Academic Search Premier, Wilson's Omnifile Full Text Select, LexisNexis Academic, Factiva, and ABI/INFORM. Aggregated databases with a stable title set such as JSTOR do not have this kind of issue. Most publishers' packages do not have this kind of issue, either, with the exception of Wiley and Sage. Inaccurate title lists lead to a simple dead-link result: the articles are simply not in the full-text databases where the OpenURL says they are. Full-text vendors and OpenURL vendors should certainly continue to strive to improve the quality of title lists. However, this study would like to call attention to the technical issues causing dead links. Abnormal volume and issue numbers (such as vol. XII and no. 1–2) are a cause of dead links. The problem can be masked if DOI is used, but DOI linking only works for targets that are publisher Web sites (CrossRef/DOI enabled targets). DOI linking does not work for a typical aggregator's database, such as EBSCO's Academic Search Premier, JSTOR, and LexisNexis Academic. The burden of this improvement is probably on the OpenURL vendors, since there is not much the index vendors and aggregated vendors can do, if for instance, publishers choose to combine two issues of a certain volume into no. 1–2.
Librarians responded to all the 432 recorded SFX dead-link emails, and managed to find 276 (out of 432, or 63.9 percent) available through full text and sent those 276 articles to users either in email attachments or by a full-text URL. Additionally, the librarians also notified users of six additional articles: five held by the library in paper or microfilm format and one free online full-text article requiring user registration. The free-but-required-registration article was from a journal in SFX's “Miscellaneous Free” target. In the remaining 150 cases, the librarians had to make apologies to the users and give them an option to request the articles through interlibrary loan (ILL). Of those 276 full-text articles or links sent to users, 231 were found from databases that were supposed to have these articles. Users could not get to the full text for various reasons, but librarians were able to retrieve them either manually or following the OpenURL. The remaining 45 articles were not available in the databases they were supposed to be in, but librarians were able to find a free online copy with Google or Google Scholar for users. 6. When were Library Users more Active Online? The SFX feedback records also find that users were more active during evenings, weekends, and holidays than during office hours.
7. Analyses and Discussions
226
X. Chen / Serials Review 38 (2012) 222–227
Figure 2. Confusing full-text links on Sage site. The correct one is on the right, and the bottom link takes users to a different article.
Similarly, an important technical issue is linking to articles in supplemental issues. Again, DOI may solve the problem of linking to the publishers' Web sites, but cannot help with aggregated databases. At present, it seems that there is no standard in indexing issue number and page numbers for articles in supplemental issues, as indicated in the examples of this study. Page numbers sometimes prefixed with an S and sometimes suffixed with an S, and various words or abbreviations of supplement such as “sup,” “suppl,” and “supplement” may be another cause of breakdown in either generating a correct OpenURL for an article or matching the article in the target. Embargo release is also a technical challenge that OpenURL linking faces. The journal Australian Psychologist has a 12-month embargo on EBSCO's Academic Search Premier and SFX acknowledged that. On December 16, 2011, SFX thought the embargo was past for issue 4 of 2010 and told users that full text from that issue was available. However, the most recent issue available on that date was issue 3 of 2010. Clearly, SFX's trigger to release embargo is not based on the actual availability of an embargoed issue but rather based on estimated month of the release of the embargoed issue. That kind of “estimate” is often not accurate and therefore causes dead links and frustrations. OpenURL link resolver vendors should find a better way to accurately reflect when an embargoed volume or issue becomes available. Book reviews and other special items in journals are another challenge to OpenURL linking. These special items normally have DOIs, so CrossRef/DOI enabled targets are probably fine, but book reviews
from aggregator's databases such as Academic Search Premier do not have DOIs and have to count on regular metadata to link. The possible cause of breakdown for a link to a book review is the complicated metadata. The author, title, and other fields are more complicated than in a typical journal article. For example, there can be two sets of authors: one is the author of the book and the other the author of the review. The burden of improvement is probably more on OpenURL link resolver vendors than on index vendors and full-text aggregators. Publishers' improvement of Web sites is an area that deserves attention. The most noticeable issue is the confusing publication dates (one online publication date and one paper publication date) for some Wiley journals. Traditional indexes normally use the paper publication date as the official date, but modern indexes such as Google Scholar can pick the online publication date as the official publication date, and users of Google Scholar sometimes get dead links when Wiley is the target. Wiley should either remove the online publication date, which is meaningless to users, or work with Google so that Google Scholar would know which date to use. Another technical issue found in this study is DOI linking breakdown due to changes in journal ownership and the publishers' Web platform. In theory, the DOI should direct users to the new/current Web site for the articles after ownership and platform changes. In reality things can get complicated and users may not get what they are looking for. Both OpenURL link resolver vendors and publishers should closely follow these changes and work together to resolve
X. Chen / Serials Review 38 (2012) 222–227
potential issues. It is always a sign of bad library service when users are the first to find out that links are not working. In addition, DOIs and other index metadata can have errors due to human or computer mistakes. Errors of this kind are not new. There is probably no error-free paper and electronic index, but index and DOI errors can more likely cause failure as well as frustration in the age of OpenURL linking. Before OpenURL came into being, wrong page numbers in paper or electronic indexes could also cause headaches, but when OpenURL linking does not work, incorrect metadata and DOIs cause bigger headaches for users who were not trained in how to retrieve articles manually. On the library service side, it is imperative for librarians to respond to users' reports in the evenings, weekends, and holidays, since that is when the users are more active online. After all, in almost two-thirds of the cases reported in this study, librarians were able to email users the full text of the desired content when they responded to users. Dead links are frustrating, but a quick email reply from a librarian with a PDF full-text attachment will certainly ease frustration, especially when the users are working under the pressure of approaching deadlines.
8. Conclusions The OpenURL link resolver is one of the greatest inventions during the past 10 or 15 years in the information industry. It has transformed the full-text searching and retrieving process. But the dead links in OpenURL linking have remained an issue. Based on user reports, this study examines dead links and divides them into different categories. In addition to full-text title list errors or quality issues, this
227
study raises technical issues that previously have not been discussed in the existing literature. These technical issues include OpenURL's handling of special journal volume and issue numbers, journal supplemental issues, embargo release dates, book reviews, DOI problems, and publishers' presentation of publication dates and full-text links on their Web sites. This study also finds that users are more active online outside of the regular office hours. The author of this study hopes that full-text vendors and publishers, OpenURL link resolver vendors, and libraries can strive to make improvements or adjustments to better serve their customers or patrons. References Chandler, A., Wiley, G, & LeBlanc, J. (2011). Towards transparent and scalable OpenURL quality metrics. D-Lib Magazine. Retrieved from http://dlib.org/dlib/march11/ chandler/03chandler.html Chen, X. (2004). Assessment of full-text sources used by serials management systems, OpenURL link resolvers, and imported E-journal MARC records. Online Information Review, 28, 428–434. Culling, J. (2007). Link resolvers and the serials supply chain: A research project sponsored by UKSG. Serials, 20(2), 111–116. Imler, B., & Tomaszewski, M. (2011). Do they “Get it”? Student usage of SFX citation linking software. College and Research Libraries, 72, 454–463. Munsona, D. (2006). Link resolvers: An overview for reference librarians. Internet Reference Services Quarterly, 11(1), 17–28. Trainor, C., & Price, J. (2010a). Chapter 3: Digging into the data: Exposing the causes of resolver failure. Library Technology Reports, 46(7), 15–26. Trainor, C., & Price, J. (2010b). Rethinking library linking: Making OpenURL better with data, data, and more data. American Libraries, 41(10), 34-34. Wakimoto, J., Walker, D., & Dabbour, K. (2006). The myths and realities of SFX in academic libraries. The Journal of Academic Librarianship, 32, 127–136. Walker, J. (2003). OpenURL and SFX linking. The Serials Librarian, 45(3), 87–100. Wiley, G., & Thomas, W. M. (2009). Improving OpenURL metadata. The Serials Librarian, 56, 282–286.