Developing and Managing Electronic Collections: The Essentials

Selecting and Evaluating E-resources for Libraries

THIS CHAPTER EXPLORES selection and management of e-resources from the perspective of those responsible for these tasks. The lines between the types of e-resources available to libraries are blurring, and many similar decisions apply regardless of type of content. This chapter addresses key decision points and criteria that guide choices for all types of e-content and concludes with an examination of the components of making the decision to cancel or remove an e-resource.

Selection Criteria

The most basic guideline for selecting e-content is to choose content that meets user needs and advances the mission of the library, but making that decision involves several considerations. These considerations are most effectively addressed in a collections policy or policies, which can both serve as a resource for library staff and inform users. A collections policy likely spells out general selection criteria and perhaps those specific to e-resources. Some libraries also have supplemental sections or policies addressing e-resources. These might cover such topics as how the library handles licenses (who negotiates them, who has authority to sign them, etc.) and services, conditions, and clauses that are required. The latter may apply to any aspect of the license, from a mandated definition of authorized users to no restrictions on interlibrary loan (ILL) to a guarantee of perpetual access. If a library is moving toward e-preferred as a replacement for print, this section or supplemental policy explains the conditions that must be met to acquire the e-content in lieu of print.

The basic criteria that librarians apply when selecting resources remain for the most part constant, regardless of the medium of delivery. These include1

• content or subject

• cost

• language

• currency

• veracity

• writing style (e.g., well written, easy to read, aesthetic aspects)

• completeness and scope of treatment

• reputation, credentials, or authoritativeness of author, publisher, editor, reviewers

• geographic coverage

• quality of scholarship

• relation to curriculum and learning outcomes

• frequency the title is referenced in bibliographies or citations

• reading or user level to which content is directed

• comprehensiveness and breadth

• frequency of updates or revisions

• access points (e.g., indexes, level of detail in the table of contents)

• ease of use

• external resources that index the publication

• physical quality (e.g., illustrations, paper and binding, format, typography, durability, visual and audio characteristics)

• uniqueness of content, capabilities, or features

• availability of equipment required for hearing or viewing audiovisual material

Not all criteria apply to all items or to all categories of materials, and additional criteria may apply to some formats, such as the quality of art in children’s books.

E-resources have additional criteria that guide their selection. These include consideration of

• provider business model

• persistency of content

• functionality of the user interface

• ease of authentication

• accessibility

• local service implications and local physical and logistical requirements

• OpenURL compliance

• output options

• compatibility with e-readers and other mobile devices

• duplication or replacement of existing library resources

• availability of data to measure use and effectiveness

• availability of descriptive metadata for local use

• licensing and contractual terms, limitations, and obligations

Currency of e-books has an added aspect because many publishers delay release of the e-book version of a title until after the print version has been available for a period of time. Libraries should be attentive to this if they make decisions about e-preferred instead of preference for print books.

Provider Business Model

According to Amit and Zott, “A business model depicts the content, structure, and governance of transactions designed so as to create value through the exploitation of business opportunities.”2 The business models used by e-content providers define what they create and deliver, conditions of delivery and access, and how they generate revenue. Content providers can be publishers, vendors, aggregators and other agents, and even individuals. Providers aim for business models that generate revenue, protect content from piracy, and appeal to libraries. The variability of business models challenges librarians as they seek content that matches local needs and expectations at an affordable price.

Publishers offer access to their own publications, which may be journals, books, or other formats. Vendors are suppliers through which libraries obtain books, serials, other materials, and services instead of dealing directly with a publisher. Many vendors offer both print books and e-books. Agents are companies that act as intermediaries between a library and a publisher, such as subscription agents or book vendors. Aggregators are third-party providers that combine the full text of journals, articles, or books originally published by multiple publishers with a common interface or search engine and generally offer e-content in packages.

Business models have various elements, all of which merit consideration. One important decision point is whether the library is paying to access the content for a period of time (e.g., subscribing to a journal or a package of e-content for a year) or purchasing the content (e.g., purchasing journal backfiles or e-books). Both are governed by licenses. If the library enters into an annual contract or subscription for access, another point for consideration is whether the library owns the content to which it subscribed for the term of the subscription. Usually, the answer is no, but this may be addressed in the license agreement.

In addition to the obvious consideration of initial cost, most e-content carries a continuing financial commitment to renew the license and, if applicable, the service agreement. Some providers offer pricing caps—a promise not to increase the yearly cost more than a set percentage annually—if the library commits to a multiyear subscription. In some cases this cap is contingent on not reducing expenditures. In other words, the provider guarantees to limit price increases if the library agrees not to cancel journal titles or reduce the number of books purchased. Big Deals, the large bundled packages of journals from a single publisher, provided in a multiyear agreement, have been debated since they were first offered. Some libraries find them cost effective and appreciate the expanded title access; others are unhappy that they are paying for lesser-used journals and do not find the overall cost justified. Popular with consortia, Big Deals protect the publishers’ revenue stream by extending the length of the contract and restricting cancellations. One aspect that libraries can find attractive is that Big Deals offers predictability of pricing for the duration of the agreement.

Cost also varies according to the business model. Access-based pricing is determined by the number of simultaneous users for which the library contracts. Beyond that maximum, prospective users are turned away. Site licenses provide for unlimited users and are based on the number of potential users (e.g., FTE students, faculty, and staff; number of researchers; citizens in a county). They may be further limited by geographic locations, IP addresses, or both. This model is attractive for content that will have high use. Another option is pay-per-view (pay-per-use, pay-by-the-drink), in which either the library or the individual pays for the content. Libraries often take this approach for journals if they believe that the total costs of articles from a title in one year will be less than the cost of the annual subscription. Some e-books are available in a similar pay-per-view model. One appealing aspect of this business model is that it allows an unlimited number of simultaneous users. On the negative side, this model can be difficult for libraries to budget. Libraries either prepay for a block of vouchers or tokens or are billed monthly; funds allocated for this service can be expended quickly. Libraries need to understand clearly the pricing schemes available and their longer-term implications. Choosing the appropriate pricing model is important.

The business model for e-journals, reference resources, and indexing and abstracting resources has stabilized in recent years. Although libraries may not always be satisfied with the options offered, they are generally understood. The business model for e-books is in flux and volatile. Many publishers do not have a clear business model that meets their goals of protecting revenue and intellectual property. Some purchasing models are similar to those for e-journals—pay-per-view, bundled (often subject collections), and patron-driven acquisition in which a title is purchased only when it is accessed (usually a predetermined number of times). Some publishers refuse to make their e-books available for library lending. Others limit access to their own integrated online platform. Some publishers also make their books available through selected vendors and aggregators. Again, this is a dynamic, rapidly changing area. If a library is interested in books from a specific publisher, checking with both the publisher and vendors and aggregators with which the library is already working is advisable.

In addition to explicit lending limitations that publishers may place on their e-books, DRM technology limits the devices on which e-books can be read, further complicating the situation for libraries loaning e-books to patrons. A 2011 survey on digital book publishing by the Association of American University Presses found that 84 percent of respondents were either very interested or interested in new business models.3 Until publishers settle on a business model for e-books, libraries need to monitor the e-book environment carefully.

Digital Rights Management

DRM technologies give copyright owners control over how digital content and services may be used. These technologies protect electronically accessible material from unauthorized use through software or hardware and establish the circumstances under which users can access the e-content. In addition to controlling simple access to digital materials, DRM can control specific activities (e.g., printing, copying, saving e-content) and can limit the number of times an activity can be performed. A license or some type of key is required to access DRM-controlled content. The controls enforced by DRM technologies can correspond to license terms or be more stringent. As is the case with license terms, DRM can impose limitations on use that do not conform to U.S. copyright law governing fair use of library materials.

Persistency of Content

This criterion has two components: the extent to which e-content remains unchanged during the term of the license, and the provision of permanent access to the e-content after the library terminates the license or the provider stops offering the product or ceases operation. Both areas should be addressed in the license.

Persistency of e-content during the license term is primarily an issue with aggregators, who license content from publishers. Aggregators may offer a package of titles, but often they cannot guarantee that the collections of titles will remain constant. This is a consideration when libraries cancel print journal subscriptions and depend on e-journals in aggregator packages. Publishers can pull titles from packages for various reasons. They may opt to offer a title only through their proprietary websites or fail to reach an acceptable agreement with the aggregator. Some publishers question the extent to which their intellectual property or revenue stream is protected. In fall 2011, Penguin pulled its books from the aggregator OverDrive because OverDrive distributed Penguin’s e-books to Amazon’s Kindle Lending Library, which did not have a contract with Penguin allowing them to do so. Penguin stated, “Due to new concerns about the security of our digital editions, we find it necessary to delay the availability of our new titles in the digital format while we resolve these concerns with our business partners.”4 Later, Penguin announced that libraries would have continued access to titles to which they already had access, but not to new titles. Furthermore, Penguin e-books loaded for reading on Kindle devices would have to be downloaded to a computer and then transferred to the device over a USB connection, eliminating the convenience of borrowing e-books from a mobile device.

Libraries generally aim for some means of affordable, practical, perpetual, or permanent access to the licensed content. This is related to the issue of whether the library is leasing access during the term of the contract or purchasing the content. A license may grant the library the right to create backup copies for preservation purposes, but this means the library must have an infrastructure in place to host and access the content. In other cases, the content provider makes a commitment to archive content through an entity or service, such as Lots of Copies Keep Stuff Safe (LOCKSS), Controlled LOCKSS (CLOCKSS), Portico, or a trusted third party that has permanent archival responsibility for the resource if the publisher ceases the publication or goes out of business.

In some cases, the library purchases the content and, therefore, owns it in perpetuity. Two models are possible. In one the library acquires the content, mounts it and manages discovery and access locally, and takes responsibility for preserving it. In the other the library purchases the content and pays the provider an annual platform maintenance fee to support ongoing hosting of the e-content and provision of the interface. This can apply to most types of content, from e-books to journal backfiles. Random House’s January 2012 announcement that raised the price of its e-books to library aggregators (OverDrive, 3M, Ingram, etc.) stated that, once a library bought a book, it could loan it an unlimited number of times and never have to pay for it again—that is, perpetual ownership.5

Functionality of the User Interface

A user interface is the system though which users interact with the content. Libraries and users want an interface that is easy, efficient, and effective. Interfaces are native to resources, generally proprietary, and vary from platform to platform. An interface should be intuitive and provide context-sensitive help screens as well as complete and helpful documentation. The interface should use clear, consistent terminology. Users should be able to determine their status (where they are and where to go next) easily, and the interface should offer easy navigation. This might involve tables of contents and indexes linking to text, ability to page forward and backward, a “jump to” page feature, or a capability to browse the text, table of contents, and index. Viewing and manipulating results should be easy. The interface should offer a spell-check and, ideally, suggest alternatives to misspelled words.

The search options should be self-explanatory, readily apparent, and offer both simple and advanced searching. The user should be able to limit searching by date, publication type, language, and target audience. When it is desirable, the user should be able to limit searching to peer-reviewed publications. Advanced searching should support Boolean searching and searching by exact phrase, keywords, author, title, series, ISBN, date, subjects, or classification. The interface should support saving searches and searching within search results. The user should be able to limit searches to resources that are full text and to which the user’s library provides access.

Comparing interfaces to the same content involves contrasting the functionality of each and comparing the costs. For example, ERIC, a bibliographic and full-text database of education research and information sponsored by the Institute of Education Sciences of the U.S. Department of Education, is available free on the Internet. It is also available from ProQuest, EBSCOhost, and OCLC. Each version has a different interface. Some libraries consider paying more for the same e-content if they believe the interface is better or, perhaps, because they are already licensing products that use that interface. The need for one uniform user interface or a limited number of interfaces may be an important factor.

One desirable function in a user interface is an alert service. This is the ability to set up customized subject or search profiles through which users are notified when new content that matches their profiles becomes available.

Most interfaces incorporate DRM, which varies from platform to platform. One typical example is limiting the reader devices to which content can be downloaded and requiring the use of proprietary software to access the content. Many library users find this frustrating and confusing, especially when the options and limitations are not immediately obvious.

Many providers offer trials for a set period of time before the library makes a selection decision. These are an effective way to test the interface and other aspects of the product. Libraries often limit access to the trial to library staff or a small set of selected users. Opening up the trial to the entire user community can raise expectations that will not be met if the library opts not to license the product. If a trial period is not available, the library should insist on some other opportunity to explore the interface and test its functionality.

Ease of Authentication

Authentication is the process by which authorized users are verified or validated. Authorized users are defined in the license clause that describes who has permission to access and use the e-content. Authentication is the security measure used by the computer system to confirm that a user is, as nearly as can be determined, who he or she claims to be. Typical means of authentication use unique information that only the user knows or to which the user has access, such as a user name and password, personal identification number (PIN) assigned by the ILS, employee ID number, or a digital signature. Because most users have many instances of passwords, PINs, and signatures, simplifying the sign-on process has become a priority for libraries.

One simplification is to avoid the sign-on process altogether by, instead, getting the e-content provider to agree to accept any user from within a given range of IP addresses as a legitimate user. Libraries give the provider a range of IP addresses on the library’s network, and the provider accepts all traffic from that range. Systems such as EZproxy (www.oclc.org/ezproxy) can extend such access outside the library’s network by first authenticating users via a local user database and then allowing their remote systems to masquerade as systems on the library network.

Other authentication options attempt to simplify the situation for users by providing a single sign-on solution. This allows users to manage one user name and password that can then be the basis of their authentication on multiple provider systems. Shibboleth is a standards-based, open-source identity management system that federates identities so that a user’s credentials from one security domain, such as a university, can be trusted by other organizations, such as e-content providers. Central Authentication Services (CAS, www.jasig.org/cas) and OpenAthens (www.openathens.net) are other attempts to make single sign-on viable. In the wider web community, OAuth (http://oauth.net) approaches single sign-on by allowing users to leverage accounts they already have with the likes of Google and Facebook to sign on to other services.

An important consideration when evaluating an authentication process is the ability to protect user privacy. Libraries can require a clause in licenses that prohibits content providers from collecting and using information about individual users of its products, including information about the content of a user’s searches.

Regardless of the type of authentication process, it should be clear and simple and involve as few clicks to navigate as possible. Ease of authentication is an important consideration.

Accessibility

Accessibility of a resource covers several areas that fall into two broad categories: technical and design components that increase accessibility for all users, and ease of access for users with disabilities. When assessing technical and design components, the library evaluates such factors as composition and organization, navigational features, adherence to standards, and system integrity (i.e., stability and accessibility of the hosting system). Consideration should be given to how accessible the resource is to users on a technical level. What are the technical requirements of the resource in terms of browser requirements, software, hardware, service, and authentication? If users do not have the appropriate software or hardware, the resource is inaccessible. Does content offer interlinking with other resources and content by linking into and out of resources? The implementation of an OpenURL link resolver can help the user find the full text of an article and identify other pertinent resources. However, users cannot take advantage of this technology if the library does not offer OpenURL enabling.

The resource should be accessible to individuals with disabilities. Some libraries are required to comply with Section 508 of the American with Disabilities Act (ADA), which mandates that U.S. federal agencies’ electronic and information technology be accessible to people with disabilities.6 Libraries that are part of entities that receive federal funds are required to make programs and communication accessible under Section 504 of the 1973 Rehabilitation Act.7 In May 2012, blind patrons of the Free Library of Philadelphia sued the library because it instituted an e-reader program using e-readers that are inaccessible to the print-disabled.8 States also may have applicable laws.

Academic libraries may be able to tap a campus disability services unit to help with these issues. Public libraries may have access to a local government office or agency responsible for ADA compliance and accommodating individuals with disabilities. School systems usually have a department or individual responsible for providing reasonable appropriate accommodations for disabled youth. The interface should be accessible or the library should provide supplemental adaptive technology (e.g., text-to-speech software) or personal services to assist disabled users. Ideally, the user should be able to zoom and increase font size. Images and graphics should have alt tags or long descriptions. The ability to bypass a row of navigation links by jumping to the start of the web page content is desirable. E-books that are presented in an accessible format on an e-book reader should allow the user to choose to read the book using text-to-speech software, braille, or magnification.

Libraries may not be able to select e-resources that address all accessibility concerns, but they should consider them when making choices. At a minimum, libraries should be prepared to offer alternative solutions to meet the needs of those with access issues.

Local Service Implications and Local Physical and Logistical Requirements

These criteria address what the library needs to provide access to and support for the resource. The library should assess the extent to which local guides and support tools must be developed, for example, a web page that offers tips and FAQ, an online tutorial, or a printed help sheet. What skill sets do library staff need to help users? To what extent should staff understand the hardware and software that the library must provide? Do in-house computers have the capacity to access and perhaps download files of the size and type offered? Does the library’s Internet connection have sufficient bandwidth capacity? Can the library provide the type of authentication the provider requires? Some resources have automatic timeouts. Although this is not necessarily a problem, the library should be able to set the parameters, such as timeout after ten minutes or thirty minutes of inactivity.

Many libraries lend e-book readers. This can help address the problem of e-content accessibility for those users who do not own these devices. A 2012 Primary Research Group report found that 39.0 percent of libraries surveyed offered e-book reading devices of some kind.9 This service was more common with public libraries, 66.7 percent of which offered at least one e-reader; 38.5 percent of academic libraries reported offering e-book reading devices. The types of readers varied—Amazon Kindles, Sony Readers, Apple iPads, Barnes and Noble Nooks, and others. Several sources rate, compare, or recommend readers. Libraries considering acquiring e-book readers should consult the most recent resources, because options change frequently, as does their compatibility with various e-book suppliers. Not only must a library budget for acquiring these devices, it must also consider replacement costs, support, and service issues.

School libraries have some unique logistical needs. They often desire easy integration of content, including multimedia resources, with other online resources to support the curriculum and state teaching standards. The ease with which this is possible (and permission to do so) should be a decision point in selecting e-content for school libraries.

OpenURL Compliance

OpenURL provides a common linking syntax and directs users to appropriate, subscribed resources for the content they are seeking.10 OpenURLs enable the transfer of metadata about an item (e.g., a journal article or book) from a resource where a citation is discovered (e.g., an abstracting and indexing database or the bibliography in an article or book) to a link resolver. A link resolver is an online utility that uses the OpenURL standard to link between a citation and the electronic full text of the resource cited. Typically, an OpenURL link resolver includes a knowledge base that records a library’s subscriptions to e-journals and other e-resources, along with details of how to encode links to these target electronic resources. Most libraries consider OpenURL technology critical functionality and generally require OpenURL source links from resources to which they subscribe. A library considering an e-resource should ensure that it is OpenURL-compliant.

Output and Delivery Options

Libraries should understand the output and delivery options available and evaluate these in terms of local needs and expectations. Options vary across products, and some may be more applicable to e-journals or to e-books, for example. Considerations include the ability to download, print, copy to clipboard, cut and paste, and send the content as an e-mail attachment, along with any limitations (e.g., frequency, number of files, and size of file) to doing so. The ease with which these activities can be performed is also a consideration. For example, how many clicks are needed to select and attach content to an e-mail message? Can the user tag a file to allow later printing? Some products support exporting citations to various bibliographic management programs such as RefWorks, Endnote, and ProCite and generating bibliographies in the format of the user’s choice. Additional options include the ability to highlight text, add bookmarks, and use annotation tools such as adding notes and comments.

Understanding restrictions enforced by DRM are important. Readers can find DRM frustrating. The web of links and instructions is often confusing, and what users perceive as onerous limitations on the ability to print and otherwise manipulate content can discourage use of these resources.

Providers frequently add output and delivery options; these should be monitored. Choosing the appropriate e-content requires understanding the output and delivery options most desired by the library and its users.

Compatibility with Mobile Devices

With an increasing volume of content accessible from and downloadable to mobile devices (smartphones, PDAs, tablet computers, e-readers), compatibility is an important criterion. Increasingly, e-content providers are offering mobile interfaces that provide most of the same features and options as the regular search interface, including access to electronic resources available from the library. The mobile interface is optimized to display web content effectively on portable devices’ small screens. Libraries considering various e-content options and providers should verify that an effective mobile interface is available.

Format compatibility is an issue with digital video files, particularly end user viewing options. Different file formats require different video players, and different video players require different file players. Common video players are RealPlayer, Windows Media Player, QuickTime, Apple iTunes, and Adobe Flash. Fortunately, most video players play a variety of file types in addition to their own proprietary file formats. Most video players can be downloaded for free, but users need to be sure they have the most recent versions of the software on their computer or mobile device, and some may need more than one player to access various file types. Users frequently expect libraries to provide technical support as they seek to download and access varied digital video files. The situation for libraries is more complicated than that of users if the library wishes to provide access within the library building. Though the video player may be free, the library must ensure that current players are loaded and that they have the hardware and software to support viewing the digital video files to which they provide access.

Library e-books that are viewed online in real time via desktop and portable computers present significantly fewer compatibility issues than e-books that are downloaded onto desktop and laptop computers, e-readers, and other mobile devices. Format compatibility for the latter will remain an issue for some time. The reader marketplace changes frequently, with new models with additional capabilities offered regularly. Not all files are forward-compatible with newer devices. DRM or its absence and the software on which portable devices run determine the devices on which e-books can be read. Aggregators are working with publishers to offer compatibility with as many devices as possible and also evaluating new devices with the goal of providing flexibility to libraries and their users. Although compatibility may seem capricious, libraries are best served if they understand the issues, what their users want, and what providers offer.

Duplication or Replacement of Existing Library Resources

Uniqueness of content, capabilities, or features is an accepted criterion for selecting all types of content. This takes on added complications when selecting e-content. Many libraries initially faced this decision when deciding whether to retain print journal subscriptions, print reference resources, and indexing and abstracting tools when they began licensing the e-versions. Although users generally prefer e-versions of these materials, libraries have struggled with deciding whether to retain print as archival copies and thus duplicate the content in print and electronic formats. This issue has become less debatable as publishers have moved away from offering print versions of these resources or have made the cost of acquiring both print and electronic versions prohibitive.

Many licenses for e-books do not permit access by more than one user. To satisfy user interest in high-demand titles, such as bestsellers and recent pub lications, libraries may purchase access to several copies, as they have done with print books. For example, a public library might aim to have no more than three holds on a title and purchase access to sufficient copies to maintain this ratio. After the period of high demand ends, the library might reduce its subscription to a single copy.

Decisions about replacing or duplicating other types of e-content are less clear. Many library users prefer traditional formats or do not have access to the technology to access e-content outside the physical library. Some print materials, such as picture books and other books for young readers, may remain the preferred format for some time. Academic libraries are moving more rapidly than public and school libraries to replace worn or missing print books with e-books when they are available and some type of perpetual access is assured.

As is the case with all selection decisions, attention must be given to the user community being served and the mission of the specific library. Deciding the extent to which print should be retained when an e-version is available or the desirability of replacing print with an e-version is not always simple.

Availability of Data to Measure Use and Effectiveness

Libraries should examine the ability of an e-resource provider to supply data that measures use and effectiveness. These data can help assess how well an e-resource is satisfying the library’s objectives and meeting the demands placed on it. Three conditions must be met to make these data available and useful. The first is identifying the desirable statistics the remote resource provider is to supply. The library community, through the International Coalition of Library Consortia (ICOLC), has taken the lead in this area. ICOLC’s “Revised Guidelines for Statistical Measures of Usage of Web-Based Information Resources” define and create a common set of basic use information requirements that all electronic products should provide.11 The statistics to be provided are

• number of sessions (logins)

• number of queries (searches)

• number of menu selections

• number of full-content units examined, downloaded, or otherwise supplied to user

• number of turn-aways, peak simultaneous users, and any other indicator relevant to the pricing model applied to the library or consortium

The second condition is that the manner in which these data are defined and counted by providers should be standardized. Project COUNTER (Counting Online Usage of Networked Electronic Resources, www.projectcounter.org) developed a single, integrated code of practice that normalizes how statistics are defined and counted by providers.12 The current code of practice covers journals, databases, books, and multimedia. E-content providers who follow this code are considered COUNTER-compliant.

The final condition is that the data be provided in a consistent manner and in a way that libraries can use. Libraries have had to download data individually provider by provider and manipulate it locally or contract with a third party, such as Serials Solutions and Scholarly Stats. The National Information Standards Organization is working with interested parties in the Standardized Usage Statistics Harvesting Initiative (SUSHI).13 SUSHI includes a versioned Web Services Description Language (WSDL) to describe the web service namespace and operation and an XML schema for COUNTER Release 4 reports. It includes a standard protocol for machine-to-machine automation of statistics harvesting that can be used by ERMs and other library systems. A few third parties are offering tools that consolidate and, in some cases, analyze COUNTER statistics.

An e-content provider should be COUNTER-compliant and able to provide the library with online access to the usage statistics. Ideally, these statistics are available in SUSHI format. At a minimum, libraries should ensure that the content provider supplies the key data points defined by ICOLC in an easily accessible manner.

Deselection

Deselection is the process of removing materials from a collection. In the case of e-content, this can mean identifying subscribed content for cancellation or removing e-books that have superseded content or are no longer relevant. Some e-book providers allow libraries to swap an e-book title with little or no use for another that might be used more. Libraries may decide to cancel e-content in response to subscription price increases and budgetary constraints, but other reasons apply. The product may not meet performance and content expectation; both may have changed over time. It may no longer satisfy the purpose for which it was selected. The content may be available from another provider with a better interface, more features, or lower cost. The library may have access to the content through another product. Library priorities may have changed, making investment in the product no longer prudent. Use may be low and no longer justify the cost. User needs may be met more cost effectively through alternative access and delivery mechanisms, such as ILL.14

Electronic Resources Use and Impact Statistics Supplementing COUNTER Data

Locally collected logins to licensed content. Using transaction logs is one method to capture use data when the provider is not OpenURL or COUNTER-compliant. These data do not measure user success in locating and accessing the specific content they are seeking, only that they logged into a resource. Users who log in from off-site may not be counted.

Affinity strings. Affinity strings are a way to track use by categories of users. Some academic libraries have access to data generated by campus systems through which individuals are assigned one or more affinity strings based on area of study or work. These might include codes for academic unit, specific academic program, or role (undergraduate, graduate student, faculty, etc.). A library using these data can sort logins by, for example, academic department and status as faculty or student. Care must be taken to protect the privacy of individual users.

Link resolver use. These data can measure the number of times users connect to a resource they locate by using an indexing or abstracting database—that is, clicks on full-text article links. These data depend on resources being OpenURL-compliant and thus actionable by the link resolver. They can capture use when a supplier is not COUNTER-compliant. Link resolver use statistics do not track activity when a user bypasses the link resolver and goes directly to the resource.

Thomson Reuters Journal Citation Reports. This bibliometric tool provides data on impact factors for some 11,000 titles in the sciences and social sciences through citation analysis. Note, however, that the frequency a journal is cited does not necessarily reflect its quality.

Deciding which of these conditions apply often requires usage data. Thus, selecting products that provide comprehensive, comparable, and easily acces sible statistics is important.15 One challenge is that usage statistics are not necessarily comparable from product to product. If, for example, a library decides to examine cost per use to determine if use justifies investment in a product, care must be taken that “use” is meaningful. Is it a measure of “full-content units examined, downloaded, or otherwise supplied to user,” or a count of logins or queries, which do not necessarily reflect user success?16 Usage statistics have little meaning if users cannot find the resource. Libraries should take care when making decisions based on these data.

Assessing user success and satisfaction with a product can be challenging. Effectiveness, impact factors, and data about outcomes can be lacking. Questions remain about how to determine which resources are of the greatest value to users and, for that matter, what makes a resource of greater or lesser value. Libraries can use focus groups to explore users’ perceptions of products. Usability testing assesses the effectiveness and efficiency of an e-resource and user satisfaction with it.17 In usability testing, representatives of the user community perform predetermined tasks while observed by researchers. Data collected are used to evaluate the degree to which the e-resource meets established criteria, including interface functionality, accessibility, compatibility with mobile devices, output options, and content that meets users’ needs.

Generally, the best approach to assessing a product for retention or cancellation involves two and sometimes three different methods. Using more than one method can help eliminate biases or skewed data that can influence an evaluation.

The least problematic time to cancel a subscription to e-content is at the point of renewal. However, some licenses (e.g., Big Deals) may limit cancellations or include specific conditions, such as provider performance failure or a library’s fiscal exigency, which must be met to permit cancellation. Libraries who wish to cancel during the term of the license should explore the possibility of getting a refund or credit, although this is not a common option.

Summary

Selecting e-resources involves additional criteria beyond those normally considered when selecting other types of resources. These include the provider’s business model; persistency of content; optimum user interface functionality; ease of authentication; accessibility that is not compromised by users’ access to computing hardware, software, or the Internet, or by their physical disabilities; acceptable impact on local services and affordable local technological and logistical support; compliance with OpenURL to support linking from citation to content through a link resolver; appropriate output options matching user needs; and compatibility with mobile devices. The extent to which an e-resource under consideration duplicates or can replace existing library resources merits consideration. The availability of data to measure use and effectiveness is critical because these data can help assess the resource’s value to users. Additional criteria are the availability of descriptive metadata and licensing and contractual terms, limitations, and obligations.

Making informed decisions when deselecting e-resources is as important as making informed selection decisions. Libraries assess currently licensed products to ensure that they continue to meet the expectations in place when they were selected and to compare them to new offerings on the market. E-resources should still fit within local priorities. Usage statistics, an important tool for assessing whether the cost of providing an e-resource is justified by the value it provides to users, is not without problems. Care should be taken when analyzing these data, and they should be considered in concert with qualitative measures of user success and satisfaction.