Articles on Discovery

To suggest more links please join the wiki, or email aarontay (@gmail account) to add new links. 


Web Scale Discovery & Search  (2013 to present) - Curated Custom flipboard magazine by Aaron Tay, covering articles, blog posts, videos and more on the topic. 

Web Scale Discovery Services (2011) (not free)- This work by Jason Vaughan is probably the first comprehensive work on Discovery systems so far available. There are chapters on WorldCat Local, Summon, Ebsco Discovery Service and Primo Central, what some are calling the "big four". The problem with works of such a nature is that they tend to become dated quickly hence the author's reluctance to create a comparison matrix (I guess that's why there is this wiki!). Still Chapters 1 (free) and in particular the two concluding chapters on "Differentiators" and "Questions to Consider" are excellent. I highly recommend this. Related interview

The Next Generation of Discovery (May 2011)- Fairly long Library Journal article. Substantial piece on what to look out for.

Federated Search Blog (Discovery Services Series) (2009-2011)- The federated search blog sponsored by Deep Web Technologies (which serves not just the library industry) focuses mostly on Federated Search. However in recent months with the rise of "Discovery" services, there has being a series of interesting posts tagged "Discovery services". While "Discovering discovery services" is a good introduction to beginners, chronologically the author first began with "Beyond federated search?" , which questioned Summon's sole reliance on indexing and harvesting while ignoring Federated Search. This led to an interesting exchange in the comments, which fueled more posts including "Beyond federated search? The conversation continues" and "Beyond Federated Search – Winning the Battle and Losing the War?" (Guest post by Ex Libris's Carl Grant) and Science source selection . It's clear to me that the author does not see Discovery, pre-harvested/indexed services to be sufficient and advocates hybrid systems. Among other concerns he is worried about libraries giving up control of content control, and argues that federated systems are still necessary to gain access to content that cannot be easily harvested. Not everyone agrees as you can see in the comments. For instance one argument is that, libraries never had control even with federated search as not all connectors were available and the costs and disadvantages of federated search does not automatically mean a hybrid system is better.

Building Comprehensive Resource Discovery Platforms (March 2011)- Marshall Breeding argues "While losing the EBSCOhost content in Primo Central is a short-term disruption, it is not necessarily a long-term obstacle to the development of Primo Central in particular or a general mark against this model of discovery based on large aggregate indexes." , advocates that libraries "have an interest in the highest level of cooperation between the publishers and providers from which they license content and the organizations that offer discovery systems".

Searching Online Resources in New Discovery Environment: A Stateof-the-Art Review (March 2011)- Review of differences between federated search, web scale discovery, next-generation OPACS. Explains very briefly concepts like openURL, dois, z39.50, SRU/SRW, XML, OpenSearch etc. Also briefly describes WorldCat Local, Primo Central, EDS and Summon.

Discovering Choices: The Web Scale Discovery Marketplace (Oct 2011)- presentation from Athena Hoeppner at Electronic Resources & Libraries 2011, March 2011. Very current and detailed comparisons of Summon, EDS, Worldcat local and Primo Central.

The Ins and Outs of Evaluating Web-Scale Discovery Services (April 2012) - Basic but fairly comprehensive.

Presentations by Vendors

Actions Speak Louder than Words: Analyzing large-scale query logs to improve the research experience (July 2013) - Describes how Serialssolutions implements their "Relevance Metrics Framework (RMF)" to measure the effectiveness of their search algorithms. Measures like "Abandonment" (search or session level) , "Number of queries to first click", "Mean Reciprocal Rank (MRR)" and "Discounted Cumulative Gain (DCG)" are explained.

Debates & Controversies

Orbis Cascade Alliance - EBSCO and Ex Libris (May 2013)- Exchange of letters, releases between EBSCO and Ex Libris about availability of EBSCO content data on Ex Libris platforms - Primo Central.

Stakeholders Strive to Define Standards for Web-Scale Discovery Systems (Oct 2012) - About lack of standards for comparisons

Web-Scale Discovery: A Review of Summon, EBSCO Discovery Service, and WorldCat Local (July 2010) (not free) - Charleston Advisor, Summon 3 5/8 stars, EDS 3 1/2 stars, and WorldCat Local 3 1/8 stars (via the Distance Librarian , I don't have access)

Interview About Summon with Jane Burke, Vice President of Serials Solutions (April 2010) - Jane argues for the superiority of Summon vs competitors. "Summon has caused a genuine stir in the market and we're seeing other new services––EDS and Primo Central, for example––repurposing existing products to attempt to do what Summon does. But, in our experience, to meet the needs of the net-generation requires a whole new approach to discovery and built-for-purpose technology to enable it. For example, while EDS and Primo Central continue to rely on federated search and munging result sets, Summon preharvests content to a single index.". This led to a war of words "EBSCO Responds to Jane Burke Interview", "Serials Solutions Responds to Letter by EBSCO's Tim Collins" and "Ex Libris Primo Responds to Interview by Jane Burke"

EBSCO Discovery Service (EDS) vs. Serials Solutions Summon Faceoff (Nov 2010)- Summary of moderators questions at Charleston Conference.

Discovery Face-Off Draws a Crowd, but Leaves Many Wanting More (Nov 2010)- Library Journal article covering Charleston Conference.

EBSCO Publishing Participates in a Discovery Showdown at the Charleston Conference (Nov 2010)- How the Charleston Conference Summon-EDS show down went according to EBSCO.

Conclusions from the Charleston Conference "Face-Off" (Nov 2010) & "Gladiators" to perform sleight-of-hand at Charleston Conference (Oct 2010)- Ex Libris's point of view about the Charleston Conference Summon-EDS show down. Carl Grant suggests 3 questions to ask.

Discovery Services: Over-Hyped and Under-Performed *updated* (Dec 2010) - Comment by Abe Lederman, President and CTO of Deep Web Technologies on Charleston Conference

Another facet of the “library bypass strategies” (Nov 2009)- Carl Grant of Ex-Libris argues for the importance of content neutrality. Essentially he argues that librarians must remain in control of what content is searchable or be bypassed. Presumably relying just on a unified index controlled by a vendor cedes control and "you’ve setup your library to be bypassed and ultimately replaced in the information value chain."

EBSCO Exclusives Trigger Turmoil  (Jan 2010)- Coverage of EBSCO Exclusive details.

Heads they win, tails we lose: Discovery tools will never deliver on their promise (Jan 2011)- Iris a reference and instructional librarian reflects on the whole debate over EBSCO removing content from Primo Central. Iris relates the candid conversations she had with the vendors.

Library experiences, evaluations & case studies 

Evaluation and Comparison of Discovery Tools: An Update (June 2014) - Evaluat fourteen major discovery tools (three open source and ten proprietary), benchmarking sixteen criteria recognized as the advanced features of a “next generation catalog.” Some of the features have been used in previous research on discovery tools. The purpose of the study was to evaluate and compare all the major discovery tools, and the findings serve to update librarians on the latest developments and user interfaces and to assist them in their adoption of a discovery tool.

Evaluation of a Resource Discovery Service, FindIt@Bham (April 2014) - University of Birmingham - Pretty standard paper on user feedback of the launch of Primo Central. Consistent with almost every paper in this area, undergraduates were most satisfied with the discovery service compared to Post graduates and academic staff who were by far the least satisfied. Eg About 30% of academic staff think Primo Central is worse or much worse than previous catalolgue, with slightly better results when comparing with the federated service.

Implementing discipline-specific searches in EBSCO Discovery Service (Jan 2014) - University of Maryland University College - Describes implementation of discipline specific filters in Ebsco Discovery Service in widgets and default search box. They were used about 15% of the time. Do note that this level of filter is at the database level (e.g All articles in Business Source Complete = Business discipline), which was later superceded by EDS's Discipline Limited Searching which classifies at the journal level.

Maximizing academic library collections: measuring changes in use patterns owing to EBSCO Discovery Service  (Jan 2014) - Western Carolina University - One of the latest studies multi-institution usage studies to control for impact  of discovery layers. This one uses "aggregated, site-level EBSCOhost statistics on the total number of abstract views and full-text retrievals" from NC LIVE, ASU, and UNCA as a control for factors that do not include implementation of a discovery layer. Results show evidence for increasing full-text downloads in Ebsco databases is weak (because other institutions rise as much), but abstracts views appear to have increased significantly. Also "WCU experienced a demonstrable increase in the use of abstracts and A&I databases in contrast to other studies"

Evaluating library discovery tools through a music lens (Jan 2014) - Using music material, provides a check-list for evaluating discovery tools. 

Studies 3 hypothesis - (1) EDS will result in increased use of the library’s e-resources, (2) EDS will result in increased use of the 
library’s print collection, and (3) EDS will result in decreased use of interlibrary loan for materials.  - (2) was not supported, checkouts decreased 28 percent from the previous year  Somewhat unusual is besides just looking at COUNTER, checkout statistics etc of the institution, statistics from peer institutions like ASU,  UNCA was used as controls.

"Pretty Rad": Explorations in User Satisfaction with a Discovery Layer at Ryerson University  (Dec 2013) - Ryerson University - Pretty standard survey on students satisfaction with Summon. 40% are very satisfied. Similar to other studies, Graduates students were slightly less satisfied,  found it slightly harder to use. Comparing to respondents who have only used Summon, vs those who have used subject databases, the later group was less satisfied. 

Discovery or Displacement? A Large Scale Longitudinal Study of the Effect of Discovery Systems on Online Journal Usage (Nov 2013) - Fascinating large scale study at  Charleston Conference 2013 on effect of implementation of discovery service on full-text downloads. 6 libraries each of Primo Central, WorldCat Local, Summon, EDS was chosen for analysis. Studied  "COUNTER JR1 total full text article views for the 12 months before vs 12 months after implementation date". Studied 3 factors - institution, publisher and discovery service and impact on full-text downloads. For Discovery service providers alone as factor, it seems Primo Central and Summon do increase full text articles significantly more than the other 2. 

"Where are my Marc records?" - Librarians' perception of discovery tools (Sept 2013) - Université de Liège - ULG  - Survey of 72% of all Library staff of perceptions of discovery tool. Asks agreement of 22 survey statements on discovery, eg. "With a discovery tool, end-users will less often ask librarians for help.", "Discovery tools are only useful for beginners and freshmen". TSome of the results are segmented to only Information Literacy librarians. 19% Disagree or Strongly disagree the discovery tool will have a positive impact, though 45% don't know. This is stated to be a survey about discovery not on the specific tool Primo Central implemented at the library but it's likely respondents are using their experience of Primo Central that was just implemented to answer, in particular the last question about impact on library work.

Paths of Discovery: Comparing the Search effectiveness of EBSCO Discovery Service, Summon, Google Scholar, and Conventional Library Resources (Aug 2013) - Bucknell University and Illinois Wesleyan University - Overall resources found by students using EDS is rated significantly higher by librarian raters than Summon users. Some of it could be due to the fact that Summon surfaces more newspaper articles and are selected by users but even after removing that and considering journal articles only, papers selected by users using EDS is still rated significantly higher for all but 1 question. EDS users also generally finish the task faster. That said, this is not a straight head to head test but rather users of Summon in one university vs users of EDS in another university, so a lot of other factors might be in play.

Promising Practices in Instruction of Discovery Tools (July 2013) - Survey of librarian practices - "Early evidence and the results of this study suggest that when teaching discovery tools, librarians are spending more time teaching transferable skills such as evaluating and refining search results and are spending less time teaching skills such as choosing a database and using database mechanics"

Implementing Summon: an unexpected journey (July 2013) -  University of Wolverhampton

Web-Scale Search and Virtual Reference Service: How Summon Is Impacting Reference Question Complexity and Reference Service Delivery (Jul 2013) - Royal Roads University - Studies the number and type of reference queries received after the launch of Summon. Queries are classified using the READ scale. Number of reference help questions actually rose. "Summon had little impact on the distribution of complex questions" ,"Question complexity is not increasing because students and faculty remain as unsure as they ever were about how to conduct library research"  though biggest change was fall in known item search questions. Summon also allows librarians to answer more questions quicker, with increased in proportion of questions taking less than 15 mins rising. 

Relevance and Phrase Searching in Summon: looking under the hood (Jul 2013) - American University of Sharjah Library - Explains why in Summon sometimes adding quotes will increase the number of results. Essentially by default without quotes, proximity operator of 200 characters is automatically turned on, but is turned off with quotes.
Improving Library Resource Discovery: Exploring the Possibilities of VuFind and Web-Scale Discovery (Jun 2013) - the Bridge Consortium (the shared library of Carleton College and St. Olaf College). Pretty interesting inside look at how the consortium moved step by step from a classical catalogue to a catalog overlay (also often called "next generation catalogues") eventually choosing Vufind over III's Encore. And finally a push towards Web Scale Discovery by selecting Summon and including it into Vufind using a two column layout of "Books & More" and "Articles" , a style popularized by Villanova University. Desire to avoid going back into the classic catalogue (III's webpac) was a major reason for choosing Vufind+Summon.

Revelation from the literature: how web-scale discovery has already changed us (May 2013) - Analysis of 53 articles On discovery.

Discovery and Analysis of the World's Research Collections: JSTOR and Summon Under the Hood (Apr 2013) - Short article, very briefly describes  ITHAKA's  Local Discovery Integration (LDI) Pilot.  If JSTOR shows zero results, users will be redirected to Summon (libraries in pilot only). ".Links to a library's discovery platform are also located at the bottom of a search results page and in a pop-up dialog box that appears automatically on the third page of results. Preliminary tracking of LDI usage by ITHAKA shows most people link out from JSTOR when faced with zero results to a search."

Results of Web-Scale Discovery: Data, Discussions, and Decisions (Apr 2013) - Grand Valley State University Library - Jeffrey Daniels Head of Technical Services and Electronic Resources Management, talks about experiences at Grand Valley State University, a pioneer Summon library. Nothing particularly new is covered that wasn't already in earlier papers, presentations by GVSU librarians 

(Mar 2013) -  Illinois State University - Compares Usability studies of Summon and EDS (See Promise Fulfilled? An EBSCO Discovery Service Usability Study ) . Summon is subjected to a somewhat unusual usability study where users start from the advanced search screen. The same tasks were also a subject of a usability study for EDS. In General Summon did worse, over 50% of scenarios were completed successfully, while EDS managed 86%. One of the scenarios (identifying records belonging to a specific database) was biased towards EDS but even dropping that Summon's score rose to only 73%. The study points to weaknesses in the usability of the advanced search screen, as well as long standing issues like difficulty finding record saving function, inconsistent popups etc, some of these are slated to be fixed in Summon 2.0, including a redesigned advanced search screen. EDS also seems to have an advantage in terms of clarity of presentation of links to full text.

Web-Scale Discovery in an Academic Health Sciences Library: Development and Implementation of the EBSCO Discovery Service (Feb 2013) - Himmelfarb Health Sciences Library - Interesting paper describing implementation process of EDS for a Medical library. After testing default was set to limiting retrieval to “Available in Library Collections” as well as  turning full-text searching off to improve relevancy which was a major issue. "Other changes made before general release included removing the automatic display of search history from “Advanced Search” and increasing the size of the “Find It @ Himmelfarb” and “Full-text @ Himmelfarb” link resolver buttons."

A Comparison of Article Search APIs via Blinded Experiment and Developer Review (Jan 2013) - Johns Hopkins Libraries - A very significant and unique paper that uses APIs from Summon, EDS, Primo, "EBSCOhost ‘Traditional’ API" and Scopus to conduct a "blind test". But results were inconclusive. Only Scopus was significantly less preferred compared to the rest. Summon came closest to be preferred at a statistically significant level but just missed out. Somewhat surprising result also was how well "EBSCOhost ‘Traditional’ API" did, not "losing" to any product (some ties), but beating EDS which you might expect to be superior. But of course the results were not statistically significant. Paper also studied the response time of the APIs used with Summon's coming up on top. It was "definitely the fastest with a median response time under one second — and consistently fast." Lots of interesting observations on what might affect preferences. 

Impact of a new discovery service at Li Ka Shing Library (Dec 2012) - Singapore Management University Library - Survey of users comparing Encore Synergy and Ebsco Discovery Service.

Reflections on teaching and tweaking a discovery layer. (2013) - Bowling Green State University - Survey of undergraduates on Summon use with the view to use it to change defaults. With regards to newspaper articles appearing in Summon 39% wanted it to appear by default the rest wanted it to appear later or not at all (but this seems to be along discipline lines). More surprising 48% wanted items beyond the library collection to appear by default. 93% wanted Summon to recommend databases based on keywords. The main weakness with this study of course is that it is based on what users say they want which might not align with their actual behavior. For sure, no Summon library I know of ships with results beyond library collection as default! 

Popp, M. P., & Dallis, D. (2012). Planning and implementing resource discovery tools in academic libraries. Hershey PA: Information Science Reference. - Massive volume with 40 chapters covering a wide spectrum of topics some theoretical, others practical implementation issues. While the literature is currently dominated by studies on Summon, this volume, as a fair variety of tools & services represented including Encore Synergy, WorldCat Local , Primo Central & Ebsco Discovery service & covers topics such as 

Teaching Outside the Box: ARL Librarians’ Integration of the “One-Box” into Student Instruction (2014 preprint)- ARLs - Analyses survey of 352 librarians from ARLs who have a teaching or instruction role on how they use and/or teach "one-box". Only 9.2% of librarians will direct students to the one-box on the library homepage after understanding the topic to be searched for. 75.4% will direct users to databases, while 15.3% gave the "other" response which mostly is a variant of "it depends" type of response. For "one shot" sessions, 14.2% "always teach" the one-box, 25.9% "often teach" the one box. It is interesting to compare this with The impact of serial solutions’ summon™ on information literacy instruction: Librarian perceptions . Top 3 reasons not to teach one box includes subject databases are more appropriate, search results are confusing and ineffectiveness of one-box due to technical/implementation reasons. In short, librarians are lukewarm about "one box" for multiple reasons , in line with other earlier studies such as  Culture Shock: Librarians' Response to Web Scale Search  and the already mentioned impact of summon in information literacy instruction.

Discovery layers and the distance student: online search habits of students (2013 preprint) - RRU Library - Detailed survey plus comparison of pre/post Summon usage of databases

Collection Usage Pre- and Post-Summon Implementation at the University of Manitoba Libraries (Dec 2012) - University of Manitoba Libraries - Compares COUNTER stats and print circulation for 2 years post Summon implementation. Pretty standard results similar to other usage studies like Way(2010)'s GVSU study.

Findit@Flinders: user experiences of the Primo discovery search solution (Dec 2012) - Survey and usability test conducted. Usability test shows slightly higher completion rates than Voyager (Opac). 

Discovery Systems: Analyzing the Gap Between Professors' Expectations and Student Behavior (Nov 2012) - Virginia Tech - Interesting assignment by Professor asking students to : “locate two scholarly articles 
that are related to the main point of the assigned article.” using Summon. One interesting finding, users mentioned the subject term facet..... Addition comments also on when Summon can be useful in the research process. 

Beyond boolean, towards thinking: discovery systems and information literacy (Oct 2012) - Hong Kong Baptist University Library 

How are libraries designing their search boxes? (I) , How are libraries designing their search boxes? (II) , 
How are libraries designing their search boxes? (III) - Articles, Databases and Journals (Oct 2012)
- Survey by Blogger Aaron Tay on how Summon libraries are designing their search boxes ,lots of analysis with screenshots of representative examples.

(Self-) Discovery Service: Helping Students Help Themselves (Sept 2012) -  University of Maryland University College (UMUC) - Discussion about when to use subject-specific database vs when to use Onesearch (Ebsco discovery service). Protocol is suggested for reference desk, classes and for different research needs. In many ways, nothing ground-breaking but still interesting.

Academic Libraries and Discovery Tools: A Survey of the Literature (Aug 2012) - Literature review.

How Web-Scale Discovery Changes the Conversation: The Questions Librarians Should Ask Themselves (Aug 2012) - Good all-round set of questions that one should consider when implementing a discovery service. This article shows how diverse and wide-ranging the impact a discovery service will have on various aspects of the library including information literacy, marketing, website design, collection development, acquistions & eresource management, marketing .

Search Query Quality and Web-Scale Discovery: A Qualitative and Quantitative Analysis (Aug 2012)  - Montana State University - Divides searches into Summon into 7 types. In particular low quality searches are defined as "URL searches" (e.g, Invalid (searches that should be done on other domains such as websites, faqs) and Natural language language searches (more than 3 words with stop words and not a known item term as determined by searches in world cat). High quality searches were defined as searches for databases/journal titles, known item searches (searches for particular item where title or author is known), subject searches and Boolean operators. Across 2 semesters , 91% and 92% were high quality searches. Of the high quality searches only 3% uses boolean. 

Bridging the Gap from Wikipedia to Scholarly Sources: A Simple Discovery Tool (Aug 2012) - Stevens Institute of Technology -
Description of interesting bookmarklet that can do a Summon search by pulling in the title from the Wikipedia page they are on.

Usability Testing of a Web-Scale Discovery System at an Academic Library (Aug 2012) - Tulane University - Usability Testing of Primo Central. - Well designed usabilty study which led to specific recommendations to change. 

Information Literacy Opportunities within the Discovery Tool Environment (Aug 2012) - Practical guide with the following tips Focus on Developing Search Terms, Focus on Developing Search Terms, Focus on Developing Search Terms, Use the Discovery Tool as a Scaffold for Subject-Specific Databases, Develop Supplemental Subject Guides.
Creating an Actionable Assessment Framework for Discovery Services in Academic Libraries (Aug 2012) - Emory University Libraries -
Emory uses UserVoice service to manage support requests in Primo Central. plus various statistics to track via Google Analytics and build-in Primo statistics to measure success. E.g Percentage of Total Visitors in Primo > Percentage of Total Visitors in OPAC, Percentage of Return Visitors to Primo > Percentage of Return Visitors to OPAC, Percentage of Sessions resulting in Sign-In = Number of Sign-Ins/Number of Total Visitors.

Searchability and Discoverability of Library Resources: Federated Search and Beyond (Aug 2012) - Focuses on history of federated search, describes the main players, also describes the main Discovery players which feels surprisingly dated,

Catalog Maintenance in the Discovery System: Opinions of Early Adopters (Aug 2012) - survey of early adopters of discovery systems including Encore, Aquabrowser, Summon, Worldcat local etc and their perceived importance of clean ILS metadata. They were asked to chose between 2 views "expose errors" (discovery leads to exposure of metadata errors in ILS due to facets etc) or "bury data" (Discovery has milions of items in the index, ILS records tend to be buried). Surprisingly, the results are equally split.

A Tale of Two Systems: A Case Study on the Implementation of Two Discovery Systems at Davidson College (Aug 2012) - Davidson College - An interesting paper because it talks about a failed Summon implemention, some issues included the inability to handle print & online records in A-Z lists (fixed now), relevance ranking issues and disappointment "when, in the course of the next few releases of Summon, many of the proposed enhancements we liked had not only not occurred but had been dropped entirely, while a number of changes that we didn't like had been made". In the end the library moved towards OCLC's WorldCat local and WorldShare.

Discovering the Discovery Tool: The Introduction and Impact on Research and Instruction at Seminole State College of Florida (Aug 2012) - Minole State College of Florida - Describes users reactions to Primo Central. The results don't look particularly positive to me, "Asked to describe student responses to the single search box, 50 percent of respondents indicated that students were indifferent, 30 percent felt that students were confused, and 20 percent responded that students seemed pleased" - respondents were faculty or librarians.

A Glimpse at Discovery Tools within the HBCU Library Landscape (Aug 2012) - Survey of HBCU library directors on Discovery, with 26 responses. 13 or 50% have already implemented Web Scale Discovery, with Ebsco Discovery Service being most popular (5).

Implementation of a Discovery Layer: The Franklin College Experience
 (Aug 2012) - Franklin College - Implementation of Primo

Smoke Gets in Your Eyes: Trials and Triumphs of Implementing the Ex Libris Primo Discovery Service in a Small Regional Consortium
 (Aug 2012) - NC-PALS (North Carolina-Piedmont Automated Library System, Inc.) consortium - Talks about challenges of implementing SFX/Primo Central with  Web 2, a legacy OPAC solution from SirsiDyni - challenges include supporting consortium's reciprocal borrowing arrangement and other consortium related issues.

Search Me: Eastern Michigan University's Journey through the Highs and Lows of Implementing the Summon Discovery Tool
 (Aug 2012) - Eastern Michigan University - Describes the implementation issues working with Voyager ILS and SFX & content type mapping of ebooks due to single record practice of combining into one record.

The Evaluation of Discovery Services at Lynchburg College: 2009−2010
 (Aug 2012) - Lynchburg College - Fairly standard article on the process of evaluation of Summon vs EDS. The same common story is told about how both tools are equally matched, neither is perfect, with Summon having a better UI, but EDS having slightly better results due to Summon's heavier weighting on newspapers articles. In the end went for EDS mainly because EDS had a "a preponderance of journal articles or books on the first page of results."

Search Query Quality and Web-Scale Discovery: A Qualitative and Quantitative Analysis (Aug 2012) - Montana State University - Studies "high quality" vs "Low quality" searches in Summon. Low quality = Natural natural searches, searches for library services & other items better suited for a site search and URL searches (e.g High quality is everything else including Boolean, Subject, Known item, database/journal search. Arguably this is a very low standard for "high quality".  About 91.5% are considered "high quality". Most common search besides blank search is Of the "high quality", 70% are subject searches, 21.7% known item, 5% database/journal and 3.3% boolean. 

Information Literacy & Summon Blog (Jul 2012)  - blog set up to support the Information Literacy and Summon event held at Sheffield Hallam University on 18/Jul/2012

Branding library discovery services - what are libraries doing? (June 2012) - Survey of 130 summon libraries of which at least 43% retain the brand name Summon.

Comparative User Experiences of Next-Generation Catalogue Interfaces (Jun 2012) - "The author has undertaken a task-based usability test of vendor-provided next-generation catalogue interfaces and Web-scale discovery tools (Encore Synergy, Summon, WorldCat Local, Primo Central, EBSCO Discovery Service)"

Implementing a Discovery Layer: a Rookie's Season (Apr 2012) - PALNI (Private Academic Library Network of Indiana) consortium's - "Currently, only five of the 23 institutions are using Primo as their primary default search tool. "

Inventing the Can Opener: Getting the Most Out of Discovery (Apr 2012) University of Colorado - Short discussion about Encore Synergy.

Top 10 Discovery Tool Myths (Mar 2012) -  Some interesting ideas if you are new to the field eg "Discovery tools compete with Google.", 

Investigations into Library Web-Scale Discovery Services (Jan 2012)  UNLV Libraries - Comprehensive appendix of forms , survey questions to use with library staff, vendor and early adopter library references when evaluating Discovery options. 

Usability Test Results for a Discovery Tool in an Academic Library (Jan 2012James Madison University - Usability study of EDS, one of two currently available. Drop down facets and limits were used but federated search was not and eventually removed.

The impact of serial solutions’ summon™ on information literacy instruction: Librarian perceptions. (Dec 2011) . International Survey" The impact of Web-scale discovery systems on instruction is not known. This study of instruction librarian perceptions of Summon's impact on instruction and student information literacy skills is based on a survey. The survey reveals librarians’ ambivalence toward Summon. While some librarians agree that Summon has the potential to change the way librarians teach information literacy skills, it has not been fully integrated into the classroom."

How Users Search the Library from a Single Search Box (Dec 2011)  NCSU - Describes NCSU's experience with Quicksearch a self created search solution that pulls in results from Catalogue, Articles (Summon), recommended database titles, Journal titles, website, FAQs, institutional repository and more. The discovery solution is mentioned only briefly but it is an interesting paper showing that while users usually search for articles and books in 23% of cases other types of sources from websites,faqs etc need to be included.

Be realistic, demand the impossible: Comparison of 4 discovery tools using real data at the EPFL Library  (Dec 2011)   EPFL Library - Compared  Summon, EBSCO Discovery Service, Primo Central Total Care and Worldcat Local with Swiss meta-catalog Swissbib and Google Scholar as references. "Overall, none of the tested products met our expectations. Admittedly, our full list of criteria was a hard one to fulfill, but the important shortcomings identified during our tests and shared by all contestants all concerned essential features.... As a result, there was no winner in our competition."

Discovering What Works: Librarians Compare Discovery Interface Experiences (Dec 2011)- LJ Reviews - Short article by Librarians on experiences with EDS, World Cat Local, Summon, Primo.

KonSearch Usability StudyEvaluation of the new literature search engine of the University of Konstanz (Oct 2011) - University of Konstanz . Summon Usability test. Four main methods - "These were 1. a focus group and an online questionnaire, which were combined in the „Asking Users‟ method, 2. a „summative user test‟, 3. a „formative user test‟ and 4. „an eyetracking study" - In general Summon was more usable than the existing OPAC, though there are weaknesses in known item search and some minor usability issues (though some appears to be fixed now). 

A Student Perspective on e-Resource Discovery: Has the Google Factor Changed Publisher Platform Searching Forever? (Aug 2011) - Comparison of LISTA vs Sage vs Emerald vs Summon vs Google Scholar for " “student information seeking behaviour,” - relatively basic comparison of number of relevant results in top 10, ie precision at 10.

Promise Fulfilled? An EBSCO Discovery Service Usability Study (Aug 2011) - Illinois State University - Interesting comparisons with federated search usability studies and use of pre/post limiters and refinements.

The Detail Behind Web-Scale: Selecting and Configuring Web-Scale Discovery Tools to Meet Music Information Retrieval Needs (Aug 2011) - East Carolina University. General review on needs of music library users and what features a Discovery tool should ideally have to support them.

Web scale discovery: the user experience - Edwin Cowan University Library. "Library One Search (Summon) was by far the preferred navigation path."

Taking Discovery Systems for a Test Drive (Mar 2011) - American University . Compares Aquabrowser with World Cat local.

Culture Shock: Librarians' Response to Web Scale Search  (Feb 2011)- Edith Cowan University Library . Surveys of librarians reactions.e.g. Librarians unsure what is in the index. "For example, questions were raised around the percentage of titles in CINAHL or Academic One File which were covered by Summon. This data was very difficult to provide as Serials Solutions deals with publishers and not with database providers" and "Lack of confidence in the coverage led to concerns that full text may not be found and students would not recognise how to move beyond Library One Search to more specialized resources". Some statistics.

Liverpool’s Discovery
 (Feb 2011)- University of Liverpool's experience as beta tester of both Summon and EDS. They eventually went with EDS, though it's not very clear why. They seem to find Summon's interface better but "
But the interface is something that is relatively easy to fix" One reason given is the ability to "create multiple profiles, subject-oriented subsets of discovery resources".  

Experiences from the Field: Choosing a Discovery Tool for YOUR Unique Library (2011) - At 2011 Charleston Conference. Short descriptions of work done in various libraries including  Johns Hopkins University, American University Washington College of Law, Kent State University & OhioLINK consortium, Montana State University Library, George Washington University Libraries

Gotta keep 'em separated: the single search box may not be right for libraries (2011) - Swinburne University of Technology. Library

Brubaker, N., Leach-Murray, S., & Parker, S. (2011). Shapes in the Cloud. Online, 35(2), 20-26. Retrieved from EBSCOhost - Describes experiences of librarians at PALNI (Private Academic Library Network of Indiana) consortium. Handy table of differences between Primo Central (which was ultimately picked), Summon, Ebsco Discovery Service and Encore Synergy. Goes into differences between each discussion of how Encore Synergy "is going out on a limb here with its philosophies". Most comparisons exclude Encore Synergy so this is valuable

Implementing Web-Scale Discovery in an Academic Research Library (Oct 2010)- Arizona State University - Slides, implementation/surveys of Summon.

A Blunt Assessment of Search Discovery Tools (Oct 2010)- Short description of Montana State University library's experience with World Cat Local and Summon.

“’Summon’-ing Music: A Discovery Tool Journey” (Oct 2010East Carolina University. ppt

Article Discovery Investigation (April 2010)- University of Michigan Libraries - An evaluation of Metalib, Google Scholar, and Summon, user surveys etc. Recommends Summon.

The Impact of Web-scale Discovery on the Use of a Library Collection (2010)- Grand Valley State University "Grand Valley State University Libraries implemented Serials Solutions' web-scale discovery tool, Summon, during the fall of 2009. This case study explores whether Summon had an impact on the use of the library's resources during its first semester of implementation. An examination of usage statistics showed a dramatic decrease in the use of traditional abstracting and indexing databases and an equally dramatic increase in the use of full-text resources from full-text database and online journal collections. The author concludes that the increase in full-text use is linked to the implementation of a web-scale discovery tool."

Do students want a One-stop-shop to help them navigate their way around the maze of library resource? (2010)-  University of Huddersfield - Masters Dissertation, usability studies of Summon at University of Huddersfield. The results are what you probably expect, with most users preferring it to Metalib (the federated search) , majority not using the advanced functions or Boolean etc. There's a fairly good literature on why some are against One stop shops.

UA WorldCat/NEOS Libraries Catalogue/EBSCO Discovery  Comparison Chart (2010)- University of Alberta comparison of features in catalogue vs Worldcat local vs EDS

Discovery Services Task Force Recommendation to University Librarian (Dec 2009)- Oregon State University Follow up report. Recommends Summon. Compares Summon vs WorldCat Local

An Evaluation of Serials Solutions Summon As 
the Dartmouth College Library (Nov 2009)- Dartmouth College Library. Comprehensive evaluation of Summon experience and user needs.

The difference between federated search and discovery services (May 2009), Additional observations about unified discovery services (Aug 2009)- Very early (2009ish) thoughts by librarians from Grand Valley State University, the first adopter of Summon.The second piece has some interesting comments (to me anyway).

WorldCat Local Task Force Report to LAMP  (March 2009)- Oregon State University Focuses on WorldCat Local. "compares the features of WCL with three other existing resource discovery products 
including: OSU’s LibraryFind™, Serials Solutions’ Summon, and Innovative Interfaces’ Encore." 

Other Summon Case Studies - Short summaries

Misc comments on other issues

Convenience and its Discontents: Teaching Web-Scale Discovery in the Context of Google (Jan 2012) - ACRLog blog - Does Summon require instruction? "information literacy instruction isn’t necessary because students know how to get what they want from Google. If the new tool is like Google, then why does it require instruction?"

Article search, and catalog search (Aug 2011)- Interesting discussion on whether users want "one search" and discussion of alternatives. Comments are particularly interesting, from people who have done studies on this aspect. Villanova, NCSU, University of Michigan Libraries etc in particular are examples to look at for alternatives. 

Is Summon the search solution we want it to be? (March 2011)- Dean Giustini, a medical librarian who is working on a paper on how to use Summon to tech information literacy, find that "The idea of providing one-search for users in medicine — mostly physicians and medical students — is very difficult for me to justify (and teach)." - Interesting to see what other librarians in the field teaching think.

Should academic libraries default conservative on discovery systems? (Dec 2010) - The author talks about the slowness of OCLC WorldCat Local compared to classic catalogues. My sense though "Discovery" here is used more as a synonym for "Next Generation catalogues" (e.g. Primo, Aquabrowser, Encore) rather than what some call "Web Scale Discovery tools" (e.g. Summon, EDS, Primo Central).

Is Discovery better than google scholar ? (Oct 2010) - Summary of internet librarian 2010 presentation. Google scholar is broader, but many users end up using only the free sources because they can't figure out or are too lazy to get the full text via the library's subscription.

Summon @ Huddersfield (May 2010)- "In my view, one decision to implement one of these products – which ever it is – will have a chain reaction. "

Summon 'web scale'? I don't think so (Sept 2009)- Thoughts about suitability of the term "web scale". Comparison of OCLC WorldCat Local vs Summon.

Discussion on LSW Friendfeed - Librarians discussing how Summon is a boon for inter-disciplinary research.Another discussion on how to creatively present resources that are not in the unified index.

Wikis/blogs/rough notes on issues

Unified Resource Discovery Comparison - This wiki! The feature comparions matrix needs lot of work!

Information Literacy & Summon - 2013 edition was held at Manchester Metropolitan University Business School - 

eLibrary - Wordpress blog by Bcu Library , now discussing Summon

RESOURCE DISCOVERY AT LEEDS MET LIBRARY - Blog set up by Leeds Met Library to study this area.

SUMMON 4 HN - "The purpose of the project is to provide the JISC community with a case study of the implementation, evaluation and launch of Summon, a “next generation” resource discovery product from Serials Solutions, by two early UK adopters — the Universities of Huddersfield and Northumbria."

Hacking - using APIs etc

Hacking Summon - When the Oregon State University Libraries selected Serials Solutions’ Summon as its discovery tool, the implementation team realized that they had an opportunity to implement a set of “hacks” that would improve the overall user experience. This article will explore the space between Summon’s out-of-the-box user interface and full developer API, providing practical advice on tweaking configuration information and catalog exports to take full advantage of Summon’s indexing and faceting features. The article then describes the creation of OSUL’s home-grown open source availability service which replaced and enhanced the availability information that Summon would normally pull directly from the catalog.

GVSU Custom Summon Searches -Create Summon Search boxes without use of Javascript

Summon and LibGuides in Drupal -  University of Michigan Libraries , among others shows how the amazing all-in-one search works on MLibrary's website.

Bridging the gap from Wikipedia to scholarly sources: a simple library bookmarklet - Presentation - "When the bookmarklet is clicked, the search terms default to the title of the page (so it works especially well with Wikipedia articles), and a prompt is displayed that allows the user to edit the terms, if so desired. When they press the OK button, the search is automatically executed in the Discovery tool and the results are displayed in a new browser window." - Code is available, incorporates Google Analytics (optional). This uses Ebsco Discovery Service, but MLibrary has one that uses Summon.

RISE (Recommendations Improve the Search Experience) Project blog - Describes work on a recommender system. "Our EZProxy log files contain records coming from both our Ebsco Discovery Solution and from SFX.  So we’ve been investigating how to get more data about the entries in the log files.  The approach that we have been taking is to use the log files as the base layer of data and then query other systems to pull in further information to add to the database to help the recommendations. So we are using the EDS API to draw in subject data and looking at the SFX API to do something similar.  Through a combination of ISSNs, DOIs and other techniques we able to add in data such as journal tiles and article titles."

Metasearching meets central indexes - Differences between federated search (aka Broadcast search aka Distributed search) and discovery (aka common index aka vertical search) , how to merge them together.

Webcasts (not all free)