To suggest more links please join the wiki, or email aarontay (@gmail account) to add new links.
Web Scale Discovery & Search (2013 to present) - Curated Custom flipboard magazine by Aaron Tay, covering articles, blog posts, videos and more on the topic.
Web Scale Discovery Services (2011) (not free)- This work by Jason Vaughan is probably the first comprehensive work on Discovery systems so far available. There are chapters on WorldCat Local, Summon, Ebsco Discovery Service and Primo Central, what some are calling the "big four". The problem with works of such a nature is that they tend to become dated quickly hence the author's reluctance to create a comparison matrix (I guess that's why there is this wiki!). Still Chapters 1 (free) and in particular the two concluding chapters on "Differentiators" and "Questions to Consider" are excellent. I highly recommend this. Related interview
The Next Generation of Discovery (May 2011)- Fairly long Library Journal article. Substantial piece on what to look out for.
Federated Search Blog (Discovery Services Series) (2009-2011)- The federated search blog sponsored by Deep Web Technologies (which serves not just the library industry) focuses mostly on Federated Search. However in recent months with the rise of "Discovery" services, there has being a series of interesting posts tagged "Discovery services". While "Discovering discovery services" is a good introduction to beginners, chronologically the author first began with "Beyond federated search?" , which questioned Summon's sole reliance on indexing and harvesting while ignoring Federated Search. This led to an interesting exchange in the comments, which fueled more posts including "Beyond federated search? The conversation continues" and "Beyond Federated Search – Winning the Battle and Losing the War?" (Guest post by Ex Libris's Carl Grant) and Science source selection . It's clear to me that the author does not see Discovery, pre-harvested/indexed services to be sufficient and advocates hybrid systems. Among other concerns he is worried about libraries giving up control of content control, and argues that federated systems are still necessary to gain access to content that cannot be easily harvested. Not everyone agrees as you can see in the comments. For instance one argument is that, libraries never had control even with federated search as not all connectors were available and the costs and disadvantages of federated search does not automatically mean a hybrid system is better.
Building Comprehensive Resource Discovery Platforms (March 2011)- Marshall Breeding argues "While losing the EBSCOhost content in Primo Central is a short-term disruption, it is not necessarily a long-term obstacle to the development of Primo Central in particular or a general mark against this model of discovery based on large aggregate indexes." , advocates that libraries "have an interest in the highest level of cooperation between the publishers and providers from which they license content and the organizations that offer discovery systems".
Searching Online Resources in New Discovery Environment: A Stateof-the-Art Review (March 2011)- Review of differences between federated search, web scale discovery, next-generation OPACS. Explains very briefly concepts like openURL, dois, z39.50, SRU/SRW, XML, OpenSearch etc. Also briefly describes WorldCat Local, Primo Central, EDS and Summon.
Discovering Choices: The Web Scale Discovery Marketplace (Oct 2011)- presentation from Athena Hoeppner at Electronic Resources & Libraries 2011, March 2011. Very current and detailed comparisons of Summon, EDS, Worldcat local and Primo Central.
The Ins and Outs of Evaluating Web-Scale Discovery Services (April 2012) - Basic but fairly comprehensive.
Presentations by VendorsActions Speak Louder than Words: Analyzing large-scale query logs to improve the research experience (July 2013) - Describes how Serialssolutions implements their "Relevance Metrics Framework (RMF)" to measure the effectiveness of their search algorithms. Measures like "Abandonment" (search or session level) , "Number of queries to first click", "Mean Reciprocal Rank (MRR)" and "Discounted Cumulative Gain (DCG)" are explained.
Discovery versus Disintermediation: the new reality driven by today’s end-user (Feb 2010)- Presentation by Jane Burke , ProQuest
The Long Tail in Research Collections: Moment of Need Meets Discovery Speed (2010)- Presentation by John Law, ProQuest
The complete discovery solution (October 2010)
Debates & Controversies
Orbis Cascade Alliance - EBSCO and Ex Libris (May 2013)- Exchange of letters, releases between EBSCO and Ex Libris about availability of EBSCO content data on Ex Libris platforms - Primo Central.
Stakeholders Strive to Define Standards for Web-Scale Discovery Systems (Oct 2012) - About lack of standards for comparisons
Ultimate Debate Program on Web-Scale Discovery Services (June 2011)- Not really a debate recording at http://www.ustream.tv/recorded/15662070
Web-Scale Discovery: A Review of Summon, EBSCO Discovery Service, and WorldCat Local (July 2010) (not free) - Charleston Advisor, Summon 3 5/8 stars, EDS 3 1/2 stars, and WorldCat Local 3 1/8 stars (via the Distance Librarian , I don't have access)
Interview About Summon with Jane Burke, Vice President of Serials Solutions (April 2010) - Jane argues for the superiority of Summon vs competitors. "Summon has caused a genuine stir in the market and we're seeing other new services––EDS and Primo Central, for example––repurposing existing products to attempt to do what Summon does. But, in our experience, to meet the needs of the net-generation requires a whole new approach to discovery and built-for-purpose technology to enable it. For example, while EDS and Primo Central continue to rely on federated search and munging result sets, Summon preharvests content to a single index.". This led to a war of words "EBSCO Responds to Jane Burke Interview", "Serials Solutions Responds to Letter by EBSCO's Tim Collins" and "Ex Libris Primo Responds to Interview by Jane Burke"
EBSCO Discovery Service (EDS) vs. Serials Solutions Summon Faceoff (Nov 2010)- Summary of moderators questions at Charleston Conference.
Discovery Face-Off Draws a Crowd, but Leaves Many Wanting More (Nov 2010)- Library Journal article covering Charleston Conference.
EBSCO Publishing Participates in a Discovery Showdown at the Charleston Conference (Nov 2010)- How the Charleston Conference Summon-EDS show down went according to EBSCO.
Conclusions from the Charleston Conference "Face-Off" (Nov 2010) & "Gladiators" to perform sleight-of-hand at Charleston Conference (Oct 2010)- Ex Libris's point of view about the Charleston Conference Summon-EDS show down. Carl Grant suggests 3 questions to ask.
Discovery Services: Over-Hyped and Under-Performed *updated* (Dec 2010) - Comment by Abe Lederman, President and CTO of Deep Web Technologies on Charleston Conference
Another facet of the “library bypass strategies” (Nov 2009)- Carl Grant of Ex-Libris argues for the importance of content neutrality. Essentially he argues that librarians must remain in control of what content is searchable or be bypassed. Presumably relying just on a unified index controlled by a vendor cedes control and "you’ve setup your library to be bypassed and ultimately replaced in the information value chain."
EBSCO Exclusives Trigger Turmoil (Jan 2010)- Coverage of EBSCO Exclusive details.
Why ProQuest Bought ebrary (Jan 2011)
Heads they win, tails we lose: Discovery tools will never deliver on their promise (Jan 2011)- Iris a reference and instructional librarian reflects on the whole debate over EBSCO removing content from Primo Central. Iris relates the candid conversations she had with the vendors.
Library experiences, evaluations & case studies
Evaluation and Comparison of Discovery Tools: An Update (June 2014) - Evaluat fourteen major discovery tools (three open source and ten proprietary), benchmarking sixteen criteria recognized as the advanced features of a “next generation catalog.” Some of the features have been used in previous research on discovery tools. The purpose of the study was to evaluate and compare all the major discovery tools, and the findings serve to update librarians on the latest developments and user interfaces and to assist them in their adoption of a discovery tool.
Evaluation of a Resource Discovery Service, FindIt@Bham (April 2014) - University of Birmingham - Pretty standard paper on user feedback of the launch of Primo Central. Consistent with almost every paper in this area, undergraduates were most satisfied with the discovery service compared to Post graduates and academic staff who were by far the least satisfied. Eg About 30% of academic staff think Primo Central is worse or much worse than previous catalolgue, with slightly better results when comparing with the federated service.
Implementing discipline-specific searches in EBSCO Discovery Service (Jan 2014) - University of Maryland University College - Describes implementation of discipline specific filters in Ebsco Discovery Service in widgets and default search box. They were used about 15% of the time. Do note that this level of filter is at the database level (e.g All articles in Business Source Complete = Business discipline), which was later superceded by EDS's Discipline Limited Searching which classifies at the journal level.
Maximizing academic library collections: measuring changes in use patterns owing to EBSCO Discovery Service (Jan 2014) - Western Carolina University - One of the latest studies multi-institution usage studies to control for impact of discovery layers. This one uses "aggregated, site-level EBSCOhost statistics on the total number of abstract views and full-text retrievals" from NC LIVE, ASU, and UNCA as a control for factors that do not include implementation of a discovery layer. Results show evidence for increasing full-text downloads in Ebsco databases is weak (because other institutions rise as much), but abstracts views appear to have increased significantly. Also "WCU experienced a demonstrable increase in the use of abstracts and A&I databases in contrast to other studies"
Evaluating library discovery tools through a music lens (Jan 2014) - Using music material, provides a check-list for evaluating discovery tools.
Maximizing academic library collections: measuring changes in use patterns owing to EBSCO Discovery Service (Jan 2014) - Western Carolina University
Studies 3 hypothesis - (1) EDS will result in increased use of the library’s e-resources, (2) EDS will result in increased use of the
library’s print collection, and (3) EDS will result in decreased use of interlibrary loan for materials. - (2) was not supported, checkouts decreased 28 percent from the previous year Somewhat unusual is besides just looking at COUNTER, checkout statistics etc of the institution, statistics from peer institutions like ASU, UNCA was used as controls.
"Pretty Rad": Explorations in User Satisfaction with a Discovery Layer at Ryerson University (Dec 2013) - Ryerson University - Pretty standard survey on students satisfaction with Summon. 40% are very satisfied. Similar to other studies, Graduates students were slightly less satisfied, found it slightly harder to use. Comparing to respondents who have only used Summon, vs those who have used subject databases, the later group was less satisfied.
Discovery or Displacement? A Large Scale Longitudinal Study of the Effect of Discovery Systems on Online Journal Usage (Nov 2013) - Fascinating large scale study at Charleston Conference 2013 on effect of implementation of discovery service on full-text downloads. 6 libraries each of Primo Central, WorldCat Local, Summon, EDS was chosen for analysis. Studied "COUNTER JR1 total full text article views for the 12 months before vs 12 months after implementation date". Studied 3 factors - institution, publisher and discovery service and impact on full-text downloads. For Discovery service providers alone as factor, it seems Primo Central and Summon do increase full text articles significantly more than the other 2.
"Where are my Marc records?" - Librarians' perception of discovery tools (Sept 2013) - Université de Liège - ULG - Survey of 72% of all Library staff of perceptions of discovery tool. Asks agreement of 22 survey statements on discovery, eg. "With a discovery tool, end-users will less often ask librarians for help.", "Discovery tools are only useful for beginners and freshmen". TSome of the results are segmented to only Information Literacy librarians. 19% Disagree or Strongly disagree the discovery tool will have a positive impact, though 45% don't know. This is stated to be a survey about discovery not on the specific tool Primo Central implemented at the library but it's likely respondents are using their experience of Primo Central that was just implemented to answer, in particular the last question about impact on library work.
Paths of Discovery: Comparing the Search effectiveness of EBSCO Discovery Service, Summon, Google Scholar, and Conventional Library Resources (Aug 2013) - Bucknell University and Illinois Wesleyan University - Overall resources found by students using EDS is rated significantly higher by librarian raters than Summon users. Some of it could be due to the fact that Summon surfaces more newspaper articles and are selected by users but even after removing that and considering journal articles only, papers selected by users using EDS is still rated significantly higher for all but 1 question. EDS users also generally finish the task faster. That said, this is not a straight head to head test but rather users of Summon in one university vs users of EDS in another university, so a lot of other factors might be in play.
Promising Practices in Instruction of Discovery Tools (July 2013) - Survey of librarian practices - "Early evidence and the results of this study suggest that when teaching discovery tools, librarians are spending more time teaching transferable skills such as evaluating and refining search results and are spending less time teaching skills such as choosing a database and using database mechanics"
Implementing Summon: an unexpected journey (July 2013) - University of Wolverhampton
Web-Scale Search and Virtual Reference Service: How Summon Is Impacting Reference Question Complexity and Reference Service Delivery (Jul 2013) - Royal Roads University - Studies the number and type of reference queries received after the launch of Summon. Queries are classified using the READ scale. Number of reference help questions actually rose. "Summon had little impact on the distribution of complex questions" ,"Question complexity is not increasing because students and faculty remain as unsure as they ever were about how to conduct library research" though biggest change was fall in known item search questions. Summon also allows librarians to answer more questions quicker, with increased in proportion of questions taking less than 15 mins rising.
Relevance and Phrase Searching in Summon: looking under the hood (Jul 2013) - American University of Sharjah Library - Explains why in Summon sometimes adding quotes will increase the number of results. Essentially by default without quotes, proximity operator of 200 characters is automatically turned on, but is turned off with quotes.
Improving Library Resource Discovery: Exploring the Possibilities of VuFind and Web-Scale Discovery (Jun 2013) - the Bridge Consortium (the shared library of Carleton College and St. Olaf College). Pretty interesting inside look at how the consortium moved step by step from a classical catalogue to a catalog overlay (also often called "next generation catalogues") eventually choosing Vufind over III's Encore. And finally a push towards Web Scale Discovery by selecting Summon and including it into Vufind using a two column layout of "Books & More" and "Articles" , a style popularized by Villanova University. Desire to avoid going back into the classic catalogue (III's webpac) was a major reason for choosing Vufind+Summon.
Revelation from the literature: how web-scale discovery has already changed us (May 2013) - Analysis of 53 articles On discovery.
Discovery and Analysis of the World's Research Collections: JSTOR and Summon Under the Hood (Apr 2013) - Short article, very briefly describes ITHAKA's Local Discovery Integration (LDI) Pilot. If JSTOR shows zero results, users will be redirected to Summon (libraries in pilot only). ".Links to a library's discovery platform are also located at the bottom of a search results page and in a pop-up dialog box that appears automatically on the third page of results. Preliminary tracking of LDI usage by ITHAKA shows most people link out from JSTOR when faced with zero results to a search."
Results of Web-Scale Discovery: Data, Discussions, and Decisions (Apr 2013) - Grand Valley State University Library - Jeffrey Daniels Head of Technical Services and Electronic Resources Management, talks about experiences at Grand Valley State University, a pioneer Summon library. Nothing particularly new is covered that wasn't already in earlier papers, presentations by GVSU librarians
(Mar 2013) - Illinois State University - Compares Usability studies of Summon and EDS (See Promise Fulfilled? An EBSCO Discovery Service Usability Study ) . Summon is subjected to a somewhat unusual usability study where users start from the advanced search screen. The same tasks were also a subject of a usability study for EDS. In General Summon did worse, over 50% of scenarios were completed successfully, while EDS managed 86%. One of the scenarios (identifying records belonging to a specific database) was biased towards EDS but even dropping that Summon's score rose to only 73%. The study points to weaknesses in the usability of the advanced search screen, as well as long standing issues like difficulty finding record saving function, inconsistent popups etc, some of these are slated to be fixed in Summon 2.0, including a redesigned advanced search screen. EDS also seems to have an advantage in terms of clarity of presentation of links to full text.
Web-Scale Discovery in an Academic Health Sciences Library: Development and Implementation of the EBSCO Discovery Service (Feb 2013) - Himmelfarb Health Sciences Library - Interesting paper describing implementation process of EDS for a Medical library. After testing default was set to limiting retrieval to “Available in Library Collections” as well as turning full-text searching off to improve relevancy which was a major issue. "Other changes made before general release included removing the automatic display of search history from “Advanced Search” and increasing the size of the “Find It @ Himmelfarb” and “Full-text @ Himmelfarb” link resolver buttons."
A Comparison of Article Search APIs via Blinded Experiment and Developer Review (Jan 2013) - Johns Hopkins Libraries - A very significant and unique paper that uses APIs from Summon, EDS, Primo, "EBSCOhost ‘Traditional’ API" and Scopus to conduct a "blind test". But results were inconclusive. Only Scopus was significantly less preferred compared to the rest. Summon came closest to be preferred at a statistically significant level but just missed out. Somewhat surprising result also was how well "EBSCOhost ‘Traditional’ API" did, not "losing" to any product (some ties), but beating EDS which you might expect to be superior. But of course the results were not statistically significant. Paper also studied the response time of the APIs used with Summon's coming up on top. It was "definitely the fastest with a median response time under one second — and consistently fast." Lots of interesting observations on what might affect preferences.
Impact of a new discovery service at Li Ka Shing Library (Dec 2012) - Singapore Management University Library - Survey of users comparing Encore Synergy and Ebsco Discovery Service.
Reflections on teaching and tweaking a discovery layer. (2013) - Bowling Green State University - Survey of undergraduates on Summon use with the view to use it to change defaults. With regards to newspaper articles appearing in Summon 39% wanted it to appear by default the rest wanted it to appear later or not at all (but this seems to be along discipline lines). More surprising 48% wanted items beyond the library collection to appear by default. 93% wanted Summon to recommend databases based on keywords. The main weakness with this study of course is that it is based on what users say they want which might not align with their actual behavior. For sure, no Summon library I know of ships with results beyond library collection as default!
Popp, M. P., & Dallis, D. (2012). Planning and implementing resource discovery tools in academic libraries. Hershey PA: Information Science Reference. - Massive volume with 40 chapters covering a wide spectrum of topics some theoretical, others practical implementation issues. While the literature is currently dominated by studies on Summon, this volume, as a fair variety of tools & services represented including Encore Synergy, WorldCat Local , Primo Central & Ebsco Discovery service & covers topics such as
Teaching Outside the Box: ARL Librarians’ Integration of the “One-Box” into Student Instruction (2014 preprint)- ARLs - Analyses survey of 352 librarians from ARLs who have a teaching or instruction role on how they use and/or teach "one-box". Only 9.2% of librarians will direct students to the one-box on the library homepage after understanding the topic to be searched for. 75.4% will direct users to databases, while 15.3% gave the "other" response which mostly is a variant of "it depends" type of response. For "one shot" sessions, 14.2% "always teach" the one-box, 25.9% "often teach" the one box. It is interesting to compare this with The impact of serial solutions’ summon™ on information literacy instruction: Librarian perceptions . Top 3 reasons not to teach one box includes subject databases are more appropriate, search results are confusing and ineffectiveness of one-box due to technical/implementation reasons. In short, librarians are lukewarm about "one box" for multiple reasons , in line with other earlier studies such as Culture Shock: Librarians' Response to Web Scale Search and the already mentioned impact of summon in information literacy instruction.
Discovery layers and the distance student: online search habits of students (2013 preprint) - RRU
Library - Detailed survey plus comparison of pre/post Summon usage of databases
Collection Usage Pre- and Post-Summon Implementation at the University of Manitoba Libraries (Dec 2012) - University of Manitoba Libraries - Compares COUNTER stats and print circulation for 2 years post Summon implementation. Pretty standard results similar to other usage studies like Way(2010)'s GVSU study.
Findit@Flinders: user experiences of the Primo discovery search solution (Dec 2012) - Survey and usability test conducted. Usability test shows slightly higher completion rates than Voyager (Opac).
Discovery Systems: Analyzing the Gap Between Professors' Expectations and Student Behavior (Nov 2012) - Virginia Tech - Interesting assignment by Professor asking students to : “locate two scholarly articles
that are related to the main point of the assigned article.” using Summon. One interesting finding, users mentioned the subject term facet..... Addition comments also on when Summon can be useful in the research process.
Beyond boolean, towards thinking: discovery systems and information literacy (Oct 2012) - Hong Kong Baptist University Library
How are libraries designing their search boxes? (I) , How are libraries designing their search boxes? (II) ,
How are libraries designing their search boxes? (III) - Articles, Databases and Journals (Oct 2012) - Survey by Blogger Aaron Tay on how Summon libraries are designing their search boxes ,lots of analysis with screenshots of representative examples.
(Self-) Discovery Service: Helping Students Help Themselves (Sept 2012) - University of Maryland University College (UMUC) - Discussion about when to use subject-specific database vs when to use Onesearch (Ebsco discovery service). Protocol is suggested for reference desk, classes and for different research needs. In many ways, nothing ground-breaking but still interesting.
Academic Libraries and Discovery Tools: A Survey of the Literature (Aug 2012) - Literature review.
How Web-Scale Discovery Changes the Conversation: The Questions Librarians Should Ask Themselves (Aug 2012) - Good all-round set of questions that one should consider when implementing a discovery service. This article shows how diverse and wide-ranging the impact a discovery service will have on various aspects of the library including information literacy, marketing, website design, collection development, acquistions & eresource management, marketing .
Search Query Quality and Web-Scale Discovery: A Qualitative and Quantitative Analysis (Aug 2012) - Montana State University - Divides searches into Summon into 7 types. In particular low quality searches are defined as "URL searches" (e.g facebook.com), Invalid (searches that should be done on other domains such as websites, faqs) and Natural language language searches (more than 3 words with stop words and not a known item term as determined by searches in world cat). High quality searches were defined as searches for databases/journal titles, known item searches (searches for particular item where title or author is known), subject searches and Boolean operators. Across 2 semesters , 91% and 92% were high quality searches. Of the high quality searches only 3% uses boolean.
Bridging the Gap from Wikipedia to Scholarly Sources: A Simple Discovery Tool (Aug 2012) - Stevens Institute of Technology - Description of interesting bookmarklet that can do a Summon search by pulling in the title from the Wikipedia page they are on.
Usability Testing of a Web-Scale Discovery System at an Academic Library (Aug 2012) - Tulane University - Usability Testing of Primo Central. - Well designed usabilty study which led to specific recommendations to change.
Information Literacy Opportunities within the Discovery Tool Environment (Aug 2012) - Practical guide with the following tips Focus on Developing Search Terms, Focus on Developing Search Terms, Focus on Developing Search Terms, Use the Discovery Tool as a Scaffold for Subject-Specific Databases, Develop Supplemental Subject Guides.
Creating an Actionable Assessment Framework for Discovery Services in Academic Libraries (Aug 2012) - Emory University Libraries - Emory uses UserVoice service to manage support requests in Primo Central. plus various statistics to track via Google Analytics and build-in Primo statistics to measure success. E.g Percentage of Total Visitors in Primo > Percentage of Total Visitors in OPAC, Percentage of Return Visitors to Primo > Percentage of Return Visitors to OPAC, Percentage of Sessions resulting in Sign-In = Number of Sign-Ins/Number of Total Visitors.
Searchability and Discoverability of Library Resources: Federated Search and Beyond (Aug 2012) - Focuses on history of federated search, describes the main players, also describes the main Discovery players which feels surprisingly dated,
Catalog Maintenance in the Discovery System: Opinions of Early Adopters (Aug 2012) - survey of early adopters of discovery systems including Encore, Aquabrowser, Summon, Worldcat local etc and their perceived importance of clean ILS metadata. They were asked to chose between 2 views "expose errors" (discovery leads to exposure of metadata errors in ILS due to facets etc) or "bury data" (Discovery has milions of items in the index, ILS records tend to be buried). Surprisingly, the results are equally split.
A Tale of Two Systems: A Case Study on the Implementation of Two Discovery Systems at Davidson College (Aug 2012) - Davidson College - An interesting paper because it talks about a failed Summon implemention, some issues included the inability to handle print & online records in A-Z lists (fixed now), relevance ranking issues and disappointment "when, in the course of the next few releases of Summon, many of the proposed enhancements we liked had not only not occurred but had been dropped entirely, while a number of changes that we didn't like had been made". In the end the library moved towards OCLC's WorldCat local and WorldShare.
Discovering the Discovery Tool: The Introduction and Impact on Research and Instruction at Seminole State College of Florida (Aug 2012) - Minole State College of Florida - Describes users reactions to Primo Central. The results don't look particularly positive to me, "Asked to describe student responses to the single search box, 50 percent of respondents indicated that students were indifferent, 30 percent felt that students were confused, and 20 percent responded that students seemed pleased" - respondents were faculty or librarians.
A Glimpse at Discovery Tools within the HBCU Library Landscape (Aug 2012) - Survey of HBCU library directors on Discovery, with 26 responses. 13 or 50% have already implemented Web Scale Discovery, with Ebsco Discovery Service being most popular (5).
Implementation of a Discovery Layer: The Franklin College Experience (Aug 2012) - Franklin College - Implementation of Primo
Smoke Gets in Your Eyes: Trials and Triumphs of Implementing the Ex Libris Primo Discovery Service in a Small Regional Consortium (Aug 2012) - NC-PALS (North Carolina-Piedmont Automated Library System, Inc.) consortium - Talks about challenges of implementing SFX/Primo Central with Web 2, a legacy OPAC solution from SirsiDyni - challenges include supporting consortium's reciprocal borrowing arrangement and other consortium related issues.
Search Me: Eastern Michigan University's Journey through the Highs and Lows of Implementing the Summon Discovery Tool (Aug 2012) - Eastern Michigan University - Describes the implementation issues working with Voyager ILS and SFX & content type mapping of ebooks due to single record practice of combining into one record.
The Evaluation of Discovery Services at Lynchburg College: 2009−2010 (Aug 2012) - Lynchburg College - Fairly standard article on the process of evaluation of Summon vs EDS. The same common story is told about how both tools are equally matched, neither is perfect, with Summon having a better UI, but EDS having slightly better results due to Summon's heavier weighting on newspapers articles. In the end went for EDS mainly because EDS had a "a preponderance of journal articles or books on the first page of results."
Search Query Quality and Web-Scale Discovery: A Qualitative and Quantitative Analysis (Aug 2012) - Montana State University - Studies "high quality" vs "Low quality" searches in Summon. Low quality = Natural natural searches, searches for library services & other items better suited for a site search and URL searches (e.g www.facebook.com). High quality is everything else including Boolean, Subject, Known item, database/journal search. Arguably this is a very low standard for "high quality". About 91.5% are considered "high quality". Most common search besides blank search is facebook.com. Of the "high quality", 70% are subject searches, 21.7% known item, 5% database/journal and 3.3% boolean.
Information Literacy & Summon Blog (Jul 2012) - blog set up to support the Information Literacy and Summon event held at Sheffield Hallam University on 18/Jul/2012
Branding library discovery services - what are libraries doing? (June 2012) - Survey of 130 summon libraries of which at least 43% retain the brand name Summon.
Comparative User Experiences of Next-Generation Catalogue Interfaces (Jun 2012) - "The author has undertaken a task-based usability test of vendor-provided next-generation catalogue interfaces and Web-scale discovery tools (Encore Synergy, Summon, WorldCat Local, Primo Central, EBSCO Discovery Service)"
Implementing a Discovery Layer: a Rookie's Season (Apr 2012) - PALNI (Private Academic Library Network of Indiana) consortium's - "Currently, only five of the 23 institutions are using Primo as their primary default search tool. "
Inventing the Can Opener: Getting the Most Out of Discovery (Apr 2012) University of Colorado - Short discussion about Encore Synergy.
Top 10 Discovery Tool Myths (Mar 2012) - Some interesting ideas if you are new to the field eg "Discovery tools compete with Google.",
Investigations into Library Web-Scale Discovery Services (Jan 2012) UNLV Libraries - Comprehensive appendix of forms , survey questions to use with library staff, vendor and early adopter library references when evaluating Discovery options.
Usability Test Results for a Discovery Tool in an Academic Library (Jan 2012) James Madison University - Usability study of EDS, one of two currently available. Drop down facets and limits were used but federated search was not and eventually removed.
The impact of serial solutions’ summon™ on information literacy instruction: Librarian perceptions. (Dec 2011) . International Survey" The impact of Web-scale discovery systems on instruction is not known. This study of instruction librarian perceptions of Summon's impact on instruction and student information literacy skills is based on a survey. The survey reveals librarians’ ambivalence toward Summon. While some librarians agree that Summon has the potential to change the way librarians teach information literacy skills, it has not been fully integrated into the classroom."
How Users Search the Library from a Single Search Box (Dec 2011) NCSU - Describes NCSU's experience with Quicksearch a self created search solution that pulls in results from Catalogue, Articles (Summon), recommended database titles, Journal titles, website, FAQs, institutional repository and more. The discovery solution is mentioned only briefly but it is an interesting paper showing that while users usually search for articles and books in 23% of cases other types of sources from websites,faqs etc need to be included.
Be realistic, demand the impossible: Comparison of 4 discovery tools using real data at the EPFL Library (Dec 2011) EPFL Library - Compared Summon, EBSCO Discovery Service, Primo Central Total Care and Worldcat Local with Swiss meta-catalog Swissbib and Google Scholar as references. "Overall, none of the tested products met our expectations. Admittedly, our full list of criteria was a hard one to fulfill, but the important shortcomings identified during our tests and shared by all contestants all concerned essential features.... As a result, there was no winner in our competition."
Discovering What Works: Librarians Compare Discovery Interface Experiences (Dec 2011)- LJ Reviews - Short article by Librarians on experiences with EDS, World Cat Local, Summon, Primo.
KonSearch Usability StudyEvaluation of the new literature search engine of the University of Konstanz (Oct 2011) - University of Konstanz . Summon Usability test. Four main methods - "These were 1. a focus group and an online questionnaire, which were combined in the „Asking Users‟ method, 2. a „summative user test‟, 3. a „formative user test‟ and 4. „an eyetracking study" - In general Summon was more usable than the existing OPAC, though there are weaknesses in known item search and some minor usability issues (though some appears to be fixed now).