Design, query, and evaluate information retrieval systems
Introduction
Information retrieval (IR) systems are an important component of the information profession. Librarians and LIS professionals use IR systems consistently, whether it is a database they are searching to answer a reference question, an instruction session showing students how to search through databases, or even a simple Google search to find information. “Information retrieval is concerned with the actions related to the organization and processing of and access to information in all forms and formats” (Chowdhury, 2010, p.93). One of the primary functions of a library is to provide access to information, and the IR system is the tool for LIS professionals and users to search through and find information. The primary objective of an IR system is to match an information need with the information that will fulfill it (Weedman, 2021a). It is essential to understand how an IR system is designed to allow for a more effective and efficient search process, and it is important to evaluate IR systems to ensure usability and effectiveness at fulfilling queries.
Design
Designing an IR system, like with anything, is a complex process that requires the designers to understand the user for it to be successful. The design process varies depending on what is being created. IR systems consist of a linear design process with the basic structure of planning and design followed by implementation and, finally, evaluation, which consists of going back to the first stage to ensure effectiveness (Weedman, 2021b). The first step of the design process is to conduct a requirements analysis to determine if a problem exists, what it entails, and to decide how to begin the process (Weedman, 2021b). As a part of the analysis process, designers need to have in-depth knowledge of the users of the system, their information needs, and search capabilities (Weedman, 2018). The goal of an IR system is to aggregate and discriminate, finding a way to search for all the information a user needs and only relevant information, which can be a complex process. To achieve this goal, IR systems are designed to use attributes to connect fields and values placed in the search to metadata stored in the document (Tucker, 2021a). The storage design process of an IR system is critical to the success of a search query. Depending on how the IR system was indexed, such as assigning subject headings through natural language, controlled vocabularies, or tagging, the query can match an information document. Good design will increase the likelihood of finding a match between a query and a document (Weedman, 2021a). However, disambiguation can occur if clarifying information is not presented, which makes the search process more complex in finding the right information. LIS professionals need to understand the design process to better instruct database research to students and create better information queries.
Query
Querying is the process of submitting a search to an information retrieval system. Two fundamental things to know to search an IR system effectively are that the user has to make the search fit the system, and a search that fails in one system may succeed in another (Weedman, 2018, p.178). A query occurs when a user has an information need or a problem to solve. Once they identify this need or problem, they access the IR system to retrieve relevant information. There are two types of searches; the known item search occurs when the user needs to find a specific document, and the subject search occurs when the user is researching a broad topic or exploring an information need (Tucker, 2021a). However, searching in the system is rarely simple. As the user gets to understand the indexing processes of an IR system, they can evolve their querying skills and aggregate better results (Tucker, 2021b). It can be difficult to know how to articulate particular information needs or problems; using advanced searching techniques and IR system tools can assist users with better search results. Using Boolean Operators (AND, OR, & NOT) in the search process allows users to expand or limit their search terms in conjunction with nesting words together and using phrases in parentheses to find a whole term (Brown, 2021). Using the indexed controlled vocabulary in the system, typically through the Thesaurus, allows users to use the correct word or phrase to aggregate better results (Brown, 2021). Users can utilize proximity searching to find words near each other in a document or truncation and wildcards to limit search options (Brown, 2021). Understanding the limiters and facets of each IR system can allow users to limit their results greatly. Users can also use the information in a document, such as reference citations, to search for related results through citation chaining (Brown, 2021). By having these advanced querying capabilities, LIS professionals can better meet the needs of their communities.
Evaluate
Evaluation is a critical piece in the information retrieval system design process and plays a role during different stages and levels (Tucker, 2021c). Factors involved in the evaluation process are the performance of the IR system when the user is searching, the outcome of the results from the user’s query, and the environment of what is known about the user, the system, and the content of the system (Tucker, 2021c). The evaluation process is critical to the success of the IR system; to ensure a good product, the system's performance needs to be evaluated. The most fundamental way of evaluating the performance of a system is through relevance, which is how well the system retrieves relevant information to meet the user’s query (Brown, 2021; Tucker, 2021c). Relevance asks how well the system is able to aggregate (recall) and discriminate (precision) information to ensure all and only relevant information is retrieved. Evaluating a systems indexing language, both the design and the application of the language terms is another aspect of evaluating the relevance of the IR system design (Tucker, 2021c). A critical piece in the evaluation process is evaluating the usability of the system navigation and the user experience of the interface design (Tucker, 2021c). Conducting user research and usability tests throughout the design process will allow for better product results during the implementation stage. Observing how users interact with the system, what the navigation process shows about the usability of the interface, and any flaws or issues that come up in this stage, including evaluating the search engine optimization experience, can be corrected for implementation. Evaluation does not just occur in the IR system design process; LIS professionals need to understand how to evaluate systems, processes, and functions of the LIS profession to be able to make improvements to provide access to information.
Evidence
My first piece of evidence is an exercise done for Info 202 Information Retrieval System Design. This exercise consisted of creating two rules for an information retrieval system; in this case it was a system created for types of socks. This exercise gave me the opportunity to learn why standards of rules are necessary to ensure consistency of the information that is stored in the IR system. The rules written told the indexer of the IR system what they specifically needed to know in order to create a record in the database. This exercise shows my understanding of the complexity of IR systems and my understanding of the requirements for the rules to be clear, comprehensive, and concise in order to ensure a document or item can be retrieved in a query. This exercise shows my knowledge of how to design and query IR systems.
My second piece of evidence is a database searching exercise for Info 244 Online Searching. In this exercise, we were required to use the Dialog database to conduct four types of queries containing a news search, an author search, a topic search, and a known search. In this exercise, we were given extensive queries to dissect and practice searching using various search limiters and expanders to find the best information to meet our information needs. Each question consisted of creating a search plan, including which limiters or facets we planned to use, any contingency plan if too few or too many results were found, and the evaluation of our search strategy. This exercise also consisted of evaluating the indexing of an information item to understand how the query results were determined. This exercise shows my knowledge of the querying process in IR systems. This exercise shows my ability to interpret information needs into a search strategy, how I use advanced tools to limit or expand my search results, my analysis of a search results page, and my understanding of the indexing of an information document to determine relevance to my query.
My third piece of evidence is a website analysis for Info 287 User Experience. This analysis consists of a user experience evaluation of the College of Southern Nevada (CSN) Libraries website. This analysis needed to answer the four basic questions of browsing and searching a website, including “where am I?”, “where can I go?”, “what can I do?”, and “how can I search?”. I conducted this analysis across the entire CSN Libraries website and separated my analysis between what was done well and what needed improvement. Typical website conventions such as the name as a link at the top of the page, an identifiable search box, and good action links were prominent on the homepage. Improvement needed to be made to the navigation menu, particularly in finding basic information about the library and matching menu links with the name of the web page. This analysis shows my knowledge of information retrieval system usability and user experience practices in evaluating the ease of use when navigating IR systems including search engine optimization and the design of the website.
Conclusion
As a future academic librarian, I plan to use my knowledge of IR system design, querying, and evaluation to provide quality reference interactions and database instruction to library users. Evaluation of systems, processes, and services is an important part of the LIS profession and is what allows problems or issues to be made and improved upon to provide quality services and resources to our communities. I am currently a member of the assessment team at my academic library, and I plan to continue to dedicate my time to assessment, particularly user experience, through professional development and training through ACRL.
References
Brown, C. C. (2021). Librarian’s guide to online searching: Cultivating database skills for research and instruction. (6th ed.). Libraries Unlimited.
Chowdhury, G. G. (2010). Basic concepts of information retrieval systems. In V. M. Tucker (Ed.), Information Retrieval System Design: Principles & Practice. (6.1 ed., pp. 93-105). Academic Pub.
Tucker, V. M. (2021a). Lecture 2: Introduction to information retrieval systems. In V. M. Tucker (Ed.), Information Retrieval System Design: Principles & Practice. (6.1 ed., pp. 64-80). Academic Pub.
Tucker, V. M. (2021b). Lecture 6: Search. In V. M. Tucker (Ed.), Information Retrieval System Design: Principles & Practice. (6.1 ed., pp. 317-326). Academic Pub.
Tucker, V. M. (2021c). Lecture 7: Evaluation. In V. M. Tucker (Ed.), Information Retrieval System Design: Principles & Practice. (6.1 ed., pp. 349-357). Academic Pub.
Weedman, J. (2018). Information retrieval: Designing, querying, and evaluating information systems. In K. Haycock & M.J. Romaniuk (Eds.), The Portable MLIS. (2nd ed., pp. 171-185). Libraries Unlimited.
Weedman, J. (2021a). Designing for search. In V. M. Tucker (Ed.), Information Retrieval System Design: Principles & Practice. (6.1 ed., pp. 119-142). Academic Pub.
Weedman, J. (2021b). The design process. In V. M. Tucker (Ed.), Information Retrieval System Design: Principles & Practice. (6.1 ed., pp. 219-232). Academic Pub.