Evaluation and assessment. These are two concepts I have learned much about in my schooling and work experience. Evaluation and assessment of LIS services are an integral part to the information organization. Through evaluation and assessment practices, information organizations can learn about the needs of its users including how to improve and expand on its programs and services. Through my work experience, it almost comes as second nature to think about ways to assess and evaluate programs and services. Whenever a new program or service is about to launch, we are already thinking of ways to evaluate and assess the new program or service. Evaluation and assessment isn't only used to improve such services, but it is also used to show the value of the organization to it's stakeholders (Gilman, 2018). Evaluation and assessment methods are also used to advocate for the organization by showing its importance, relevance, and value the organization has to it's users and the overall community.
Similarly to the research process, the assessment and evaluation process follows a linear style. A question or issue is identified, research methods and/or data collection methods are identified, data is analyzed and interpreted, results are integrated into operational and strategic planning, and lastly, results are disseminated (Bakkalbasi, 2017). However, before beginning this process, an information organization must keep in mind that a well-designed assessment and evaluation plan must be feasible, produce actionable results, and be linked to the the overall mission and goals of the organization (Bakkalbasi, 2017). What follows is a in-depth look at each of the five assessment stages.
Within this first step, the organization is deciding what they want to know. For libraries, this may include how to improve remote services, how to improve programming, or how to improve technologies in the organization. In this step it is important to get stakeholders on board with the purpose of the assessment (Bakkalbasi, 2017).
Identifying assessment methods and data gathering
Within this stage, the organization must decide how data will be collected. This is where the debate of quantitative versus qualitative methods may be discussed. There may also be new methods and instruments available in the marketplace (Bakkalbasi, 2017). What should be not be forgotten in this stage is the importance of identifying the target population, discussing appropriate sampling technique, encouraging participation, and specifying the data collection period (Bakkalbasi, 2017).
Analyzing and interpreting the data
In this stage, significant and meaningful conclusions will be drawn from the data that is collected and interpreted (Bakkalbasi, 2017). In this stage it's important to present significant conclusions that stakeholders can understand and use, as well as present on any data limitations to enhance the "usefulness" of the findings (Bakkalbasi, 2017).
Integrating results into operational and strategic planning
In this penultimate stage, data results must be integrated into operational and strategic planning through practical and identifiable means (Bakkalbasi, 2017). In this stage is where findings are used to make future recommendations and close any gaps regarding the original question/issue (Bakkalbasi, 2017).
Disseminating results
In this last stage, clear and transparent communication of overall results should be made with stakeholders while reporting on decisions made based on the data collected (bakkalbasi, 2017).
Reference services is one area of the library organization that typically becomes the focus of assessment and evaluation initiatives (Gilman, 2018). Because reference is typically the first point-of-contact for many library users, it is important to continually assess and evaluate reference services. Tools used for the evaluation of reference services can range from suggestion boxes at the reference desk, surveys (in-house and online), interviews, observations, focus groups, case studies, and even user experience (UX) (Cassell & Hiremath, 2018). Deciding on which tool to use depends on the organization and the overall reference question/issue. What needs to be highlighted is the purpose of the evaluation. Establishing a purpose for evaluation will ensure that stakeholders, staff, and users will be fully on-board with the evaluation process. The overall purpose of evaluating reference services can lead to compelling arguments for:
evaluating practices, performance, procedures, and services of the existing reference environment;
foreshadowing reference services, formats, and practices of the future; and
marketing the value of reference services to others, such as the community at large, funding agencies, stakeholders, and the reference staff themselves (Cassell & Hiremath, 2018, pp. 413-414).
Luckily, there is a large volume of professional literature and measurable criteria that information organizations can reference when conducting assessments of reference services. The Reference and User Services Association (RSUA) of the ALA has several toolkits and resources that can be used to evaluate reference services. Specifically, I became familiar with the Guidelines for Behavioral Performance of Reference and Information Service Providers while evaluating three different reference interactions for a course discussion posts. These guidelines help map out the goals of successful adult reference transactions under five areas including visibility/approachability, interest, listening/inquiring, searching, and follow-up (RUSA, n.d.). Furthermore, these guidelines provide sub-guidelines specifically for the environment of the reference transaction (i.e., general, in-person, or remote/virtual). There are many tools resources specifically for the assessment and evaluation of LIS reference services.
This first piece of evidence demonstrates my understanding of the concept of measurable criteria and it's importance, the importance of the evaluation of professional activates, and how evaluation can help improve such services. With these three discussion posts, we were tasked with having to reach out and use reference services at three different libraries, using three different reference methods. For the first post, I used a virtual reference chat feature at a local county law library (note: this first post was originally supposed to be a face-to-face task. However, there were no libraries open for face-to-face interactions due to COVID-19.); for the second post, I called a local community college's reference help desk; and for the third post, I once again used a virtual reference chat service for a local public city library. The purpose of these discussion posts was to evaluate each reference interaction using the measurable Guidelines for Behavioral Performance of Reference and Information Service Providers by RUSA. We had to discuss the reference interaction and what aspects of the RUSA guidelines the reference librarian followed, or did not follow. I end each discussion post with a "why it matters" comment in which I discuss why that specific reference transaction matters and what could have been done to improve it based on my evaluation.
This next piece of evidence demonstrates my ability to focus on the evaluation of professional services and how evaluation can improve the design of information services. For this group project, we were tasked with having to locate a real-life website, evaluate it's current site map, redesign it's site map, and provide further recommendations for user testing and assessments. Working with two other classmates, I was mainly responsible for contributing to the existing site map evaluation discussion, writing up the redesigned site map discussion, and writing up the recommendations for further testing and evaluation. This project helped me further understand the importance of assessing and evaluating information services, as well as recommending usability testing and future evaluations in regards to library websites.
Assessment and evaluation is a necessity for an LIS organization. Through assessment and evaluation, LIS organizations can improve on their services by understanding their users better, learning where gaps are in regards to their services, and advocate for the organization by showing it's value and contributions to the community. No library service is perfect, which is why we need assessment and evaluation methods to continue to learn and develop ways to improve our services best suited for our type of users. Throughout my schooling and work experience, I learned a great deal about the evaluation process and the concept of measurable criteria. Thinking about the future, I know that I will be able to successfully contribute to any evaluation process in any information organization where I work.
Bakkalbasi, N. (2017). Assessment and evaluation, promotion, and marketing of academic library services. In Gilman, T. & Lynch, B. P. (Eds.), Academic librarianship today. (pp. 211-221). Rowman & Littlefield.
Cassell, K. A. & Hiremath, U. (2018). Reference and information services: An introduction. ALA Neal-Schuman.
Gilman, T. (2018). Learning and research institutions: Academic libraries. In Hirsh, S. (Ed.), Information services today: An introduction. (pp. 81-93). Rowman & Littlefield.
RUSA, Reference and User Services Association. (n.d.). Guidelines for Behavioral Performance of Reference and Information Service Providers. https://www.ala.org/rusa/resources/guidelines/guidelinesbehavioral
Photo credit: view of the Sierra Nevada mountain range from the Ancient Bristlecone Pine Forest, CA by: Michael Van Aken