Embedded Files
Evaluate programs and services using measurable criteria.
Evaluate programs and services using measurable criteria.
Introduction
Introduction
One of the most critical processes librarians and other information professionals can perform is evaluating their existing programs, services, and overall operations using measurable criteria. With limited resources and funding, it is vital to ensure that the library or institution is fulfilling its intended mission and successfully performing its duty to meet the needs of its community members to the best of its ability and in a cost-effective manner. Evaluation offers an opportunity to highlight what is working well and what needs to be changed or improved to adapt to its community members’ rapidly evolving needs.
Explication
Explication
Importance of Measurable Criteria
Importance of Measurable Criteria
Measurable criteria are the established standards by which the information institution and professionals use to determine the effectiveness and value of the current programs and services. These standards not only provide a guide that directs library operations and services but also give significance to the collected and analyzed data. “Libraries are at heart part of a social system; they exist to serve the needs of people, to help them live, learn and develop and to act as part of the social glue which holds communities together” (Brophy, 2006, p. 3). Because of this, information professionals need to know what is being achieved in order to continue to provide the programs and services that are currently successful or improve the ones where it’s needed.
Evaluation of Professional Activities and Services
Evaluation of Professional Activities and Services
At the center of an information institution’s purpose is its users. “As information organizations strive to become more user centric, they are paying increasing attention to assessing user needs and designing services to meet those needs” (Krasulski, 2022, p. 210). While assessing their programs and services, information professionals need to consider various questions, as suggested by Drucker (as cited in Matthews, 2018):
● What is our mission?
● Who is our customer?
● What does the customer value?
● What are our results?
● What is our plan?
The answers to these questions aid in developing or readjusting the programs and services necessary to meet the users’ needs.
Evaluation Plan
Like any other process, it is essential to have a plan. An evaluation plan provides a roadmap or clear direction for how and why an evaluation should be done. “The plan will help to ensure that nothing is forgotten, will assist in allocating resources and will clarify thinking about each stage of the evaluation” (Brophy, 2006, p, 10). The following topics should be considered in an evaluation, as suggested by Brophy (2006):
● What is to be evaluated: Information professionals should consider the “who, what, where, why, and how’s” of their programs and services.
○ Who is the intended target?
○ What outcomes should be expected?
○ Where can the outcomes be found?
○ Why are these programs and services being provided?
○ How are the resources allocated, and how have the activities and processes been arranged to produce the intended outcomes?
● Stakeholder analysis: internal and external stakeholders need to be taken into consideration, and research must be done to determine who will be impacted and what the concerns are.
● Agreement on the timeframe: the length of the evaluation process needs to be established, whether it will be short or extensive.
● Selection of appropriate performance indicators: performance indicators provide a way to measure what is being achieved and whether or not the institution is meeting its established goals and objectives.
● Selection of appropriate methods and the development of data collection instruments: methods of collecting data can be either qualitative or quantitative.
○ Qualitative: relates why and how something occurs, i.e., why users access specific programs and resources (this data can be obtained by observations, focus groups, interviews, etc.)
○ Quantitative: relates to numerical, measurable data, i.e., how often users access specific programs and services (this data can be obtained from surveys, transaction logs, counting, etc.)
● Consideration of linking multiple methods: multiple methods can be used while evaluating; combining different methods provides different insights.
● Selection of a sample to survey or decision to undertake a whole population study: data can be collected by sampling a part of the population or studying the population as a whole.
● Design of pilot studies: pilot studies can provide insight into the effectiveness and feasibility of a study on a smaller scale allowing information professionals to make needed adjustments and improvements.
● Revision of the data collection instruments, sampling, and distribution in light of the results obtained from piloting: the scope of changes needs to be indicated
● Data analysis procedures: suggests how data will be analyzed.
● The presentation of results: refers to how the results should be presented/disseminated
● Any ethical considerations should be analyzed: ethical standards are observed at all times.
● The plan should detail the resource requirements: this provides details on who is responsible for what.
● An initial statement should be made of the limitations of the intended study: denotes design or methodologies that affected or influenced the study.
● A statement as to who will be expected to take action on the basis of findings
Evaluation Criteria, Models, & Tools
Various evaluation criteria, models, and tools can be used for performance assessments. Each institution can use the best approach to determine how its performance must be measured. The following is a sample of what can be considered.
Matthews (2018) highlighted the seven evaluation criteria:
● Extensiveness: how much an institution provides in terms of programs, services, and resources.
● Efficiency: how well resources (time and money) are utilized.
● Effectiveness: how well the programs and services meet the institution’s objectives and the users’ needs.
● Service quality: how well the institution delivers its programs, services, and resources.
● Impact: whether or not the programs, services, and resources made a difference in an activity or situation.
● Usefulness: how useful or appropriate were the programs, services, and resources to the users.
● Outcomes: refers to fundamental changes and improvements in various areas of users’ lives (knowledge, skills, behaviors, etc.)
The Balanced Scorecard is an effective tool for measuring organizational performance. It connects an organization’s mission and vision to actions by evaluating and linking performance measures to four perspectives (Brophy, 2006; Wilson et al., 2008). These perspectives include
● Customer: as in, customer or stakeholder value, satisfaction, and retention; performance is based on the customer’s perspective
● Financial: financial performance is based on the effective use of resources
● Internal Processes: the quality and efficiency of the organization’s programs, services, and resources.
● Learning & Growth: also referred to as organizational capacity, highlights the essential role of human capital, infrastructure, and culture in an organization’s performance (Balanced Scorecard Institute, n.d.)
The balanced scorecard can assist information professionals in communicating the value of their programs, services, and resources by visualizing the cause and effect of their objectives and the key strategies that were implemented.
Benchmarking is “the purposeful comparison of processes and data” between multiple organizations, which allows information professionals to gauge the performance of their programs, services, and resources (Brophy, 2006, p. 158). Benchmarking establishes baselines for “best performance” and highlights areas that could be improved. The key measures often include the following, as listed by the Public Library Association (n.d.)
● Expenditures: the current/recurrent costs needed to support all library activities and services
● Staff: FTE librarians with master’s degrees and other library employees
● Collection: physical and digital items and items purchased, leased, or borrowed through a consortium
● Circulation: annual circulation of all items (physical and digital)
● Visits: annual visits (per capita)
● Programs: the number of programs and the total attendance.
By using the results from comparing the key measures, information professionals can improve the quality and efficiency of their programs, services, and resources by creating new processes or changing existing ones.
Evidence
Evidence
For one of my discussion posts in INFO 210, I had to initiate a phone reference session and evaluate it based on the RUSA (Reference and User Services Association) Guidelines. The RUSA criteria included Visibility, Approachability, Listening and Interest, Inquiring, Searching, and Follow-up. The overall experience was a positive one resulting from the library and the librarian successfully meeting all of the criteria. The phone number for the reference desk was visible and findable in multiple locations. The librarian who returned my phone call was welcoming, listened well, asked clarifying questions, and was able to find what I needed in a short amount of time. Afterward, I received a follow-up email to gauge the level of satisfaction. Following the guidelines led to a positive interaction which is essential when serving others. The positive interactions lead to trust and further utilization of the services.
For our final group project in INFO 232, my teammates and I had to evaluate the programs and services of a public library of our choosing using the benchmarks and measures of success established in our class readings by Amy Sonnie, “Advancing Racial Equity in Public Libraries,” and Amy Garmer, “Libraries in the Exponential Age” and “Rising to the Challenge: Re-envisioning Public Libraries.” We evaluated where the library was at, what they accomplished, and what programs and services they were still working on improving. While each of us had the respective sections we were responsible for, we all added to, reviewed, and revised each section. A significant portion of my work was the summary, demographics and statistics, and examining current library trends. Through our evaluation, we found a few areas the library needed to improve, one of which was providing more access to mobile technology (i.e., providing mobile hotspots, Chromebooks, tablets, and e-Readers for checkout). Evaluations of programs and services provide information professionals an opportunity to see what they are doing well and where they have room to improve, which is important to ensure their continued support of the community and their customers.
In INFO 281, I learned how to evaluate existing library policies, practices, programs, and services that either helped or hindered homeless customers’ access to libraries and library services. As it turns out, there are quite a few ways library management can create barriers that prevent homeless customers from meeting their needs. For my final project, I had to develop methods to enhance services for homeless customers. This assignment and the class as a whole demonstrated how evaluating services and programs and comparing them to existing criteria like customer needs could confirm whether or not libraries are fulfilling their goals and objectives and effectively serving their customers.
Conclusion
Conclusion
Library programs and services have a direct impact on their community members and customers, making it essential to continually assess and evaluate their quality and overall outcomes and achievements. Library information professionals can compare collected data, feedback, and observation to the established criteria and standards to measure performance. This process highlights their achievements and calls attention to where changes are needed. The various evaluation criteria, models, and tools ensure that libraries and other information institutions support their customers’ needs to the best of their abilities.
References
References
Balanced Scorecard Institute. (n.d.). Balanced scorecard basics. Retrieved November 5, 2022, from
https://balancedscorecard.org/bsc-basics-overview/
Brophy, P. (2006). Measuring library performance: Principles and techniques. Facet Publishing.
Krasulski, M. (2022). Accessing information anywhere and anytime. In S. Hirsh (Ed.), Information services today: An
introduction (3rd ed., pp. 203–215). Rowman & Littlefield Publishers.
Matthews, J. R. (2018). Evaluation: An introduction to a crucial skill. In K. Haycock & M. Romaniuk (Eds.), The portable
MLIS (2nd ed., pp. 255–264). Libraries Unlimited.
Public Library Association. (n.d.). Benchmark briefings. Retrieved November 5, 2022, from
https://www.ala.org/pla/data/benchmarkbriefings
Wilson, D. D., Tufo, T. D., & Norman, A. E. C. (2008). The measure of library excellence. Macmillan Publishers.
Page updated
Google Sites
Report abuse