Evaluating OER
By the end of this module section, you should be able to:
Investigate the available reuse options for OER - adopt, adapt, combine and create
Identify perspectives on evaluating and defining ‘quality’ as it relates to course materials
Utilize relevant rubrics for evaluating OER
In the previous section, Finding OER, you focused on organizing your search and finding relevant OER. This section will focus on elements of evaluating OER.
First things first, what do you want to do with that OER?
The first part of evaluating an OER is asking yourself what you want to do with that OER. Do you want to adopt and use as is? Or, do you want to adapt and modify the content to meet your needs? If you found an OER that matched your learning outcomes perfectly, but some modification was required, does the license on that resource allow you to modify? Or, is it licensed in a way that does not allow for modifications or derivatives? If modifications are not allowed, you may want to consider another resource. So first, before diving into rubrics, consider the license for the OER and what the permissions allow.
What do I want to do with the OER I found?
Adopt
Adapt
Combine
Create
Evaluating OER Checklist
The following questions can help guide you when selecting and evaluating OER. The list below is also available in PDF format from Affordable Learning Georgia
Clarity, Comprehensibility, and Readability
Is the content, including any instructions, exercises, or supplemental material, clear and comprehensible to students?
Is the content well-categorized in terms of logic, sequencing, and flow?
Is the content consistent with its language and key terms?
Content Accuracy and Technical Accuracy
Is the content accurate based on both your expert knowledge and through external sources?
Are there any factual, grammatical, or typographical errors?
Is the interface easy to navigate? Are there broken links or obsolete formats?
Adaptability and Modularity
Is the resource in a file format which allows for adaptations, modifications, rearrangements, and updates?
Is the resource easily divided into modules, or sections, which can then be used or rearranged out of their original order?
Is the content licensed in a way which allows for adaptations and modifications?
Appropriateness
Is the content presented at a reading level appropriate for higher education students?
How is the content useful for instructors or students?
Is the content itself appropriate for higher education?
Accessibility
Is the content accessible to students with disabilities?
If you are using Web resources, does each image have alternate text that can be read?
Do videos have accurate closed-captioning?
Are students able to access the materials in a quick, non-restrictive manner?
Supplementary Resources
Does the OER contain any supplementary materials, such as homework resources, study guides, tutorials, or assessments?
Have you reviewed these supplementary resources in the same manner as the original OER?
More on evaluating accessibility will be covered in Module 6: Accessibility
Evaluation Rubrics
There are plenty of rubrics and evaluation tools available. Your department already may use one for evaluating other course material or textbooks for adoption. If they do, use that! Outside of considering if you want to exercise the 5Rs and whether the licensing on the resources allows for it, evaluating OER should not be any different than evaluating other course material under consideration for adoption.
Suggestions for easy-to-use and widely-adopted rubrics for evaluation include:
Checklist for Evaluating Open Educational Resources (OER)- From ACC Instructional & Faculty Development Department
Peralta Online Equity Rubric - From Peralta Community College District this rubric is not specific to OER, but it can be applied effectively when evaluating OER. The rubric strives to make students' online course experience more equitable and looks at addressing students' access to technology to increasing an instructors commitment to inclusion and addressing common forms of bias. It is updated frequently, with the newest version released in October 2020.
Getting Organized! Curriculum Mapping
Another successful approach to evaluate an OER is to use a course map template to track course outcomes, activities, and teaching resources. A course map, also known as a curriculum map, is a record of teaching and learning that can provide faculty an opportunity to align OER with course learning outcomes. An added advantage to course mapping is unearthing unintentional gaps or redundancies in your learning outcomes. Additionally, you can use a course map to document the license for the resource, keep track of where the resource lives online, and organize comments as you compile more resources.
To learn more about curriculum mapping, click here!
ACC Office of Distance and Alternative Learning uses Quality Matters (QM), a faculty-centered course mapping framework that assists faculty in creating a course map for their online course; list each course outcome and identify the learning resources, activities, and assessments for each. If you are redesigning an online or hybrid course, we recommend faculty contact their campus instructional designer to help set up the course map before you begin. It will save time by organizing your map as you begin to compile OER. You can learn more about QM and its approach to course mapping at ACC's Distance Learning website.
The ACC Learn OER modules were developed by adapting several existing OER; and a course map was used to track, organize, and evaluate content. The ACC Learn OER template is available for use. Retrieve a copy below, as well as a sample course map:
Reflection: A Comment On Quality
Often, in conversations surrounding the evaluation of OER, questions emerge relating to quality, such as, "is the quality of the OER as good as commercially produced copyrighted course material?" As you find and evaluate OER, challenge yourself to consider HOW you define and measure quality. Consider instead, the effectiveness of the OER meeting your teaching and learning goals.
Take a minute to read this 2013 blog post from David Wiley, On Quality and OER. After reading and reflecting, do you agree or disagree with this statement:
“For educational materials, the degree to which they support learning is the only meaning of quality we should care about.”
Knowledge Check!
Remember the OER you found in the last section? Now it is time for you to evaluate what you found.
Choose an OER to evaluate - Using a rubric or checklist evaluate the OER.
Go back to the template you filled out in Module 5 - OER Search Template - Compile
Select an OER from that list to evaluate
Choose your department’s rubric (or use this checklist - Checklist for Evaluating (OER) )
Evaluate your OER using the selected rubric (use sections as appropriate)
Map It - Practice course mapping.
Using the same course outcome and identified OER, map it!
Align the OER to the specific learning outcome(s), document the license, collect the URL, and comments. Consider sharing the course map with a colleague for further review and discussion.
Decide - Now the choice to use the selected OER is yours. Will you adopt, adapt, combine, or create?
Before moving on to the next module, review your Knowledge Check! Answers here.
Attributions
Information for this module was consulted and adapted from
"How to Find and Evaluate OER" by Abbey Elder is licensed under CC BY 4.0
"Identifying, Finding, and Adopting OER" by SUNY OER Services is licensed under CC BY 4.0
"OER Evaluation Criteria" by Affordable Learning Georgia is licensed under CC BY 4.0