Assessing the Quality of OER

Why is it Important to Assess the Quality of an OER? 

The quality level of open educational resources (OER) is an ongoing concern in many fields. In a world in which “you get what you pay for” is commonly touted as a rule, it is understandable that many would have concerns about whether a free or open-licensed resource could hold up to the same scrutiny as a resource that requires payment to access. However, there are several factors that can be considered separately in order to determine the quality of an OER. Using the provided rubric in this section, individuals will gain a tool that will assist them in ensuring that the OER being accessed:

• Contains credible information

• Permits the type of usage that is desired

• Is not outdated

• Is accessible to individuals with disabilities

• Provides a complete and thorough overview of the topic(s)

• Is pedagogically sound

Are there other reasons to assess OER quality? What about other categories of quality? Use the Padlet below to contribute your thoughts.

Learning Objectives 

By the end of this guide, learners will be able to:

•  Understand how maintenance, adaptability, credibility, accessibility, completeness, and design and presentation affect the overall quality of an OER

•  Apply their understanding of rubric items by thoroughly assessing their own OERs

An Introduction to and Overview of Items Included in the Rubric 

Now that you have thought about why it is important to assess the quality of an OER before you consider using it, here is an overview of a rubric for OER evaluation. This section will provide a general overview of the items included in the rubric, and links to OERs that were used to inform the creation of the rubric. The idea for a rubric came from the resource Rubrics for Evaluating Open Educational Resources (OER) Objects, which is published by the non-profit educational reform organization Achieve. Please note that the only material directly reused from a source is the section on Adaptability. This material is an adaptation of Defining the "Open" in Open Content and Open Educational Resources, which was originally written by David Wiley and published freely under a Creative Commons Attribution 4.0 license at http://opencontent.org/definition/. Items included in the rubric cover the following topics: 

Maintenance 

• This item prompts users to consider the general functionality of an OER, as well as think about the overall maintenance of the website. For example, is the source functional? Does it show evidence that it has been updated? Are all links in working order, etc.?

Adaptability

• This section of the rubric asks users to locate the copyright for the source to determine how “Open” the source is, as well as consider how content can be adapted. We know that Creative Commons Licensing allows for five general activities (known in OER as the 5Rs) a user can engage in when it comes to OER: Reuse, Retain, Revise, Remix, and Redistribute. Generally speaking, the more activities a user can engage in, the more “Open” the OER is considered to be. This section was sourced from David Wiley’s Defining the “Open” in Open Content and Open Educational Resources. This text contains a detailed description of the 5Rs. 

Credibility

• This section prompts users to think critically about how credible the OER is and how credible the contributors are. It also asks users to consider whether or not the source contains bias, and whether or not the OER is a good fit for the intended audience. This section is a synthesis of material found in this Step-by-Step Guide for Evaluating Credibility from BYU University.  

Accessibility

• This section of the rubric seeks to comprehensively cover how accessible an OER is and provide guidance for evaluating the accessibility options that are offered as part of the resource. Some items to consider are, for example, the scope of closed captioning on videos, transcripts, enlarged/colored text, and visibility of images. This section was informed by a checklist published by the Bureau of Internet Accessibility. 

Completeness

• This item asks users to consider how comprehensive and complete the OER is. Users are asked to evaluate the overall flow of material as well as assess the quantity of resources available. This section of the rubric also prompts readers to consider what information may be missing from the overall collection. 

Design and Presentation

• This section provides guidance for users to evaluate the design, aesthetics, and navigability of an OER. This item is especially important for teachers and instructional designers who may want to use whole pieces of an OER as part of their instruction, and the rubric will allow users to make their own determinations. This section is inspired by a faculty guide published by the BCOER, who are a collection of post-secondary librarians dedicated to quality Open Educational Resources. 

The Rubric

Using the Rubric: 

• Please note this rubric is not an exhaustive list, but is intended as a comprehensive resource to help you make the best choices for your audience. 

• As you use the rubric, make sure to circle or highlight where your item falls so that you score it correctly. You also have the option to tally the total score as you work. Below, you will find some general guidance on scoring. The highest score any item can achieve is five, with the highest overall score being 30. The complete rubric is included within this document for your convenience, but the authors recommend downloading the pdf or accessing the editable .docx document. 

• Each item also contains a link to a poster that provides visual examples and explanations of descriptors in the rubric. 

Explore Rubric Examples

Below you will find a poster for each section. These posters are included as visual guides to assist you as you work your way through the rubric.  

Using the Rubric: A Guided Example

For guidance on how to use the rubric, please review this VoiceThread. An editable Google Slides version of the presentation can be found here. For a transcript of the presentation, click here.