Evaluating & Integrating EdTech Tools

Introduction

As educators, our goal is to teach students, not just by transferring knowledge to them, but by creating meaningful learning experiences that support their knowledge and skill development. Digital tools and apps can help create these meaningful learning experiences. However, far too often, technology is used as a “$1,000 pencil” where it acts as a substitute to traditional teaching tools like pencils, paper, chalkboards, and whiteboards (November, 2013).

NOTE: In this chapter, we will discuss effective ways to evaluate, select, and teach with digital tools and apps. For a brief overview of key concepts from this chapter, watch the Teaching with Digital Tools and Apps video embedded above.

Transforming Teaching with Technology

Frameworks for Evaluating Digital Tools & Apps for Instruction

When you are looking for a new technological tool to improve teaching and learning in your classroom, it is easy to get swept away by the bells, whistles, and promises of something shiny and new. It is important to keep your instructional goals and sound pedagogical principles front and center as you evaluate tools.

First and foremost, do you need technology at all? Some of the brightest and shiniest (and most expensive) educational tools merely replicate the same learning experience as a typical chalk and board classroom. To really make a difference, the new tool needs to go beyond, offering improved efficiencies, enhanced communication, increased engagement, and/or the opportunity to do something that would not even be possible without the new tool (think virtual reality tours). In the following section we will present three technology integration models that you can use to evaluate and inform how you teach with digital tools and apps: the SAMR Model, the PICRAT Matrix, and the TPACK Model.

SAMR Model

Take a moment to reflect on your teaching...Are you just substituting technology for another tool by having students type a paper rather than writing one by hand, giving students a digital handout rather than a printed handout, or asking students to create a presentation slide rather than a poster on paper? Or, are you using technology to empower your students to learn in ways that would not be possible without it?

The SAMR model can be used as a guide for evaluating and selecting digital tools and apps that will create meaningful teaching and learning experiences. Dr. Ruben Puentedura (2015) developed the SAMR model to present educators with a framework for evaluating and improving their use of technology in their practice. Although the SAMR model is not founded in educational research, the simplicity of the model has made it a popular professional development tool for teacher educators, technology coaches, administrators, and professional development specialists.

SAMR stands for Substitution, Augmentation, Modification, and Redefinition and describes four means of technology integration. Watch the What is the SAMR Model and what does it look like in schools? video embedded below to learn more about each of the four parts of the model.

PICRAT Matrix

While the SAMR model is often used to encourage a linear progression of technology use from substitution to redefinition, the PICRAT matrix, developed by Royce Kimmons, adds an additional layer of consideration: “What is the student’s relationship to the technology?”

The PICRAT matrix (see Figure 1) is a grid that displays the intersection between teacher use of technology (replace, amplify, transform; cf. Hughes, Thomas, & Scharber, 2006) and student use of technology (passive, interactive, creative). According to Kimmons, “many of the most exciting and valuable uses of technology for teaching rest firmly in the top-most and right-most sections of this grid. For this reason, teachers need to be encouraged to evolve their practice to continually move from the bottom-left (PR) to the top-right (CT) of the grid” (para. 13).

Figure 1. PICRAT Matrix by Dr. Royce Kimmons is licensed under CC BY 3.0

Consider the last time you used a digital tool or app for teaching...Where did it fall on this grid?

How might you shift your use of the tool to the top right corner of the grid? Or, is there another digital tool or app that you could use instead to provide students with transformative, creative learning experiences?

Keeping these questions in mind can help you evaluate and identify digital tools and apps that will effectively impact teaching and learning.

Learn more about the PICRAT model by exploring Kimmons, Graham, and West’s article: The PICRAT Model for Technology Integration in Teacher Preparation or watching the PICRAT for Effective Technology Integration in Teaching video embedded below.

TPACK Model

OVERVIEW

The technological pedagogical and content knowledge framework (TPACK; Mishra & Koehler, 2012) provides a lens for examining the relationship among three core knowledge domains for educators: pedagogy, content, and technology.

The framework displays intersections among these three domains to represent new types of knowledge required of educators:

Technological content knowledge,
Technological pedagogical knowledge, and
Technological pedagogical content knowledge.

Using the TPACK Framework to Evaluate Content-Based Digital Tools & Apps

The TPACK Framework is particularly helpful for evaluating digital tools and apps that provide content information to students, such as digital tutorials and animations, virtual tours and games, interactive simulations, wikis, blogs, interactive videos, podcasts, data repositories, and virtual manipulatives.

When evaluating these types of tools and apps, you need to examine how information is presented (pedagogical features), what information is presented (content knowledge), and how the tool or app is designed to influence pedagogy and content knowledge (technology).

Pedagogy

How does the tool or app teach the material to students? We recommend trying every digital tool and app on your own first to experience it the way a student would. Then, use the following questions to evaluate the pedagogical features:

How is the information presented to learners? Is the content provided in multiple formats (e.g., text, audio, video, manipulatives)? Is the information chunked and organized to facilitate learning (Trust, 2013)?
What type of learning experience does the tool provide? (Hint: Review the Evaluating the Learning Experience chapter).
Does the tool provide feedback? If there is not some form of feedback, then how will students know whether they are understanding the topic and how to improve their actions?
What is the instructional quality of the feedback? Good feedback needs to be “goal referenced, tangible and transparent, actionable, user friendly, timely, ongoing and consistent, focused, proactive, and descriptive” (Create effective feedback with education technology, 2019, para. 14).
Does the tool provide goals and outcomes to guide learners’ progress? “A well-designed technology tool—whether it's a website, video, app, simulation, or tutorial—should display the learning goal so that students know what they need to master to achieve the goal” (Trust, 2013, para. 4).
Is the tool developmentally appropriate for your students? (see Few preschool apps are developmentally appropriate, report finds)
Does the tool allow students to control the pace of the material and draw connections to their prior knowledge? (Moreno & Mayer, 2007).

Content Knowledge

As you explore the app from the perspective of the learner, it is important to assess the information that is presented to determine whether it will effectively support, facilitate, and enrich students’ content knowledge. Use the following questions to guide your evaluation:

What information is presented to learners?
What type(s) of knowledge might students acquire or develop when engaging with the tool (i.e., factual, conceptual, procedural, metacognitive - review the Evaluating the Learning Experience chapter for more information)?
Is the information provided accurate, trustworthy, and credible? Are sources cited within the tool?
Does the tool feature up-to-date information?
Is text-based information well-written and free of errors?
Does the tool connect content to real world examples? Students need to know how to apply what they learn to their daily lives.
Can the content be modified or remixed? Some tools, like NewsELA allow you to modify how, and what, information is presented to support learners’ understanding of the material.
Does the tool provide links to additional resources or materials for further development of content knowledge?

Technology

In the previous chapters of this book, you learned how to examine the user experience and accessibility of digital tools and apps to evaluate how the technology impacts the learning experience. In addition to user experience and accessibility, you should critically examine the features and design of digital tools and apps to understand how and why they are created to influence pedagogy and content knowledge.

While there are thousands of digital tools and apps listed as “educational tools,” not all of these are designed wholeheartedly to support educators’ practices. Some tools are designed to replace teachers, like artificial intelligence teaching and grading tools. Other tools steal student work or data and sell it to make a profit (see the Evaluating the Cost, Privacy, and Data chapter). And, there are tools that use algorithms to make important decisions that can negatively influence teachers’ jobs or students’ academic success.

Unfortunately, “people have a tendency to treat technology and data as neutral, sterile and immune to mortal failings. Yet the digital tools we use at schools, jobs and home don’t simply fall from the sky—humans produce them. And that means human biases can and do slip right into the algorithms that have increasing power over our lives” (Koenig, 2019, para. 1).

When educators fail to engage in critical digital pedagogy, “we end up with tools — like ProctorU and Turnitin — that not only try to anticipate (or invent) the needs of teachers, but ultimately do damage by working directly at odds with our pedagogies” (Morris & Stommel, 2017, para. 11). In the article, “A Guide for Resisting EdTech: The Case Against Turnitin,” Morris and Stommel provided some suggested questions and prompts to consider when evaluating technologies as teaching tools:

What do the tool developers say the tool does? What does it actually do?
Why was the tool designed? To replace good teachers? To make money from students?
Does the tool rely on artificial intelligence (AI) to provide feedback or grade work? If so, who designed the AI program? What biases might be built into the program?
Does the tool negatively impact, or harm, a certain group of students (e.g., students who do not speak “proper” English, students who are creative thinkers, students who do not have sufficient access to technology at home)?
Does the tool make money from students’ work?
Does the tool collect and record student data without their permission (consider the use of AI tools like Amazon Echo or Siri in classrooms that are constantly recording everything that is said)?

Articles and Resources Worth Exploring

Taking a look under the hood of digital tools and apps is a critical step to ensuring that you select tools that are designed to benefit you and your students, rather than take advantage of, or benefit from, you and your students (Morris & Stommel, 2017). Additionally, looking at the intersection among technology, pedagogy, and content (i.e., technological pedagogical content knowledge) when evaluating digital tools and apps will help you uncover the purpose and impact of tools on teaching and learning and determine which tools are the most appropriate, effective, and supportive for learners.

If you are concerned that there is a lot to keep in mind when evaluating digital tools and apps, it can be helpful to create a checklist, or spreadsheet of criteria, that you find to be the most important when assessing technologies. To get started, make a copy of the Evaluating Digital Tools & Apps Google Sheet. Then, add or revise the criteria to create an evaluation tool customized for you.

Designing Instruction with Technology

Frameworks for Integrating Digital Tools & Apps in Instruction

“Don't let technology get in the way of good teaching and learning" (Steve Wheeler, 2014, para. 9).

Now that you have explored different models and features to consider when evaluating and selecting digital tools and apps for teaching, let’s discuss how you can design effective instruction with technology. According to Kurt (2017), “The instructional design process consists of determining the needs of the learners, defining the end goals and objectives of instruction, designing and planning assessment tasks, and designing teaching and learning activities to ensure the quality of instruction” (para. 3).

Instructional design models can be helpful in deciding what technology to use and how to use it to facilitate high quality student learning experiences. In this section, we will discuss three instructional design models that can guide your use of technology for teaching and learning: backward design, universal design for learning (UDL), and Bloom’s taxonomy.

Backward Design

Have you ever discovered a new digital tool on social media, or from a conversation with a colleague or friend, and thought, “I’m going to use that in class tomorrow,” and then designed a lesson entirely around the digital tool only to discover, after doing the exit tickets for the day, students learned more about the tool than the content? This can happen to the best of us.

A good way to avoid getting distracted by “new technology sparkles” when coming across new tools is to consider the end product that fulfills your instructional objective—a technique commonly referred to as “Backward Design” or “Understanding By Design” (Wiggins & McTighe, 1998).

Backward design (or backward planning/mapping) is about designing with the end in mind. Where do you want students to end up after a lesson? What knowledge and skills do they need to showcase? What are the desired results of the lesson?

By focusing on the end result, you can work backward from there to determine what content students need to learn, how to best support their learning through activities and assessments, and how they will show whether they have achieved the desired results. This instructional design process puts the focus on learning, especially higher order thinking, rather than on technology use or delivering instruction. Instead of starting with a digital tool or app and figuring out how to fit it into your practice, the backward design model will help you identify when and where to use technology to ensure students meet the end goals.

For example, imagine you are planning a 3D shapes lesson for your second or third graders and you want to incorporate some sort of digital tool and app. Using the backward design model, you would first decide what you want your students to know at the end of the lesson (e.g., recognize 3D shapes in the environment) and then find a digital tool or app to help achieve the goal(s). After exploring some shapes in the environment together as a class, you plan to let the students use iPads to take their own pictures of shapes in the environment that you will then showcase on the screen at the end of the lesson. The photos will then serve as an assessment of their understanding and achievement of the desired results.

To help you get started using the backward design model, consider what you expect to see your students produce as a result of the lesson that will demonstrate they learned what you needed them to learn. Write this down, and then, as you review a possible tool, compare its capabilities and affordances to your desired outcome (see Table 1).

Table 1

Bloom's Taxonomy

Have you ever had your class create something (e.g., project, video, poster) and found that students remembered the information from those design projects far more than what they remembered about the text you read in class?

The revised version of Bloom’s taxonomy’s features the action of “creating” as the top tier of the cognitive processes as it requires the mastery of all the other tiers (Armstrong, 2019). Remembering is at the base of the six cognitive processes. Students need to approach a subject or topic from the lowest level (i.e. remembering) and master the foundational knowledge before they can move on to higher levels of thinking and advanced knowledge acquisition. This means that when you design a lesson or activity at the top of Bloom’s taxonomy, students must work their way from the bottom up. That is, they must learn how to remember, explain, and apply information before they can analyze, critique, and create knowledge.

Bloom’s taxonomy is a hierarchical ordering of cognitive skills. Its revised edition (2001) consists of six categories: Remember, Understand, Apply, Analyze, Evaluate, and Create (see Figure 5). According to Armstrong (2019), “These ‘action words’ describe the cognitive processes by which thinkers encounter and work with knowledge” (para. 9).

Figure 5. “Bloom’s taxonomy” by Vanderbilt University Center for Teaching licensed under CC BY 2.0

Bloom’s taxonomy can serve as an instructional design tool to help you select the most appropriate apps and tools based on the level and depth of cognitive knowledge students need to acquire. Take a few moments to explore Kathy Shrock’s Bloomin’ Apps page, which features interactive tables of apps and tools for each tier of Bloom’s taxonomy, or The Padagogy Wheel V5 (see Figure 6).

As you design instruction, consider how you might start with the higher order thinking skills (analyze, evaluate, create) to encourage students to build their knowledge through active hands-on, minds-on learning experiences (visit the Evaluating the Learning Experience chapter to learn more about Bloom’s Taxonomy and higher order thinking skills).

Figure 6. “The Padagogy Wheel V5” by Allan Carrington is licensed under CC BY NC SA 4.0

For more information on Bloom’s taxonomy, explore Vanderbilt University Center for Teaching’s guide to Bloom’s Taxonomy, Iowa State University Center for Excellence in Learning and Teaching’s Revised Bloom Taxonomy page, and Christine Persaud’s (2018) Bloom’s Taxonomy: The Ultimate Guide.

Conclusion

Ultimately, when teaching with digital tools and apps, you need to spend some time deciding which tools to use and how to use those tools to create innovative, authentic, student-centered learning experiences. In this chapter, we discussed three technology integration frameworks (SAMR, PIC-RAT, and TPACK) to guide your evaluation and selection of digital tools and apps. We hope that these frameworks will empower you to use technology to create transformative learning experiences. We also showcased how to use the TPACK model to evaluate content-based digital tools and apps. By critically evaluating the pedagogical features, content knowledge, and technology together, you can make an informed decision about which tools will positively benefit your teaching and student learning.

We discussed how to use instructional design models to guide your selection of digital tools and apps based on the desired results of the lesson and cognitive processes you want your students to develop. We also reviewed how to use these models to increase access to learning, create student-centered learning experiences, and provide students with choice over what and how they learn.

Lastly, we emphasized the importance of incorporating digital literacy and safety standards into teaching and learning to prepare students for success as learners, citizens, and professionals. The DQ framework can serve as a guide for identifying the technical, cognitive, and social-emotional competencies students need when using technology.

We hope this chapter will help you to feel more empowered and encouraged to evaluate, and teach with, digital tools and apps.

References

Armstrong, P. (2019). Bloom's taxonomy. Retrieved from Vanderbilt University Center for Teaching website: https://edtechbooks.org/-dpW

CAST. (2018, August 31). The UDL Guidelines. Retrieved from http://udlguidelines.cast.org/

Choice Boards, Menus, & Tic-Tac-Toe. (2017, July 14). Retrieved from https://edtechbooks.org/-CbvH

Create effective feedback with education technology. (2019, July 2). ViewSonic. Retrieved from https://edtechbooks.org/-ZtWk

DQ Institute. (2019). DQ global standards report 2019. Retrieved from https://edtechbooks.org/-rkI

Feathers, T. (2019, August 20). Flawed algorithms are grading millions of students’ essays. Vice. Retrieved from https://edtechbooks.org/-iqgz

Hughes, J., Thomas, R. & Scharber, C. (2006). Assessing Technology Integration: The RAT – Replacement, Amplification, and Transformation - Framework. In C. Crawford, R. Carlsen, K. McFerrin, J. Price, R. Weber & D. Willis (Eds.), Proceedings of SITE 2006--Society for Information Technology & Teacher Education International Conference (pp. 1616-1620). Orlando, Florida, USA: Association for the Advancement of Computing in Education (AACE). Retrieved April 10, 2020 from https://edtechbooks.org/-vkd.

Kharback, M. (2017, February 4). A very good visual on how to evaluate educational apps to use in your class [Web log post]. Educational Technology and Mobile Learning. Retrieved from https://edtechbooks.org/-eehS

Kimmons, R. (2018). Technology integration: Effectively integrating technology in educational settings. In A. Ottenbreit-Leftwich & R. Kimmons (Eds.), The K-12 educational technology handbook. Retrieved from https://edtechbooks.org/-zqvn

Kimmons, R. (2018). K-12 Technology Frameworks. In R. E. West, Foundations of Learning and Instructional Design Technology: The Past, Present, and Future of Learning and Instructional Design Technology. EdTech Books. Retrieved from https://edtechbooks.org/-cia

Klein, A. (2019, October 2). Ed-tech usage levels are low: What should schools do? Education Week. Retrieved from https://edtechbooks.org/-LacQ

Koehler, M. J., Mishra, P., Kereluik, K., Shin, T. S., & Graham, C. R. (2013). The Technological Pedagogical Content Knowledge Framework. Handbook of Research on Educational Communications and Technology, 101–111. doi: 10.1007/978-1-4614-3185-5_9

Koenig, R. (2019 August 20). The new Jim Code? Race and discriminatory design. EdSurge. Retrieved from https://edtechbooks.org/-edL

Kurt, S. (2017, July 1). Definitions of instructional design. Educational Technology. Retrieved from https://edtechbooks.org/-pzm

Michela, E. (2018). Universal design for learning: Teacher planning for technology integration. In A. Ottenbreit-Leftwich & R. Kimmons (Eds.), The K-12 educational technology handbook. Retrieved from https://edtechbooks.org/-pWS

Moreno, R., & Mayer, R. (2007). Interactive Multimodal Learning Environments. Educational Psychology Review, 19(3), 309–326. doi: 10.1007/s10648-007-9047-2

Morris, S. M. (2017, June 15). A guide for resisting edtech: The case against Turnitin. Hybrid Pedagogy. Retrieved from https://edtechbooks.org/-rgG

November, A. (2013, February 10). Why Schools Must Move Beyond One-to-One Computing. Retrieved from https://novemberlearning.com/article/why-schools-must-move-beyond-one-to-one-computing/

Puentedura, R. (2015). SAMR: A brief introduction. Retrieved from https://edtechbooks.org/-KdSC

Shulman, L. S. (1986). Those Who Understand: Knowledge Growth in Teaching. Educational Researcher, 15(2), 4–14. doi: 10.3102/0013189x015002004

Tatter, G. (2019, February 12). Tests and stress bias: Strengthening the correlation between student stress levels and high-stakes tests. Retrieved from Harvard Graduate School of Education website: https://edtechbooks.org/-GdL

Trust, T. (2013). Evaluating Open Educational Resources: A Guide for Teachers. Association for Supervision and Curriculum Development, 8(12). Retrieved from http://www.ascd.org/ascd-express/vol8/812-trust.aspx

Vanderbilt University Centre for Teaching. (2016). Bloom's taxonomy [Photograph]. Retrieved from https://edtechbooks.org/-qrH

Vincent, T. (2012, March 4). Ways to evaluate educational apps [Web log post]. Learning in Hand. Retrieved from https://edtechbooks.org/-nxq

Wiggins, Grant, and McTighe, Jay. (1998). Backward Design. In Understanding by Design (pp. 13-34). ASCD.

Suggested Citation

Barrett, M., Zajchowski, C., & Zinn, F. (2020). Teaching With Digital Tools & Apps. In T. Trust, Teaching with Digital Tools and Apps. EdTech Books. Retrieved from https://edtechbooks.org/digitaltoolsapps/teachingwithdigital

Michelle Barrett is a graduate student at the University of Massachusetts Amherst in the learning, media, and technology master’s degree program. She completed her bachelor of education degree at Mary Immaculate College, Ireland.

Chrystal Zajchowski studied astronomy at the University of Massachusetts Amherst and graduated with her bachelor of arts degree in 2019. She is now studying secondary earth science education to get her master’s degree in Fall 2021.

Citation

Trust, T. (2020). Teaching with Digital Tools and Apps (1st ed.). EdTech Books. https://edtechbooks.org/digitaltoolsapps

Licensing

CC BY-NC-SA: This work is released under a CC BY-NC-SA license, which means that you are free to do with it as you please as long as you (1) properly attribute it, (2) do not use it for commercial gain, and (3) share any subsequent works under the same or a similar license.

Page updated

Report abuse