ELRC 7501 Evaluation in Educational Technology

Module 5: Mastering Data Collection and Analysis

Module 5 is where theory met practice, as we tackled the challenges of data collection and analysis in educational technology evaluation. We explored different types of data analysis and how to use these techniques to generate actionable insights. Beyond analysis, we focused on creating clear and impactful reporting plans for stakeholders, ensuring our findings informed decision-making and drove meaningful change in educational settings.

The assignment we had strengthened our ability to analyze data, create actionable insights, and present findings using digital tools. Just as in real-world evaluation work, collaboration and feedback are essential. I went with option 2 and designed an infographic that visualized different data collection methods used in educational technology evaluation.

To properly evaluate in EdTech, you're required to use a combination of data collection methods to ensure a thorough understanding of its effectiveness. Surveys, LMS analytics, Interviews, and Observations each provide unique insights that contribute to a solid evaluation. Surveys are valuable because they allow researchers to gather feedback from a large number of learners efficiently. They provide both quantitative and qualitative data, making them useful for identifying trends, engagement, and perceived effectiveness. Self-reported data can be biased or lack depth, which is why surveys are often paired with other methods. LMS analytics provide objective, real-time data on learner behavior, such as login frequency, time spent on tasks, and completion rates. This method is highly reliable because it captures learner interactions rather than self-reported perceptions. LMS data alone does not explain why learners behave a certain way, making it necessary to pair it with qualitative methods. Interviews offer in-depth insights by allowing learners to share their personal experiences with EdTech tools. This method helps discover motivations, challenges, and unexpected benefits that quantitative methods might overlook. Interviews can be time-consuming and are usually done with a smaller sample. Observations provide real-world insights by capturing how learners and teachers use technology in their classroom environments. This method helps identify usability issues and engagement patterns but may be influenced by observer bias. By combining these methods, educators can balance reliability, effectiveness, and depth, leading to more accurate and meaningful evaluations of EdTech's impact.