Metrics for Effective Teaching

Measuring my teaching impact..

"These data-driven insights are a Statistician's dream :), and I practice what I preach...if my students analyse data and provide recommendations in my courses, so must I!" Dr. Letetia Addison

Course metrics have provided me with data-driven insights which contribute to my analysis of effective teaching. Over the years, I have analysed a few resources including:

Overall Course pass rates
Distributions of marks in summative assessments and final course grades
Student Feedback Forms - Quantitative and Qualitative
Student Testimonials

My Process:

In medium-sized (20 to 59 students) to large course cohorts (over 60 students), where summative assessments are mainly structured questions, such as MATH 1115, MATH 1192, MATH 2274, and MATH 2275, I have used a combination of Course pass rates and Student Feedback to form a gauge of the effectiveness of my teaching methods.

Note: See more about these course descriptions on the Course Materials section.

Impact and Growth:

In more recent times, I have incorporated in-class feedback from formative assessments such as interactive and collaborative activities to gauge the effectiveness of teaching in smaller cohorts (less than 20 students) such as DISC 1011, BIOL 6206, ECNG 6710 and PSYC 6013. See more of these in my Interactive Learning Strategies and Innovative Assessments Strategies.
I have also recently incorporated Peer Observation and Peer Testimonials as part of the effective teaching feedback loop to enhance my approaches. I plan to continue the use of these in future iterations of my courses.

Let's have a look at some of my Course Metrics to measure Effective teaching. Also see my Student Feedback and Student Testimonials for more valuable feedback.

Data-driven insights: A Snapshot

Quick Recap of Course Pass Rates for various Courses over time.

Here is a closer look into my analysis process, the insights I have gained from the data and some recommendations on the specific cohort at the end of the semester. 🤓

PSYC 6013 - Advanced Statistics and Research Methods for Psychology

Course Overview

This is a level II course in Probability Theory. It assumes a knowledge of basic Calculus and Probability as covered in MATH 1142 and MATH 1151. The course fulfils the probability requirement for the undergraduate level Mathematics and Statistics programs and is aimed at those whose future careers involve heavy use of probability and statistical methods. It provides the probabilistic foundation needed for more advanced courses in Statistics. The course is also beneficial to Actuarial Science students who intend to sit the professional exams. This semester 88 students were registered on Banner.

Selected Grade Distribution Bar Chart 2019-2020

Figure 1. Frequency distribution of number of students by grades

I produce these graphs each semester as part of my Post-course metrics for analysis. See grade score allocations here.

Insights from this cohort

Additional examples were done in class and these assisted students with their understanding of the course material.
Students were generally very consistent with assignment submission and examination attendance, in this base semester where I taught the course for the first time.
Despite the fact that some students were working professionals, most made the effort to attend classes and/or email the lecturer within office hours.

Also see my Post-course Reflection 2019-2020 and Post-Course Reflection 2020-2021, which include SWOT Analyses for course improvements.

Five Year Trend for Final Mark Distribution for 2019 to 2023

Figure 2. Bar Chart Showing Average Final Marks for PSYC 6013 course from 2019 to 2023

Figure 3. Line Chart Showing Average Final Marks for PYSC 6013 from 2019 to 2023

The line chart displays the trend of average final marks for the PSYC 6013 course over five academic years from 2019/2020 to 2023/2024. Here are some key insights:

General Trend:
- The overall trend shows slight fluctuations in the average final marks across the years.
High Average in 2019/2020:
- The average final marks in 2019/2020 are relatively high compared to other years. This could be due to a variety of factors such as the assessment methods, student cohort, or teaching approach during that year.
Drop in 2020/2021:
- There is a noticeable drop in the average final marks in 2020/2021. This year corresponds to the height of the COVID-19 pandemic, which might have impacted students' performance due to the sudden shift to virtual learning and associated challenges.
Recovery and Stability:
- Following the dip in 2020/2021, there is a gradual recovery in the average marks in subsequent years. This could indicate an adaptation to new learning environments or improvements in teaching strategies and student support systems.
Slight Decline in 2023/2024:
- The average marks show a slight decline in 2023/2024. This could be due to various factors such as changes in curriculum, assessment difficulty, or the academic preparedness of the student cohort.

Figure 4. Box Plot showing Average Final Marks for PYSC 6013 from 2019 to 2023

The box plot provides a visual representation of the distribution of final marks for each academic year from 2019/2020 to 2023/2024. Here are some insights:

Median and Quartiles:
- The medians (middle line in each box) show the central tendency of final marks for each year. The medians are generally high, reflecting good overall performance.
- The interquartile range (IQR), represented by the height of each box, varies across years. The IQR is smallest in 2020/2021, indicating less variability among student marks, and largest in 2022/2023, indicating higher variability.
Outliers:
- There are a few outliers in the dataset, particularly in 2022/2023 and 2023/2024. Outliers can indicate exceptional performances or students who struggled significantly compared to their peers.
Distribution Spread:
- The spread of marks is more extensive in some years, such as 2022/2023 and 2023/2024, compared to others like 2019/2020 and 2020/2021. This suggests that in certain years, there was a wider range of student performance.
Consistency:
- 2020/2021 shows the highest consistency in final marks, with a narrow IQR and no outliers, suggesting that students performed more uniformly that year.

Conclusing Insights

Overall, the graphs indicate how external factors (like the pandemic) and internal changes (curriculum, assessment methods) can influence student performance over time. This 5-years trend has enabled me to have a deeper understanding of the trends observed in order to improve the course in the next iteration.

Previous courses under my stewardship 2010 -2019

Course Overview

This is a level II course in Probability Theory. It assumes a knowledge of basic Calculus and Probability as covered in MATH 1142 and MATH 1151. The course fulfils the probability requirement for the undergraduate level Mathematics and Statistics programs and is aimed at those whose future careers involve heavy use of probability and statistical methods. It provides the probabilistic foundation needed for more advanced courses in Statistics. The course is also beneficial to Actuarial Science students who intend to sit the professional exams. This semester 88 students were registered on Banner.

General Statistics & Trends

The pass rate had a noticeable increase from 81% in Semester I 2018/19 to 83% to 87% in Semester I 2019/20. Approximately 45% of the class received grades in the ‘A’ bracket.
The mean coursework mark (35.34) which was higher than the previous first semester marks for the past 5 years.
The mean total mark was 68.67, which was also higher than the mean total mark in the same time period.
The mean mark in the final examination was 33.36, which reflected a similar increase.

See this link for the full spreadsheet of grades.

Figure 5. Frequency distribution of pass rates(/100%)

Figure 6. Frequency distribution of number of students by grades

Some Insights

Additional examples were done in class and these assisted students with their understanding of the course material.
Students were generally very consistent with assignment submission and examination attendance, compared to previous semesters.
Despite the fact that some students were teachers with the Ministry of Education or working professionals, most made the effort to attend classes or visit the lecturer within office hours.
Some students also made use of the Mathematics Help Centre, which I coordinate, to ensure that they clarified course material.
Some students who had low class attendance and/or were repeaters, failed the examination.
Repeaters should be encouraged to maintain proper attendance even if they think they know the course material.

Figure 7. Semester Pass rates for MATH 2275 - STATISTICS I

COURSE PASS RATE: MEAN = 86.8% +/- 8%

Minimum Pass Rate - 75%, Maximum Pass Rate - 98%

I taught at least 75% of these cohorts and although the size of classes varied over time in size, it was mostly medium-sized (40 to 60 students). It is notable that the course maintained a favourable to high pass rate over time, with supplementary materials, problem sets and solutions provided to students over the semester.

Page updated

Google Sites

Report abuse