Welcome to Structured Conversations with Faculty about Learning Analytics!
Welcome to Structured Conversations with Faculty about Learning Analytics!
Embedded Files
The Structured Conversations initiative was launched to gather perspectives from classroom practitioners about how they would like to use learning data in support of academic achievement. We use the term Structured Conversations to describe a structured, scripted process for generating, grouping, and prioritizing ideas in response to a question prompt. Sessions are organized and facilitated locally with access to a common script and set of supporting resources.
Stepping Stones: A faculty development curriculum for learning analytics use
Stepping Stones: A faculty development curriculum for learning analytics use
Your faculty have questions: help them find answers with the Learning Analytics Stepping Stones curriculum!
The Stepping Stones curriculum is a cross-institutional effort from the Unizin Consortium to prepare faculty to use learning analytics data ethically, effectively, and equitably. This foundational curriculum is focused on learning analytics available within individual Canvas course sites.
What do our faculty want?
What do our faculty want?
We asked faculty across our institutions to address this question:
“What information about students’ digital course activity and performance, or course activities and content, would be useful to you in order to support teaching and learning? “
We synthesized faculty feedback from across participating institutions and used it to identify themes and priorities.
Here's what they said:
Here's what they said:
We want to understand student behavior in order to positively impact learning.
We want to know how to use data ethically and effectively.
We want access to holistic data sets and flexible tools.
We want to analyze curricular trends.
A deeper dive
A deeper dive
The results above were generated by sorting faculty responses into high-level categories. Below, we take a closer look at the questions faculty put forward. In addition, we asked our experts to respond.
We want to understand student behavior in order to positively impact learning.
We want to understand student behavior in order to positively impact learning.
More specifically, faculty reported wanting to be able to:
Establish baselines, standards and norms
Support students and intervene on their behalf
Adjust instruction and revise course elements
I want to know how time spent accessing Canvas correlates to performance because it could inform student study habits.
I want to know how to get a unified overview of student time-on-task because I can better give personalized feedback to students.
I want to know what Canvas behaviors are used by the most successful (i.e. top 10%) students because then I could share these data with my class to encourage them to use these.
I want to know if my students who check Canvas often are the ones who struggle or the more solid learners because it would tell me if students need to be checking Canvas more often in order to be more successful.
I want to know when and if students are reviewing material before an exam because I want to be able to correlate that with performance to motivate students.
I want to know they actually read the material prior to class because I engage in flipped classroom learning.
As instructors, we all want our students to feel excited about our course content, and sometimes we use the word “engagement” to indicate motivation, curiosity, and excitement. We need to keep in mind that learning analytics do NOT provide insight into a student’s motivation. Instead, learning analytics can give us insight into the behaviors and skills that are indicators of students' success or struggle.
Our responsibility as instructors is to create a learning experience that requires students to interact with the content, to practice, and show their learning. Learning analytics can give us insights and indicators that prompt us to ask, what's going on here? Is there something that I should change? Is there some action I should take? The data can help us frame relevant, timely questions. Sometimes, our students are the only ones who can answer these questions.
Sydney Brown, PhD, Assistant Director, Center for Transformative Teaching at University of Nebraska–Lincoln
We want to know how to use data ethically and effectively.
We want to know how to use data ethically and effectively.
More specifically, faculty reported wanting to be able to:
Recognize ethical considerations and opportunities
Make valid interpretations of learning analytics data
Foster a growth mindset and positive impact
I want to know if it is appropriate to use analytics to measure student participation and give participation grades (using analytics to determine if students have watched the video) because I have been told not to.
I want to know what controls are placed on the data collected because I am concerned about student privacy.
I want to know the limitations of the analytics because it would help me know how to interpret them.
I want to know if a student is "gaming" the analytics (e.g., by faking the mouse activity) because it would be important to know to understand whether and how the analytics could be trusted.
It goes beyond how we protect FERPA data. Ethics is the why, it's the values that guide your usage or lack thereof of that data.
What are the questions that we care about? What is reasonable to do in order to answer them? This involves students being able to and having a reason to make use of the resources on their Canvas sites, and for that information to have some meaningful connection to follow up that might be done as a result of it.
Colin DeLong, Director of University Data & Institutional Reporting, University of Minnesota
We want to be able to access holistic data sets and flexible tools.
We want to be able to access holistic data sets and flexible tools.
More specifically, faculty reported wanting to be able to:
Access holistic data sets and flexible tools
Analyze quiz patterns
Raise awareness of existing Canvas functionality
I want to know if we can see more data from third-party textbook services to make our data more comprehensive.
I want to know which specific questions within Canvas quizzes & exams students had most difficulty with, because it would be easier to modify & revise assessments & tailor them to student needs.
I want to know which quiz questions have low scores because I can review that they are marked correctly and address student misconceptions.
Currently, Unizin’s course content program is focused on negotiating discounted prices for textbooks, which will be delivered through a tool called RedShelf; and including data from user interactions with e-textbooks through RedShelf in the Unizin Data Platform, Unizin’s learning analytics system. In order to provide a more complete picture of course content use, Unizin is currently working with Pearson and Ex Libris (provider of Leganto, an LTI application that links local library content to courses) to incorporate usage data from their tools into the Unizin Data Platform.
James Russell, Kyle Unruh, Sara Bolf
The Unizin Data Services Team
Many of the questions instructors have focus on understanding patterns in response to quiz questions. For instance: Which questions had the highest number of incorrect responses? Where were students indecisive in their answer selection? Canvas’ Quiz Statistics can help instructors quickly identify answers to these questions.
Instructors also want to know how much time their students spend on questions, who is struggling and if there is improvement on questions that were repeated. This type of data is not as readily available. Nuance around discovering this data and interpreting it is provided in Stepping Stones Learning Analytics Curriculum.
Heather Maness, Assistant Director of Learning Analytics and Assessment, University of Florida Information Technology
We want to analyze curricular trends.
We want to analyze curricular trends.
I want to know how students are doing in two different linked Canvas classes (e.g. lecture and lab grades because it it would be very helpful to know if students are doing similarly or very differently in the two portions of the class.
I want to know how the data compares between different semesters because I want to gauge if my teaching methods are improving.
Analyzing curricular trends involves looking at relationships between courses, e.g., prerequisite and follow-on courses, co-requisite courses, and the like. Examples include courses across levels of an academic program, general education courses and the courses they prepare students to take, and linked lectures, lab, and/or recitation sections. For instance, academic program faculty may want to learn whether a curricular intervention in a foundational course correlates with improved performance in the follow-on course. Or, faculty may want to learn how effectively a particular general education course prepares students to succeed in related disciplinary courses. For example, a writing program director might ask whether success on a final project in a 300-level general education writing course predicts success on the first major writing assignment in a writing in the disciplines course.
Unizin institutions have also used curricular analytics to understand patterns in students' post-course behavior. For instance, an analysis of curricular data helped one institution recognize that female students were leaving the major at a much higher rate than male students after struggling with high-stakes exams in a foundational course. Further, the analysis showed that female students in the foundational course did better on project-based learning than on high-stakes exams, and in fact out-performed male students on the former. Because research has shown that project-based learning typically leads to deeper understanding and stronger skills development for all students, the department decided to expand the use of project-based learning in the foundational course. As this example suggests, curricular analytics help faculty use learning data to revise curricula and instruction to promote equity and generate deeper learning and increased academic success for all students.
Gwen Gorzelsky, PhD, Vice Provost for Academic Initiatives at the University of Idaho
Page updated
Report abuse