2016-2017 Learning Analytics
2016-17 Learning Analytics
IC Workplan 2016-17
3.1 Improve the ability of colleges to develop a culture of evidence to drive institutional decision-making.
3.2b Review disaggregated data with regard to use & success in eLearning
Notes from Spring work group meeting (May 11 & 12, 2017) - Whatcom Community College
Present:
Lisa Chamberlin (Walla Walla)
Marc Lentini (Highline)
Carrie Powell (Centralia)
John Melson (Everett)
Brady Fowler (SPSCC)
Brendan Glaser (VPI at LCC/IC Liaison to eLC)
Accomplishments:
Began working with ITC to address Data Governance issues - brought eLearning into the statewide conversation
Initiated a project to link Tableau dashboards to SMS and Canvas Data - being piloted at Centralia and Walla Walla
Narrowed the scope of inquiry to 3 questions that could be answered via Canvas Data. By addressing these questions, we will be able to discover and address a broad range of learning analytics questions, including technical processes, data governance, data ethics, skill needs and work required.
1. What percentage of courses by modality/delivery mode/distance ed code are published in Canvas each quarter. Taking into account those that are typically NOT published, like Department of Corrections/Master courses/Community Education courses.
2. Of the available Canvas features, which are being actively utilized in course navigation. This would need to be sortable by modality as well as course types (aka course code/admin unit/academic org). Need to pull out manually-created courses. Also account for continuous enrollment & default term
3.What was the last activity a student was engaged in before s/he stopped participating in online class.
Generated awareness of data issues - "myth busting" - Questions about grades & enrollment cannot be answered with Canvas data, data purity/quality, ease of creating dashboards, validity of data, ethics and equity concerns in using analytics.
Created survey
Marc presented survey findings to joint eLC & ITC meeting yesterday - link to slide deck
Lisa presented the concept that eLearning now means "Everything" Learning - Canvas is a shared resource for ALL modalities - including face to face
Unfinished/Still needs to be done between now and August 2017:
Still working on proof of concept for linking Canvas Data with SMS in Tableau dashboards.
Christopher will take questions to IC:
Can IC build a "cross-commission" collaboration into their work plan?
Can resources be allocated to centralize data access, hosting and analysis?
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Notes from Winter work group meeting (February 9 & 10)
Present: Lisa Chamberlin (Walla Walla), Marc Lentini (Highline), Sukirti Ranade (Bellevue), James Umphries (Grays Harbor).
To establish a baseline and some common definitions Marc compiled a Learning Analytics Survey that will be sent out today.
Carrie said that Centralia eLearning is working with their IR and Data Integrity Consultant on linking Tableau dashboards to the Canvas API.
This work group has determined that Canvas administrators would like to answer the following questions:
1. What percentage of courses by modality/delivery mode/distance ed code are published in Canvas each quarter. Taking into account those that are typically NOT published, like Department of Corrections/Master courses/Community Education courses.
2. Of the available Canvas features, which are being actively utilized in course navigation. This would need to be sortable by modality as well as course types (aka course code/admin unit/academic org). Need to pull out manually-created courses. Also account for continuous enrollment & default term.
We will be hosting a lunch & learn professional development session on Learning Analytics today:
Report out from Today's Lunch & Learn session
Notes from Fall work group meeting (Oct 27 & 28, 2016):
Work group session at Fall eLC meeting in Spokane
Present: Kelley Meeusen (Clover Park) - workgroup chair
John Melson (Everett), Lisa Chamberlin (Walla Walla), Kimberly Lothyan (Edmonds), Andy Heiser - ITC liaison (Skagit Valley), Marc Lentini (Highline), Carrie Powell (Centralia), Sukirti Ranade (Bellevue)
Members from summer Learning Analytics workgroup that have moved over to Competency Based Education: Christopher Soran (Tacoma), Mark Carbon (SBCTC), Marty Heilstedt (Pierce), Steve Gance (SBCTC)
From Christopher Soran's notes:
Information from IC for the Learning Analytics Workgroup
I told IC that we will deliver a report of disaggregated data with regard to use & success in eLearning. I said that we will distill it down to 2 important questions/ideas and report on that.
Some of the questions/ideas for ELC to report on:
Which schools have strong completion numbers and what data are they using? What best practices do they use? Who are our leaders?
Who is using online evaluations? For example Evaluation Kit. How does the results of online evaluation data relate to completion rates and student retention? Think about ways to correlate online evaluation results with other eLearning data.
Canvas for outcomes assessment. Who is demonstrating outcome achievement in Canvas with the learning mastery gradebook? How is that data being collected and reported on?
Do you have requirements for faculty to do before they teach online and how does that correlate to student success? Do colleges that do not have mandatory training have lower completion rates in eLearning courses?
How are you using Canvas and technology for other ways beyond eLearning classes?
How does completion data or other eLearning data drive faculty professional development?
Kimberly L: Edmonds has pulled outcomes at the account level.
John M: It would be useful to report on how different institutions are using Outcomes (i.e., in this manner, in this number/percent of classes).
Lisa C: How do we dig into outcomes that inform practice that contributes to student success?
Marc L: We don't know what we don't know (what the tools can do, how they are being used).
Marc: What data is available to faculty in Canvas that can help them improve their classroom practice?
What things in Canvas can we pull data on and how can faculty get access to that and use it, who's actually doing that and how
how can we use data from Canvas to inform faculty professional development
Lisa: For example showing faculty the peaks and valleys of when students log in to Canvas and when are you assigning due dates?
John: For those of you who are interested, here's the Civitas Community Insights report I just mentioned (here at the fall eLC meeting workgroup discussion) on LMS use and student persistence. Take it with a grain of salt, since Civitas is in the business of marketing their capabilities, but it points to some potential questions we might consider asking.
Lisa: Can we look at best practices and see whether our system is doing that
Marc: Exploratory and Predictive (from article recommended by Mark Jenkins)
Exploratory: Students who don't log in until the 3rd day
Predictive: If you look at Canvas analytics you can see this data and do this with it
Kimberly: Knowing what data is useful, build joint training from it
Kelley: What data is available in Canvas that can inform best practice:
1. Easily available for faculty
2. Easy to get at account level
3. Hard to get that may or may not be clean
Andy: Need to tie it back into student completion rates
Marc: Deliverable:
1. Conversation with institutional researchers to find out what they're doing around retention and completion.
2. Find out what faculty have done with Canvas analytics
3. Work on Lit review - Marc will set up a googledoc to collect the following from workgroup participants:
What are some of the definitions (retention, completion)
What are the potholes
What are the data sources
Potential solutions
-----------------------------------------------
Members at Summer eLC meeting:
Workgroup chair - Christopher Soran (TCC)
James Umphries (Grays Harbor)
John Melson (Everett)
Lisa Chamberlin (Walla Walla)
Kimberly Lothyan (Edmonds)
Kelley Meeusen (Clover Park)
Andy Heiser - ITC liaison (Skagit Valley)
Jerry Lewis (Columbia Basin)
Mark Carbon (SBCTC)
Marty Heilstedt (Pierce)
Marc Lentini (Highline)
Mark Jenkins (SBCTC)
Steve Gance (SBCTC)
Carrie Powell (Centralia)
Jordan Lee (Shoreline)
Emails (copy/paste this to send an email to the group):
Notes from Summer workgroup meeting (Aug 9, 2016):
August 9, 2016 Work group session at Summer eLC meeting in Yakima. Present: James Umphries (Grays Harbor), John Melson (Everett), Lisa Chamberlin (Walla Walla), Christopher Soran (Tacoma), Kimberly Lothyan (Edmonds), Kelley Meeusen (Clover Park), Andy Heiser - ITC liaison (Skagit Valley, Jerry Lewis (Columbia Basin), Mark Carbon (SBCTC), Marty Heilstedt (Pierce), Marc Lentini (Highline), Mark Jenkins (SBCTC), Steve Gance (SBCTC), Carrie Powell (Centralia), Jordan Lee (Shoreline).
Mark Jenkins referenced still relevant 2012 study: Enhancing Teaching and Learning Through Educational Data Mining and Learning Analytics https://tech.ed.gov/wp-content/uploads/2014/03/edm-la-brief.pdf
Student Services, IT, eLearning, IR all intersect in the arena of learning analytics. Is there a policy infrastructure that needs to be built?
Kimberly said that data needed by Success Coaches at Edmonds resulted in Faculty union getting mired in ongoing negotiation about granting Canvas access. Dropout Detective is being utilized as a workaround.
Lisa: at Walla Walla the interest is in closing the accreditation feedback loop.
Group discussion resulted in the following action items:
1. Read the study "Enhancing Teaching and Learning Through Educational Data Mining and Learning Analytics."
2. Come up with 2 questions that we are seeking to discover from Canvas. As Marc said, this will expose infrastructure issues, ethical issues, data integrity issues. Lisa suggests: Can Canvas provide student success data that would be useful to individual faculty member, to the student, to administrators.
3. Lisa will propose to IC that we put together a 1/2 day discovery session with the IC "Learning Strategies and Innovation" work group.
4. Desired outcome: Process or product that schools could use regardless of resources to answer the two specific questions identified in item #2.