Working group topics will be determined on Friday night. This is a space for working group 4 to share information that they compile during the working sessions on Saturday.
Group members: Michelle Withers, Elena Bray Speth, Kelly McDonald, Jenny McFarland
Why does student metacognition (or self-regulatory behaviors; reflective self-assessment) matter?
1) Existing research on Teaching and Learning (list references, websites, etc. If possible, annotate the list)
Reference papers are attached below.
2) Classroom Strategies and Resources (e.g. Activities, problem sets, modules).
A) Promoting culture of self-regulatory/metacognition
B) Promoting acquistion of metacognitive skills
C) Promoting adoption/implementation for faculty
3) Gaps in (a) Research and (b) Resources (a long narrative or a bulleted list is fine)
4) Ideas for New Collaborations or Ways to Scale/Extend On-going Research
1) Using ASSIST or another survey and linking to student performance
2) Identify factors/barriers to participation or buy-in of active learning/metacognitive activities
3) Simple course assessment (exploratory strategy) using grade prediction at the beginning/end of exams - could link to gender
We have very different student populations and it could be interesting to compare the findings, but many variables may confound data.
4) A simple discovery survey to get information about barriers which keep students from participating in learning activities.
Questions for students about barriers to participation
1. What percentage of class time are you engaged in learning activities?
2. Which of the following contribute to lack of participation?
b. Don’t see the relevance of doing the activity
c. Don’t feel confident in ability to perform the activity or task
d. Don’t care about performance in class
3. When you aren’t engaged, what could motivate you to join in?
4. What motivates you to engage in class activities?
5. What motivates you to perform well in class overall?
6. Do you use feedback on learning activities, i.e. clicker questions, homework grades, exam grades, exam review?
7. How do you use it?
Kelly's preliminary work in the intro bio classroom with building self-regulatory skills.
Problem: students did not know how to evaluate their own understanding and knowledge.
Question: how can we help students better evaluate their knowledge/understanding.
Intervention: extensive exam review. Document posted below.
Post-exam, students need to 1) correct their mistakes, 2) explain why the answer they chose cannot be correct, 3) explain why they think they missed the question. In addition, Part II: 17 tools/behaviors are listed and students are asked to check which ones they used to prepare for exams and which ones they thing worked best for them. Part III: -- open-ended questions that vary depending on the exam review. ex. Do you feel your grade on the test reflects your knowledge and how much time studied? Has the exam review activity helped you in any way - explain.
This tool has been used for years, and the preliminary evaluation of its effectiveness will be submitted for publication VERY soon. Student's reflections (specifically responsed to #2 above) were evaluated as high- or low-quality score, based on how thoughtful their responses were. Students were binned based on the thoughtfulness of their reflections and their exam scores were compared across exam 1- exam 2. Quality score looks to be predictive of improvement on exam scores, but varies depending on the Quality group student is in. Other important findings - some students had difficulty identifying the reasons for there mistakes and may need more structure/support. Other preliminary findings were related to the tools/behaviors they were using - there was a trend toward more activities and better exam score.
It appears that students revert to basic review techniques rather than generative approaches.
Available literature on exam reviews: very little on biology and very little that includes performance measures (mostly attitudinal). Need evidence to increase students' buy-in.
Take home messages:
Reflection is a type of formative assessment; students agree that it is beneficial;
Many students mis-identify the nature of their errors;
Future questions: who are the low-reflective students? how to coach and train students for this activity?
Students are practicing in class (correcting answers) - still unsure of what is on the test
1st semester, Freshman, MCG; have TAs, 200 students, 70% active learning
Group testing - no long term affect, although Group score is higher than all individuals
Lots of time getting buy in - tests are cognitively more challenging, but will be easier if develop skills. Can level the playing field. Students that buy-in make huge gains, but some don't buy in. The latter aren't practicing. Why? Less about preparation and more about confidence. How to help them? Affect of local community is important (confidence).
Students want more instructor - developed materials.
What are the factors that keep low performers performing low? What are questions that can be asked of these students to determine what motivates/or doesn't motivate performance.
Maybe think of TA pairs - students who move from one metacognition level to another can help others.
Buy-in is achieved by sharing the theory and evidence about learning with the course students (citing the sources).
Students are asked to assign a bloom level to every question they see in class, as a way of developing their metacognitive skills. That helps students approach analysis questions with more confidence, so that students will not fear that they may have forgotten something they were supposed to remember. Students are also made aware of the few basic fundamental models that they should choose from when solving a problem.
Non-participation in class is not an option. Premium on "being publicly wrong" (see the TED-X "failure bow" talk). It is ok to practice being wrong in class.
Mid-term self-assessment: what is your goal in your class? how often do you: - come to class on time? take notes? complete pre-assignments?
etc. So students need to fill out a form that evidences their engagement in their learning process.
Identify and acknowledge barriers to your learning; what can and will you do about it?
Students are asked to predict their exam grade at the beginning and at the end of the test. This identifies under- and over-predictors.
Use Biology in Bloom's table 3 as a list of practices that students can compare their own study approaches to.
Problem - return exams and some students don't even pick up their exam - give points for picking it up? correcting it?
How can we get students to take more responsibility for their learning - wanted to flip class (got funded to do so). How do we convey complexity to students - learning mechanisms is hard. What should go out of the class, what should stay in? What should students do on their own?
Teach 1/5 sections and must teach according to agreed upon structure (of concepts, at least).
Have large gap - 80% headed to professional schools.
Structural info and vocab. is being shifted into screen casts out of class, then have HW assignments (vary in BLOOM level). Build SRF (box and arrow) model. Use vocabulary to build relationships. Students look over their groups' models. Go through mechanism in class, then students must review and re-do model. What did you change and why did you change it? Very compliant. Need to assign 3-4 HW/week, but can't grade it. Need to enable students to self-evaluate and correct their own mistakes.
Piloted last year - counted elements of reflection in student writing on HW and content-matched exam question. Used scale of 0-10 (7 was highest). Two populations emerged (0-3, low) and 4-7 (high). Students who had a deeper reflective skill in HW started out with lower score in terms of "mean number of processes represented in model." But they did better on Exam question.
Need to give students the responsibility and authority to evaluate themselves. "How do we build safety? Metacognitive work demands a safe community."
What about shuffling groups based on performance? Two schools of thought - Team-based learning permanent groups vs. the benefits of shuffling.
1. Asking students to do something cognitively challenging on their own first.
2. Learn from class - take notes and annotate.
3. Revise model and answer what you changed and why (reflection).
Has evidence across several concept areas that students with deeper reflection improve across the semester (HW1, HW2, Final Exam)
Working Groups >