Working Groups‎ > ‎

Metacognition/reflection

Working group topics will be determined on Friday night. This is a space for working group 4 to share information that they compile during the working sessions on Saturday.

Group members:  Michelle Withers, Elena Bray Speth, Kelly McDonald, Jenny McFarland

Why does student metacognition (or self-regulatory behaviors; reflective self-assessment) matter?
  • Persistence and Retention of diverse STEM majors – in particular, our competent but under-prepared student
  • Metacognitive skills linked to academic success and performance across disciplines
  • Teaching self-evaluation can benefit faculty (lessen the grading load in large classrooms)
Our Goals
  • To synthesize what is known about metacognition, specific for introductory biology (and biology in general)
  • Identify gaps in literature for evidence-based practices to promote metacognition early in introductory biology.
  • Provide resources and motivation for this community (i.e., "the choir"), the community of introductory biology faculty, and for our departments and institutions.  
1) Existing research on Teaching and Learning (list references, websites, etc. If possible, annotate the list)

Reference papers are attached below.

Additional resources:


2) Classroom Strategies and Resources (e.g. Activities, problem sets, modules).

A) Promoting culture of self-regulatory/metacognition
  • Transparency/buy-in (cite references; Bjork data testing effect) - use science education research data to inform students and get them to work with us and buy in;
  • Building a safe learning environment in which students can take risks and make mistakes
    • Matt Smith TEDx Talk - The Failure Bow
    • Value being wrong in class as an opportunity to learn, reveal student, naïve conceptions and construct improved mental models
    • Value all students, student questions and student roles (help top students recognize explaining things to struggling students helps the top students' learning and skills as well)
  • Valuing being a helper as well as a helpee (getting it wrong or needing help provides opportunities for other students to teach or understand better)
  • Use of Learning Assistants to help make connections with students

B) Promoting acquistion of metacognitive skills
  • Exam review activity - posted below "Test Corrections Exam 1_Fall 2013"
    • an addition to the test corrections: I (Jenny Knight) use almost exactly the same format for test corrections, but I also ask students to identify what learning goal was being tested in the question.
  • Students practice BLOOMing quiz, exam and formative assessment questions (could add this component to exam review)
  • Students use Table 3 of Biology in Bloom (Crowe et al. 2008; see reference below) to reflect on their study habits
  • Table 1 in Tanner_Promoting Metacognition
  • Students predict exam scores at beginning and end of test
  • Alignment of class activities and exam questions (being transparent)
  • Revision exercise: students develop pre-class constructed responses (or models); post-class, students revise their work and explain what they changed and why.
  • In class practice of question/answer analysis - why is an answer wrong or right
  • Using Learning Assistants
  • How to give students a "standard" to assess themselves and compare themselves against. Jenny M. captures student answers from previous terms and shares with students an 3-pt, 2-pt, 1-pt answer. Great idea!!!

C) Promoting adoption/implementation for faculty


3) Gaps in (a) Research and (b) Resources (a long narrative or a bulleted list is fine)

  1. The literature on metacognition and reflective practices is very rich but only few studies are specific to higher education, and particularly biology (a largely unexplored field)
  2. What are tools and best practices to promote metacognition for intro bio students? – although we have listed what we are doing in our courses at our institutions, this is not an exhaustive list.
  3. What are major barriers for students who don’t engage in reflective practices?
  4. What could be effective leverage points for implementation at the department and institutional levels? We have thought of some opportunities and venues where a culture of reflective learning could be introduced to students:
    • Freshman seminars
    • Study skills classes
    • Institution-wide orientation courses
    • Summer boot camps
    • Advisors and advising programs
    • Tutors and tutoring centers
    • Teaching & Learning Centers (for faculty, mostly?)

4) Ideas for New Collaborations or Ways to Scale/Extend On-going Research

1) Using ASSIST or another survey and linking to student performance
2) Identify factors/barriers to participation or buy-in of active learning/metacognitive activities
3) Simple course assessment (exploratory strategy) using grade prediction at the beginning/end of exams - could link to gender

We have very different student populations and it could be interesting to compare the findings, but many variables may confound data.
4) A simple discovery survey to get information about barriers which keep students from participating in learning activities.

Questions for students about barriers to participation

1.       What percentage of class time are you engaged in learning activities?

2.       Which of the following contribute to lack of participation?

a.       Boredom

b.      Don’t see the relevance of doing the activity

c.       Don’t feel confident in ability to perform the activity or task

d.      Don’t care about performance in class

3.       When you aren’t engaged, what could motivate you to join in?

4.       What motivates you to engage in class activities?

5.       What motivates you to perform well in class overall?

6.       Do you use feedback on learning activities, i.e. clicker questions, homework grades, exam grades, exam review?

7.       How do you use it?


Kelly McDonald
Kelly's preliminary work in the intro bio classroom with building self-regulatory skills.
Problem: students did not know how to evaluate their own understanding and knowledge. 
Question: how can we help students better evaluate their knowledge/understanding. 
Intervention: extensive exam review. Document posted below.
Post-exam, students need to 1) correct their mistakes, 2) explain why the answer they chose cannot be correct, 3) explain why they think they missed the question. In addition, Part II: 17 tools/behaviors are listed and students are asked to check which ones they used to prepare for exams and which ones they thing worked best for them. Part III: -- open-ended questions that vary depending on the exam review. ex. Do you feel your grade on the test reflects your knowledge and how much time studied? Has the exam review activity helped you in any way - explain.
This tool has been used for years, and the preliminary evaluation of its effectiveness will be submitted for publication VERY soon. Student's reflections (specifically responsed to #2 above) were evaluated as high- or low-quality score, based on how thoughtful their responses were. Students were binned based on the thoughtfulness of their reflections and their exam scores were compared across exam 1- exam 2. Quality score looks to be predictive of improvement on exam scores, but varies depending on the Quality group student is in. Other important findings - some students had difficulty identifying the reasons for there mistakes and may need more structure/support. Other preliminary findings were related to the tools/behaviors they were using - there was a trend toward more activities and better exam score. 
It appears that students revert to basic review techniques rather than generative approaches.
Available literature on exam reviews: very little on biology and very little that includes performance measures (mostly attitudinal). Need evidence to increase students' buy-in.

Take home messages: 
Reflection is a type of formative assessment; students agree that it is beneficial;
Many students mis-identify the nature of their errors;

Future questions: who are the low-reflective students? how to coach and train students for this activity? 

Michelle Withers
Students are practicing in class (correcting answers) - still unsure of what is on the test
1st semester, Freshman, MCG; have TAs, 200 students, 70% active learning
Group testing - no long term affect, although Group score is higher than all individuals
Lots of time getting buy in - tests are cognitively more challenging, but will be easier if develop skills. Can level the playing field. Students that buy-in make huge gains, but some don't buy in. The latter aren't practicing. Why? Less about preparation and more about confidence. How to help them? Affect of local community is important (confidence).
Students want more instructor - developed materials.
What are the factors that keep low performers performing low? What are questions that can be asked of these students to determine what motivates/or doesn't motivate performance.
Maybe think of TA pairs - students who move from one metacognition level to another can help others.

Jenny McFarland
Buy-in is achieved by sharing the theory and evidence about learning with the course students (citing the sources).
Students are asked to assign a bloom level to every question they see in class, as a way of developing their metacognitive skills. That helps students approach analysis questions with more confidence, so that students will not fear that they may have forgotten something they were supposed to remember. Students are also made aware of the few basic fundamental models that they should choose from when solving a problem.
Non-participation in class is not an option. Premium on "being publicly wrong" (see the TED-X "failure bow" talk). It is ok to practice being wrong in class.
Mid-term self-assessment: what is your goal in your class? how often do you: - come to class on time? take notes? complete pre-assignments?
etc. So students need to fill out a form that evidences their engagement in their learning process.
Identify and acknowledge barriers to your learning; what can and will you do about it?
Students are asked to predict their exam grade at the beginning and at the end of the test. This identifies under- and over-predictors.
Use Biology in Bloom's table 3 as a list of practices that students can compare their own study approaches to.

Elena
Problem - return exams and some students don't even pick up their exam - give points for picking it up? correcting it?
How can we get students to take more responsibility for their learning - wanted to flip class (got funded to do so). How do we convey complexity to students - learning mechanisms is hard. What should go out of the class, what should stay in? What should students do on their own?
Teach 1/5 sections and must teach according to agreed upon structure (of concepts, at least).
Have large gap - 80% headed to professional schools.
Structural info and vocab. is being shifted into screen casts out of class, then have HW assignments (vary in BLOOM level). Build SRF (box and arrow) model. Use vocabulary to build relationships. Students look over their groups' models. Go through mechanism in class, then students must review and re-do model. What did you change and why did you change it? Very compliant. Need to assign 3-4 HW/week, but can't grade it. Need to enable students to self-evaluate and correct their own mistakes.
Piloted last year - counted elements of reflection in student writing on HW and content-matched exam question. Used scale of 0-10 (7 was highest). Two populations emerged (0-3, low) and 4-7 (high). Students who had a deeper reflective skill in HW started out with lower score in terms of "mean number of processes represented in model." But they did better on Exam question.
Need to give students the responsibility and authority to evaluate themselves. "How do we build safety? Metacognitive work demands a safe community."
What about shuffling groups based on performance? Two schools of thought - Team-based learning permanent groups vs. the benefits of shuffling.
3 steps:
1. Asking students to do something cognitively challenging on their own first.
2. Learn from class - take notes and annotate.
3. Revise model and answer what you changed and why (reflection).
Has evidence across several concept areas that students with deeper reflection improve across the semester (HW1, HW2, Final Exam)



Subpages (1): Metacognition Mini-grant
ĉ
Jenny McFarland,
Nov 9, 2013, 9:26 AM
Ċ
Elena Bray Speth,
Nov 9, 2013, 10:51 AM
Ċ
Kelly McDonald,
Nov 9, 2013, 12:35 PM
Ċ
Jenny McFarland,
Nov 9, 2013, 9:27 AM
Ċ
Kelly McDonald,
Nov 9, 2013, 2:24 PM
Ċ
Elena Bray Speth,
Nov 9, 2013, 3:12 PM
Ċ
Kelly McDonald,
Nov 9, 2013, 12:41 PM
Ċ
Kelly McDonald,
Nov 9, 2013, 12:43 PM
Ċ
Michelle Withers,
Nov 9, 2013, 4:43 PM
Ċ
Kelly McDonald,
Nov 9, 2013, 12:44 PM
Ċ
Kelly McDonald,
Nov 9, 2013, 12:47 PM
Ċ
Elena Bray Speth,
Nov 9, 2013, 10:32 AM
Ċ
Michelle Withers,
Nov 9, 2013, 12:45 PM
Ċ
Jenny McFarland,
Nov 9, 2013, 9:57 AM
Ċ
Kelly McDonald,
Nov 9, 2013, 12:53 PM
Ċ
Kelly McDonald,
Nov 9, 2013, 12:55 PM
Ċ
Kelly McDonald,
Nov 9, 2013, 12:24 PM
Ċ
Elena Bray Speth,
Nov 9, 2013, 12:44 PM
Ċ
Kelly McDonald,
Nov 9, 2013, 12:56 PM
Comments