Podcast

Transcript:

I recently had the opportunity to have a discussion about peer and self-assessment with my friend and colleague, John Taylor.

I'll let John introduce himself later in the podcast since we just jumped straight into discussion and I forgot to introduce him at the beginning.

In this brief chat on medical curriculum, we discussed pure feedback and assessment, self-assessment and metacognitive techniques.

I hope you enjoy it.

Instruction, peer instruction is what that Eric Mazur.

Eric Mazur as the guy who coined this term, gotten to you? He was like the co-author on that.

Have you heard of him before? That sounds familiar.

He's Jessica.

Turn me, you know, she she told me about them.

Nicer.

I'm watching some videos of him and it was, you know, to me it's like this isn't shocking.

And at the same time, it, it takes some guts to just be like, you know what, the way I'm explaining this is not working.

What did you guys just talk about it amongst yourselves and write proves it.

You didn't you didn't I didn't send you the video, but he did this video where he go, He says, I did this.

They were talking.

They had this whole exam and they were doing a review of it because all the students did so poorly on it.

Here again, he was doing, He's a physics teacher, I think maybe I said that the students were just not getting this one concept.

And so he did like a two-minute explanation.

Gandhi was like, get it.

And they just weren't.

You can just tell from the students spaces that they were totally blank.

Then he gave this like ten-minute explanation with all these drawings on the chalkboard and was like, and that is why this is and they still were just like, no idea.

And so he just said, well, I know half of you got this right on the exam.

So just get with each other and explain it to each other.

Sure.

Sure.

And then you could see he was watching students like it dawns on them.

And because it was that whole thing of like being an intermediate, being somebody who understands it, but it's not an expert in it can explain it in a way that's easier than for a person who is the expert, right? Yeah, I think that's one I think that's one part of it.

I think the other part of instruction as well is when you're in a small group.

As the instructor.

You can read your audience and you can tailor your instruction to the reactions that you're getting from those three or four or five people instead of a room of 100.

So, sure, fair.

So I think you can, you know, and it becomes more of a dynamic give-and-take like question, answer, explanation, question follow up, right.

Instead of here's a 10-minute blurb.

Got it.

I think there's a few different moving parts in there that contribute to why peer instruction probably works, sir.

Yeah.

So in class then like do you see that as would it, would it make sense to try to have more opportunities for small group discussion.

So not necessarily that you have to adopt this specific idea, but giving more opportunities for instructors to be working in smaller groups.

Yeah, I think I think small groups are a really powerful tool.

And whether it's instructor-led by a faculty member or whether it's pure, pure instruction.

Yeah, I think those, those both play really important roles in higher education in general.

But I think more opportunities for those types of activities is really beneficial for the student.

Obviously, we run into time crunches in terms of how much time do we have available to do that.

But absolutely, I think really well-designed activities.

Again, it comes down to how well the activity is made yeah.

Put together.

But I think those those are really important opportunities.

Yeah.

That was what I got off on that tangent about my poly psi friend.

And he was just saying how he had been reading studies about how flipped classrooms, though they seem to work in the studies that they work in.

It's because the instructor is super engaged with the process of flipping the classroom.

So it's not just like a silver bullet? No.

This this makes Learning so much better is it requires significant involvement and you can't just say like, let's flip this classroom and that makes it for a better teaching experience.

Well, yeah, no, I think, you know, and that's getting back to my comment about blinding these studies, you know, yeah, it's impossible to do, right? Obviously, the faculty member who's undergoing this type of experiment, if you will, his passionate about teaching exactly.

Who's passionate about making the experience positive and productive for the students.

So more often than not, whatever they do in the classroom is going to, whether it's a traditional didactic lecture or a flipped classroom lecture.

They're probably the types of people who put a lot of time and effort into making it a valuable, worthwhile experience regardless.

So I think all of these learning tools have the, have a place in the classroom.

I think even traditional didactic lectures or modules recordings, what have you have a really important role as well in terms of connecting with the students and inspiring the students.

That's my, if I were to, if you were to ask me what my teaching style is, it's that it's trying to inspire students to go learn it on their own.

Yeah, trying to make this engaging and accessible, exciting so that when they go home at the end of the day, they enjoy it.

Yeah.

Right.

I'm not going to teach them everything.

I can't possibly do that.

You can give me 1 million h.

I cannot teach them everything they need to know.

That's their career in medicine is learning on their own.

But if I can somehow make it fun or exciting or fascinating, whatever, you know, whatever floats your boat.

If I can do that for them, then it's not such an arduous process at the end of the day where they're like, I got to go study.

I hate this topic, I hate this discipline.

Instead they're like, Wow, this is really amazing.

I don't know very much, but I'm gonna go study and it's gonna be great.

You know, my approach to trying to be in a higher ed educator, if you will.

So what do you think have been some of the most, just so you know, I've switched into this.

This is now being part of the podcast, so, oh, yeah.

Yeah.

Great.

Great conversation because I'm using all my good material.

So can you just say all that again? But first I'm going to introduce, you know, so for those just tuning in, This is John Taylor and he's a PhD, right? Correct.

Also DO or no, just PhD.

Phd.

Okay.

And teaches in the College of osteopathic medicine.

Does what's the title of m MG pipe 31.

Is it well, predominantly I teach the medical immunology course and other associated lectures in some of the systems courses.

But I'm a virologist by classic training or yes, viral immunologist.

So I dabble in infectious diseases and immunology and microbiology and immunology is my, my main back background, if you will.

And passionate about teaching.

And we're passionate about teaching, teaching students and sharing stories and trying to make them the best, give them the tools, what would be the best Dr.

they can be.

So yeah.

So on that topic that you had set up, like you're trying to inspire excitement about learning.

What? Have there been any activities that you've done in the classroom that or, or teaching modalities that you find to be the most, to promote that type of engagement the best.

Sure.

You know, the ones that I've done personally, I think that, you know, I've tried a few different things depending on depending on the situation.

I've tried clicker questions that was more in a Zoom based environment during the pandemic.

Which I think students generally like, although I'm not sure how, how effective it is behind the scenes.

The other two.

Sorry.

What do you mean by that? Well, because they're on their own, they're isolated.

There's no opportunity for discussion.

Yes.

Right? So it's great that they get an opportunity to receive some formative feedback.

They get to answer a question, find out that they got it correct or incorrect, and then some explanations why.

I mean, that's great.

But there's no opportunity for interaction really.

So some other things that have been able to be a part of, involves some think-pair-share activities where you provide students with a question, allow them to think about it on their own, you know, preferably in a live environment? Yes.

And then they can discuss with their peers so they can discuss with their neighbors for a minute.

And that provides them the chance to either hash out an idea, discussed what they're thinking about a problem, and provide feedback to each other? Yes.

That very informal, very relaxed atmosphere.

Then the opportunity to also share with me and the rest of the class.

And typically they're more comfortable doing so having already discussed it with their peers.

Sure.

And then we can have a little mini group discussion about the problem that the concept, you know, I've done that several times and it seems to work pretty well.

It gives those students the opportunity, like I said, to chat about things to either reinforce their own comprehension of a topic or to fill in some of those gaps in their own comprehension.

Go ahead.

I'm sorry.

No, no, no, go ahead.

Well, I was going to ask with think-pair-share is that you're doing groups of two, like, well, it's, it's pretty informal and you just take a classroom.

This is a group of students of any size, whether it's 20 students or 200.

But you just ask them to informally pair up or get together with two or three of their neighbors and turn to your left and right and discuss.

There's no formality behind it.

I mean, you could do it in a formal environment if you wanted.

But the way I do it is very relaxed and informal.

You know, how do you decide on the thing that I've been thinking about with, like the peer instruction idea, which is actually it sounds to me like you're, the Think-Pair-Share is actually very similar except that you're giving even more time to discuss and then you're giving opportunity for students to actually share with the group, with the entire class, chart those, those ideas.

And it's a little bit more of a, you're, you're also part of that dialogue as well.

So they know you're giving clarification, but you're also giving opportunity for students to consider it themselves, right? How do you decide on what concepts to dedicate that that amount of time too? That's a good question.

I think.

So far when I've used that tool, I've tried to select some fairly high yield concepts that are a summary of what we've talked about.

Ok, to that point in time, or maybe they are.

So this is like an idea that maybe synthesized as a number of other, right? Right.

We've talked about it for the last five lectures or we've talked about it for the whole course.

Sure, I'm trying to bring and they're usually I usually reserve more complex higher-order questions for those activities that really require some deeper thinking.

It's not what's two plus two.

We're really trying to stretch their boundaries of fundamental understanding of a concept or a number of concepts and trying to bring them together to answer what we're narrowly would be a very challenging, maybe board style complex or USMLE style question.

Sure.

So that's sort of how I how I've used that tool.

Because those are the ones that generate the most discussion, the most creative thinking.

And I also, what's great about them is I also received some very creative feedback from students as well.

They'll raise some very interesting points like, well, what about this? Is this a possibility and maybe things that I hadn't thought about as well.

And and we go down that rabbit hole a little bit.

But when I see those little creative questions, that's when I know students are really trying to think, think analytically about a problem.

Yeah, yeah, I love to see that.

Higher level thinking.

Yeah.

Well, I thought answer was D.

D was because of X, Y, and Z.

And, you know, when I'm sitting there going, that's a really interesting angle.

And I like what you're thinking about.

And we can talk about why it's, why it's a possibility or why it's incorrect.

But, but that's what I love to see you in the classroom, whereby more by email in general, when I get emails at whatever 11:00 P.

M.

with some really off-the-wall questions.

So yeah.

Well, so one of the things that really intrigued me about the peer instruction and intrigues me about what you're doing with the think-pair-share activity is, is that, you know, looking through complex.

I think it's complex or or one of the certification the certification board that runs the complex level examinations for osteopathic medicine.

There was NPO me that they provided.

They've got their list of all of the competencies and what are the aspects of those competencies and section of that has to do with being able to communicate the Communicate professionally and be able to make suggestions to peers, and also be able to communicate to a patient in a way that is actually understandable, right? And so one of the things that I like about the both the peer instruction in the think pair share ideas and I'm kinda going to conflate them because I think that they're essentially the same in terms of what their intent is, right? I think the peer instruction is more geared towards, like you're saying, think-pair-share is a synthesis of a whole bunch of concepts, really, really high level thinking that might be seen on board exam type questions.

Whereas the peer instruction, I think, could be utilized more for difficult concepts, but not, but still have right and very clear right and wrong answers.

And you can cover a lot more of those questions in a session, then you might be able to do with the Think-Pair-Share.

The Think-Pair-Share sounds like it's a more dedicated amount of time.

Well, right.

Right.

Yeah.

I mean, in a typical so because I've done both, I've used the Think-Pair-Share activities in a typical lecture.

Power.

Then I've also had the opportunity to run or direct some peer instruction modules as well.

Okay? So, yeah, With respect to sort of think pair share ideas.

Yeah, we only get through a handful of questions in that 45-minute period and there's usually some teaching in there as well.

So there are limited opportunities for students student interactions and teaching and feedback.

Versus the true peer instruction format where typically certainly in the early months of the MSU com curriculum, most of those activities will take the form of small groups of three or four.

Teaching.

A couple of other small groups of three or four.

So we'll do it local.

There won't be broadcast at different sites, will do it locally.

And we'll have them in groups of twelfths.

So three groups of four, if you will.

And group a teacher's Groups B and C within each little 12 student pod or group, if you will.

And then they take turns teaching each other.

And in-between then the faculty facilitator, who might be me.

We'll walk through some questions, walk through some discussion.

I think that's a really valuable learning activity as well.

Because as you said, it gives students the opportunities to practice communication skills, practice teaching skills.

Those are, as you said, those are invaluable for their careers, right? Whether they like it or not, they will be teachers.

Whether it's in an official university level of capacity or whether it's just in a clinic teaching a resident or teaching a rotating student.

They are going to be educating or teaching their patients right there.

They are entering a career that is has a significant education percentage.

You and so having the opportunity to hone those skills, whether you want to call them soft skills that aren't necessarily test it on a complex exam.

Really important skills for their future careers.

And so if we can sneak those in, those opportunities in without them really knowing, yeah, they're practicing those skills because they'll they become so board exam focused, right? First two years, anytime you take dedicated time and try to give them opportunities specifically to practice skills that aren't tested.

Those usually aren't received very well because they think they're well, I think I'm wasting my time.

Well, no, you need to learn these skills so anytime we can slip opportunities in while they're still mastering the content that they need to master.

Yeah, I think is really great.

So like I said, well-designed activities that are those peer instruction models.

I've been a part of some great ones and yeah, I think more of those the more we can fit in that are high-quality and like I said, give students those opportunities.

I think the better.

How do you get the buy-in to get students to do that? Or they just interested in discussing this stuff so that they understand it better.

And so they just come to the activity.

Are they forced to come for credit? What does that rightmost? Yeah.

So most of these activities are mandatory to get buy-in.

You know, the the couple of them that I've been involved in.

I think I think they intuitively or subconsciously recognize the value in these activities.

Sure.

They know it or not.

I think they generally appreciate different types of activities so that it's not a monotonous day after day.

Again, it depends on the quality of the activity, how well it's laid out.

There's always areas for improvement.

But at the end of the day, I think it really benefits students to have those opportunities to talk out loud about concepts with their peers.

I joke with students all the time that, well, you know, hey, Dr.

Taylor, how do I study this or that? And I joke with them, but only half jokingly saying, I talked to myself all the time.

Like I tried to explain concepts to myself constantly.

I look like a Domino's Hobbes in his office to nattering away to himself.

So it really does help you understand your own limitations.

Where do you run up against that wall if you're unable to really explain something to somebody else, if you have a friend that's great, use them to try to explain.

If you don't have to talk to yourself.

But to be able to explain something to someone else takes a much deeper level of comprehension.

So by giving students those types of opportunities in these peer instructional elements, I think really does strengthen their understanding of core concepts in medical education.

And so I don't, I like to think that they recognize the value in that.

Yeah.

Maybe subconsciously.

Do you ever explicitly say that? The reason why we're doing it in this structure is because it will help you learn it in this way like that ever shared explicitly? Yeah.

I haven't met the few that I've been a part of.

I wasn't in the design phase and more urban in the implementation and delivery phase.

So someone else created the activity and then I took it and took my group of 50 and walked him through.

But that's a good point.

I don't know if that's ever been explicitly.

I'd have to check to see the notes.

I just curious because one of the things in one of the readings from this course, instructor, Laurie Shepard, I want to say her name, but she had spoken to she was writing to this group of teachers and saying, we need to revise the way that we're doing the assessment.

Because even though we've got all these different constructivist and constructionists ideas, ultimately all that gets thrown out the window when the student's main goal is to get a good grade on the exam.

So they, they are losing the value of learning, right? When all they're trying to do is this is why there's issues of academic dishonesty or things.

Because students are gaming the system.

They're trying, they're trying to get the good grade.

That's why that's still exists.

Even though they may be really happy to participate in certain activities, right? And, and so to have students, to explain to students number one, why, why you're doing something because it gives, because this is going to help you learn it better.

You're going to do better on those boards.

Like we can't divorce the academic, the pre-eclampsia curriculum from the fact that the end goal is right, you've got to pass the board so that you can keep being a Dr.

right? But the other aspect to it is explaining this is going to help you learn this better so that you can take this for longer because you're going to have to do more boards.

You're going to want to remember this for the boards.

You're going to want to remember this for practice.

Yeah.

Yeah, No.

I mean, those are all really good points.

And I will say, while I haven't used that type of language in these activities explicitly, I do extensively use that type of description in general.

In general, when I teach, right? I'll go through a concept and I'll meet you.

Good example is blood typing.

We'll go over how the immune system is involved in blood types.

And you throw up the classic blood type chart.

And I'll tell the students at the end of the day, I do not want you to memorize this chart.

Yeah.

That does you.

Absolutely no good whatsoever.

Right.

But if you can understand what's going on in the bloodstream from an immune perspective.

I can give you this blank chart.

And ten years from now, you can fill it in.

Guaranteed, right? So that's the level that we really want our students to be able to achieve is they're able to comprehend something at that deep foundational level.

They don't have to worry about memorizing it because they just get it.

It just clicks on.

Maybe for some things There's a few details that get lost in the fog, but that scaffold is still there, right? That compri, that level of comprehension of the concept is still there.

Maybe those little boxes are empty of that detail.

But a slide right back in when you open up that textbook or that paper that you're reading, right? And they'd be able to very easily identify those knowledge gaps if you know the general structure.

They've got the scaffolding.

They can see like, oh, I can't remember what the name of this is or what its functionality is.

But they can look it up very easily because it's in the context of this greater structure that they understand right from then it slips back in.

I told him that for boards, you know, most of what I'm involved in is 18 months before they take their first board exam.

Yeah.

And so, you know, that's a long ways off.

And so one of the goals that I tasked myself with is to try to present things in such a way that they'll be able to have that framework there.

Yeah, those details are going to get lost in the fog, but they can, hopefully, if they've understood what's going on in the human body and the why things are happening.

Then those details we'll plunk back end.

And I like to think that some of these peer instruction activities give them that deeper comprehensive tool to be able to really, really understand something at a deeper level.

Yeah, yeah, That's cool.

That's very exciting to hear.

And I think, I think the second thing that I was going to ask about is for those peer teaching, like the, the peer instruction groups that you're talking about where students are teaching each other in their small group, and then as a group they're teaching other small groups, right? Is there, is there a need or value in having the students assess each other's teaching afterwards, or you don't like, Is there, is there a need for a rubric for students to say like, Oh, you did a really good job covering this, but this could have been done better, worse, or on the other hand, do the students who are doing the teaching, do they do any sort of assessment checks, formative checks with the rest of the group to see.

Are they are they picking up what I'm putting down? Right.

So that's a really good question.

I mean, we don't have any of those types of assessments built-in to any of the activities that I've been a part of.

Although we do have some simple summary clicker questions at the end of each module to assess.

Which, in a way, is a method of determining how well the teachers in the room, right? I mean, you're checking to make sure that what the teachers taught was good enough that the rest of the students can answer, correct? Right.

Right.

So that's sort of one of those checks and balances and then having a faculty preceptor there to walk through any issues is another check and balance.

But we don't have any, like I said, formal feedback given to the student teachers, if you will.

I don't know how that's sort of where you start to, you know, I said earlier we slipping this in, right? Soft skill.

And so once you start adding on layers of complexity, yes.

Even some of the activities that are designed and used now, from students perspective, already have too much complexity.

Yes, they might have a pre-assessment than the actual pre-reading and the activity and then a post assessment.

And some of the feedback that we get from students is there's just a lot going on already.

So I think you have to be careful.

Yeah.

Adding some of those extra elements, would it be useful? Yeah, certainly if you were just focused on that soft skill of how well the teacher are you? Yeah.

Then again, sometimes in these small groups, only one or two people do the talking.

So who are you really assessing? Are you assessing the group or meiosis? So you'd have to have to be very thoughtful in how you design something like that.

What are that belongs in year one or year two or maybe maybe in clerkship years, years 3.

4 where they I think it's those years where they're out there in the clinic where they really start to see the value in being able to communicate effectively.

And getting the early years they blow off communication.

Even though it's such an important skill.

And I think it's one of those soft things that they sort of, wow.

And it's not until they're out in the clinic where it might hit them in the face and realize, oh, I need to be able to talk to peers, talk with my patients.

You know, it sounds like something that would maybe get nominal buy-in if it was a, an elective, but hard to tie into like a course on virology.

Well, now, now today we're going to assess your teaching skills.

Yeah, certainly in the basic sciences.

They do tons of communication work in in their osteopathic pay.

Sure.

I mean, that's that's sort of where classic communication is covered.

Maybe not so much in the sense of formal teaching, but certainly in communicating back and forth.

So, but yeah, it'd be hard to certainly in a foundational course or even an organ system course.

Yeah, you just have to be very thoughtful and deliberate in how you design it and how you, how you make sure it's worthwhile for the students and that they don't.

Because nowadays that's the, I hate to say it, but they're very focused on those end goals of board licensing exams.

And if it's not immediately pertinent on in the mind, you'll, you'll probably get flack for word.

It's important, even though it's important, it's critical for their future as a position.

So it's right.

I think there is inherent value, but at the same time you're, and you don't want to bog down the class either.

You don't want to take too much time to focus on something that yes, it's important, but so is like typing skills and we're not going to do like a section on typing, right? Like it's, you already did the work of teaching.

You could even give students like, here's some techniques for, or here's some tips for as you're presenting this concept, things that you'd want to do.

But beyond that, you really don't necessarily want to give assessment of their teaching skill because they're like, well, that's not why I'm here.

I'm supposed I'm trying to be a Dr.

yeah.

Yeah.

Certainly in the formal environment.

Like I said, I mean, you brought up electives and we've done a couple of little things.

It wasn't even an elective course, it was just more of a volunteer outreach program.

And what column where we gather up some students and we put together a module or a topic, and they go to a local high school and they give some classes.

They get some sessions to an API high-school class, biology high-school class.

And in that context, we're dealing with only half a dozen students, but we're as a faculty member, I'm an ally.

I'm able to then really adjudicate, reevaluate someone else's teaching skills, how they put the presentation together.

Right now, what are they focusing on, how it is, how are they explaining something, some other tactics that they're avenues like that where we certainly do have opportunities to help students in a more formal way with your teaching skills.

But in a class of 300, that gets challenging, right? So not everybody wants to, certainly with formal teaching, not every wants to be a formal teacher, right? Yeah, That's more of a very select few end up taking the academia root.

Okay, right now we are at 02:00.

Thank you so much.

Seriously? Like this was yeah.

No, no.

Yeah.

Fascinating and interesting to discuss.

So absolutely.

Thanks.

We'll talk to you later.

All right.

Have a good one.