El peer teaching triplica la ganancia de aprendizaje con respecto a la metodología tradicional

posted Mar 27, 2012, 8:14 AM by Alfredo Prieto Martín   [ updated Apr 21, 2012, 1:37 AM ]
Trabajos del Grupo de Eric Mazur en Física  y del grupo  Allen en biología demuestran la eficacia del peer teaching  para aumentar la comprensión conceptual en asignaturas de ciencias. Los pdfs de los artículos que lo demuestran están más abajo. Aqui mostramos el podcast de la famosa conferencia de Eric Mazur Confesiones de un profesor convertido. Me he tomado la molestia de realizar una transcripción del audio para que entendáis el mensaje de Mazur. Escuchar esta conferencia cambiará vuestros puntos de vista sobre lo que debemos hacer para que nuestros alumnos aprendan. 

Vídeo de YouTube

Transcription of the Mazur podcast

Presenter:

So as dean of learning and teaching in the college of Science and Engineering I like to welcome you to this general interest seminar from the School of Physics and Astronomy and also jointly hosted by the Institute for Academic Development, and also extend the welcome to those people that not only can see me, but to hear me on the web cast of this lecture.

Most of all it is a great pleasure to welcome and introduce to the speaker, Eric Mazur, professor of Physics and Applied Physics at Harvard University. I think it is no exaggeration to say that Eric as scientist excels in really all three areas of academic scholarship:  research, of commercialization and of knowledge exchange, and of teaching development and delivery.

He is a distinguished researcher in optical physics with over 213 Scientific publications in twelve .In 2006 he set up a company  to commercialize black silicon a material developed in his lab but is here to give us a talk today on another element of his repertoire for which is equally well renown and is  in the area of physics education research.

Over twenty years ago he became developing a instructional technique that become known as peer instruction he did quite literally write the book on this technique, as many of you realized as a educational method has been adopted far beyond the subject of physics in many universities all over the world and is profoundly changed the way that lecturers view and use the face to face class time in lecture theaters like this, with students.

Eric talk today is going to describe his journey from a traditional lecturer to a converted lecturer so please joint me, in welcoming him, to delivery his talk.

applauses

Eric Mazur:

Well there are a Lot of you here, I guess you most all like public confessions. (laughs) Frankly is a bit intimidating I rather do this, one at one.  I guess I just have to remind myself the American   proverb:  “The more you confess, the more books you will sell”. (laughs)

 Anyway lets get started First confession: When I start teaching at Harvard, long time ago 1984   twenty seven years ago. really? …yeah it was twenty seven years ago. When I start teaching, I never ask myself the question how I am going to do this? When you Start doing something you have not done before, the first question that should come up in your mind. Didn’t. It was perfectly clear to me how was I going to teach I was going to do to my students what my teachers have done to me.

Well almost a unconscious decision and I think in retrospect in my own self defense Is Not that surprising that you tend to Project your own experiences on to the other around you    . And I Naively thought that I have learned everything I knew in the classrooms taking notes in settings not very different from this one.

Anyway so I started lecturing and as a beginning assistant professor at Harvard I was to teach the courses none of my colleagues wanted to teach: the lecture course “Physics for premedical students”. (laughs) They didn`t want to learn physics,  they were postponing the course as long as they could to the junior to senior years  they have to take it because it was a requirement premedical requirement. And you know most of the students have nothing but Horror stories about physics so they are not very disposed to be very friendly to their physics instructors. But you Know what happened to me? Was that very quickly the signals I was receiving what there, I was a good teacher. By the traditional standards I was doing a good job.

What are the traditional standards? I can think of two of them. One is the dreaded end of semester questioner. Most of my colleagues when they taught that course can close to Committing suicide at the end of semester when they saw the remarks students made about their teaching. But not so for me. We used a five point scale and I get Four point two four point eight on a five point scale or anyway in the range. So Clearly The students were happy which I was did in the classroom. That is one metric.

 What is Another metric? Another metric  is to look at the performance of students on exams.  Normally when physics is taught to premedical students in the US, I don’t know how you do it in the UK in physic courses but in the US for premedical students you avoid calculus is an algebra based course. Not so in Harvard we give Full blown Calculus based courses.  I could get the Students do Pretty complicated exam problems. Exams problems I was quite sure i do it well in time pressures of a exam involving Triple integrations and you named it  By large students did well. So in short they like what I was doing and they did well in what I considered complicated exams. Obviously I was doing the right thing.

There went for I don’t know for Six, Seven years until something happened that made me tumble my mind before I got there. I should tell you that All along there were Signs that  something was wrong but I simply ignored that signs. I give you an example, In spite of giving me high ratings on my end of semester questioners some students Write down: “Physics is boring” or “physics sucks”  I never know to make any sense of this remarks So I just Focus on the positives and ignore those negatives.

Same thing  I go to a party and people  ask me what I do for a living “I am physicist” “Oh such a Hard times is physics in high school or in college”or My dentist who recently told me I wouldn’t told back because the thing  in my mouth (laughs) “Ah physicist, physics,  I gone a Grade A for physics in college  but I really didn’t understand anything.” (laughs)

 This create a Sense of embarrassment and Especially you know I never knew where this remarks are coming from  I couldn’t imagine of somebody  who think of physics as

Could be considered been boring so to me It didn’t make any sense and I ignore it. But then In 1990 I read about a series of articles in a Journal called The American Journal of Physics which I normally didn’t read very often which occasionally has an article about education.

Now before continue I should add a disclaimer. I am a physicist so I will use the word physics a few times. You know, I give you few examples containing physics but this talk is not about physics. I am sure that a lot of that I am going to be confessing about is equally applicable to your own discipline. So if you are a chemist in the audience  you  substitute chemistry every time I say physics. If you are of the humanities do the same thing and There will be things of value to any discipline represented in this audience there will however be a few examples of physics simply because I want to make concrete.  Don´t worry if you don´t understand the physicists because is not about the physics is about the ideas on education and pedagogy,  

In order to describe to you what happened in 1990 I have to describe the content of those articles to you. The series is of four  articles and the first one describe the test called the Force concept inventory it is a thirty questions survey of 29 or 30 Questions survey  Testing students conceptual Understanding of Newtonian mechanics essentially Newton three laws.

Well, The typical curriculum in physics Is that follows. The first few weeks one or two were spent describing motion, acceleration, blasts, if you don’t understand motion is very hard describe anything in physics. Then comes the connection between forces and motion as embodied by the Newton three laws and then everything else builds on top of that. If you don’t understand, a solid understanding of Newtonian mechanics in a physics course it is very, very difficult to make sense of anything else. Yes you can maybe do the procedures but you actually make sense is very hard.

Let me give you what turns out to be the hardest question on the FCI  force concept inventory. A heavy truck and a light car Collide head on. The force during the collision, the force exerted by the heavy truck on the light car is:

A larger than that of the light car on the heavy truck

B they are equal to one another.

C the light car exerts a larger force on the heavy truck than the other way around

D there are not exerting any force on each other the are just in each other way. (laughs)

Are You laughing about that. I think no physicist in its right mind will ever come up such an answer.

Its is a difficulty of design of multiple choice question If you know your field is very easy to come up with the right answer but its very difficult come up with  plausible wrong answers because your mind does not work the way the mind of students work . So the way professor author of this test, Hestenes, a physic professor, now retired, in Arizona state university the way he designed this tests was not by  trying to invent correct distracters for the multiple choice test no, he give the free response questions and tabulate the most frequent incorrect  responses given by the students.  This one                        

I can’t remember because my mind doesn’t work in that way.

In spite of the fact that most students have no trouble reciting  the Newton’s third law who said “action is reaction the force of A on B is the same magnitude of the force of B on A”. When B and A are replaced by a small car and a heavy truck all of them suddenly they forget about Newton’s third law.  They are convinced that the truck exerts a force than the small car than the other way round.

In another article Hestenes shows data collected from about you know a dozen different universities and high schools in the southwestern US: Californian and Mexico Arizona and the interesting thing is that these tests does not contain any jargon, for example when I told you the question I just told you probably in your mind ready to formulate an answer before remind Newton’s third law. There is no jargon there is a word like force but we develop a concept of this notion of force long before we take a physics course so if you can stop anybody here  in the street of Edimbourg  you can understand any of the questions and formulate and answer. They can probably articulate the right answer.

That means that you can give this test to students  before the instruction and then you can repeat at the end of semester to see how much the instruction has done to change  misconceptions  that students had entering in the course. And One of the things has this test show is that. Is hardly any gain? (laughs)

Does make that Much differences At the beginning  and The end of the semester ? In fact there is no even correlation even semester readings for professors adward winning teachers we recognized publically are not doing any better than the worst boring dry teachers you can imagine.

So I look the data and thought No way, Not my students , not Harvard’s.  I feel challenged. To me is disgusting  

Most of the students in Harvard obtain advanced placement for physics course  Five out of five a high a score in order to place to try a physics course. Lets and placed out in a physics course

So I Thought this was not something with had any relationship with I was doing in my classroom but I feel challenged I thought you know “I gonna show that in my class the results are very different”. Unfortunately it was too Late to do a pretest we were ready half going in the semester but I was dying to show that my students with As do this tests.

So one day I walk into the classroom I said look I will Give you a quest you know if you said test they fret so don’t worry about this quest is not going to affect your final grade in any way is just to measure the efficacy of my own teaching. Suddenly as I said that, I realized that  I take the way any incentives to take this quiz right if you tell a premed(icine) is not gonna to affect your final grade show up for actually coming I said look  show up A reason for actually coming up.

I said look Next week examination caming up.  If you take this quest seriously it will pinpoint any difficulty you have as soon as I said I realized a total shunt slide. (laughs)

We were you know Discussing rotational dynamics for the physicists and systems  engineers to apply  calculate the moment of inertial switch dimensional objects by carrying out triple integrations I mean This was so way beyond the questions  and the type of questions I mean way beyond the question and the types of questions.

I was worried that my students get mad of me for wasting their time with something so trivial.

Oh boy, My worries Quickly Dispel because hardly are they taking in their seats in the computer classroom when we administer this test hang his hand and  he said: “Professor Mazur, How should I answer this questions according with you taught me or according the way I usually about this things?” (laughs)

I have no idea to answer the question (laughs) and by the time the test have been completed I have been draught of mi ivory tower. It was clear that there was something really serious going up.

SLIDE ONE MY MESSAGE

Shift focus from teaching to helping students learn

Anyway what I like to do in this Fifty minutes or so Is Get this point across. That’s my  message  I promise you.  keep awake.

Its namely that we should shift the focus from teaching to helping students learn. The idea of teaching is almost mad. You can’t put the knowledge in people’s head. The best I can think to do now is Help students learn. The more I give this talk many times and the more I say this statement the more it becomes more obvious to me,  and maybe is already obvious to you but unless at least  is obvious to me.

SLIDE 2 OUTLINE

·         Education

·         Peer instruction

·         Results

 

I’ll do this in Three parts. First I like to Talk a little about education. Many of we here are involved in education we doing work in education institutions but  if we were all take a little piece of paper and I give you a minute o two right down a  concise definition of the word education and Pulling out Black berries and i phones looking at that on line but even if you do that the definition to you find in Dictionaries is not very illuminating.   

I think many of us is There is struggle to define What it is what we mean by education. It is weird ,right, because  we are doing education so  We should have no any trouble defining what is it. I want to talk about a little bit What is it exactly . What is it we trying to accomplish? what do we mean when we say educating people? then I like to Tell you about peer instruction what I did in response to this rather Disarming Discovery that my students were learning nearly as not much as I wanted, in fact they was learning nearly as to nothing I was show you because they even don´t internalize Newton’s laws.

And In the last we will show results because I think is very important that we all apply the same scholarship, the same rigor to approach to  teaching as we doing in our discipline. I often shocked in my own department of Physicists. When I joint to the Department of Physics of twenty seven faculty six of them have won Nobel price we can hardly thing more distinguished body of Faculty but whenever over faculty launches a  discussion to talk of  education any hint of scientific approach, any hint of Scientific method would  disappear  promptly is not longer be about you know: hypothesis, collecting data, disproving or solidifying the process. Nothing of this processes for advancing the approach to teaching, No.  “My students liked it when I do a lot of demonstrations” so what ? I didn’t bear to ask any such questions or “you know, my students learn a lot off better when they saw a lot of problems in the blackboard “ What did you mean learn better? What are the data, show me the data.

I think is important that we  be rigorous. We collect data in essence I see  My classroom as an opportunity  to gather data  and I show you the data in the next section of my talk.  Liz Surman who used To be President of the Carnegie foundation has a Marvelous quote so next time you talk to a colleague tries to uses   anecdote of evidence to prove that one method of instruction is better than other. The quote is the following “The plural of anecdote is not data.” (laughs)

 Is a great quote. You Don’t validate an argument simply by telling a lot of anecdotes. That’s what I’ll do a lot of anecdotes.  So Keep you awake, OK. (laughs)

Slide 4 Education 

(progesive animation) lectures focus on delivery of information

 

So Start with education. This Is a Picture me teaching before I am converted I am standing there the Over Head projecting  my notes In a space that is not very different from this space and probably not very different  to the 99,9% of learning spaces around the globe or dating back to the Ancient Greek amphitheater designed with this one sole purpose: to focus the attention of many on to a few, the performers, the you know the theater, music,  or teaching in this case.

I would like you to describe to me the process that is occurring there on the screen behind me in a few words and I don´t want a verb to be either teaching or learning. So What is the activity that is taking place in the screen here?

Transmitting. You must have seen my talk before. (laughs)

What is  being transmitted? Facts, or information

 

Slide Animation Lectures focus on delivery of information

I think Perhaps the best way of characterizing the lecture is to say that Is a process were by  the information gets transferred from instructor to the students. It sounds ironic because the word Lecture means exactly that, is a reading of information. I think before the invention of the books This was probably the only way of transmitting information from one generation to another.

Of course the book was many hundred years ago so Why didn’t we abandon this lecture process this lecture method five hundred years ago? Well I guess until the industrial revolution  books as another commodity  but you know what My students have actually rapidly  in my face that I was doing  and I gonna accepted rapidly In my face

I gonna Tell you what happen

When I was assigned a course , Physic 11 B , The question that came up in my mind was not how I am going to teach? What was the first question that come up in my mind? what book I am going to use?

I went to colleague that taught  this course before and I said What book you used in this course and he said “oh we use the Resnik “you should order  Copies for all of the students .

of Resnik has been a classic for what sixty years I don’t know for long time

Anyway so I went  to the bookstore and ordered one hundred a 50 copies of Resnik textbook needs to be done. I don’t know how it is in the UK I got educated across the channel in Holland you don´t buy a book solely for a introductory course. Why spent a hundred and fifty dollars, or the equivalent, if the instructor is presenting the material to you? So, but anyway, I wanted the book as I went back from the bookstore to my office I thought: “Wait a minute if all the students have the book what I am going to do?” (laughs)

So I`m Back to my colleague and ask him the question. He said Don’t worry the important point is collection of introductory physic textbooks in the book selves. There are many introductory physics textbooks So Just Pick another one. (strong laughs)

So I look at the books and very quickly I was wondered I really like one For two reasons One it used a different approach one starting Newtonian mechanics started with inertial momentum I really like that, but the really reason eventually I choose the book , is the book was it was out of print, (laughs) students can’t get at  hand the book so  is the perfect book to prepare my lecture notes. So I can prepare lectures notes from the book and then in class I did Project lecture notes or I will write them on the board the letters you see. And because my approach was different I decided to hand out a copy of my lecture notes for the students. So as they walk out of the classroom at the end at the back there are stack of photocopied lecture notes from they to take with them at home.  Why do you think I handed the notes  at the end of class? So they wouldn’t just take in and go away but you now. is already Somebody admitting that there are something  wrong? (laughs)

Because Why force  they trip come and listen us professors delivering information If they can get the same information just really in the lecture notes why are they not doing that way?

That question didn’t come up to my mind. But what did happen was that after about three or four weeks Some students come to me and said Prof Mazur can we get a copy of the lecture notes at the beginning of each class period this is  for  don’t have to write down as much.

And I indeed did notes of Blackboard I handed and one hundred fifty pens in the note books of the students

Someone Describe the lecture method a s the process by the lectures notes of the teacher pass to the notebooks of the students without passing by the Brain of either. (laughs) What is Precisely what was happening in my class  I was taking all this time To take the information Out of the book, transfer into my lecture notes  and then In class the lecture notes are transferred to the note books of the students but the place were the information really needed to get the brains didn’t get.  So I started to hand out the lecture notes at the beginning of each class  period .

And much To my relieve the students would  just walking, grab the notes, and walk out, No.

They walk then,  grab the notes Seat down and They still copy everything  because they have  no time to read handouts. Next time I was teaching this same course I said you know what rather than handling these notes piecemeal  I gonna collected a set of lecture notes at the beginning of the term  sent it all for duplication and then every student has complete bucks for the entire semester .  You know what the unexpected result was? At the end of the semester half a dozen students wrote in the comment section of the semester evaluation: “Professor Mazur Lectures straight from their lecture notes” (laughs) Hello (laughs)

I mean what was supposed to do? Lecture from another set of notes that were Different from the lecture notes I handed out? (laughs) I was mad, these ungrateful students. But you  know if you stop to think about it, They have a point: I was focusing on delivering information.

So that leads us to the question: Is that, what education is? is education the transfer of information? Can say any hard In favor this statement? None, So You most all have a good idea about What education is, please tell me.  What is missing?

 thinking about something, internalizing  something, Interaction between the student and the teacher, and the other students,  making sense, making  meaning, that’s right .

Is not to have information and only be able to replicate that information. We want Student do more, right. Whenever we teach, we in fact, we want our students to solve problems that we have Not yet solved. Right?

We want the Next generation to be able to Solve problems this generation have not solved. Unfortunately I think a lot of education is completely focused on replication. The professor delivers information to the students. The students regurgitate that same information back on the exam. You ask a question that deviates a little bit in context or otherwise from what has been talked and students say “professor we have Never seen a problem of these type” “we never  done anything like this”.

But Think about it you want to create or maybe Stimulate people to be able to later  solve Problems they  have never seen before not only problems they have seen because a problem you have seen before is no longer problem, by definition, right .

SLIDE Education

Education is not just information transfer

Anyway so I think the important part is that you know, Students, as you said,  need to … I don’t remember the word … internalize, assimilate, build the mental model, you  need to be able to distill out of the information Something that you can apply in a different context this first becomes clear when I gave this Force Concept Inventory.Here are the data that I promised you before. In 1990 I taught a completely Traditional lecture based (course) test. The FCI has  twenty nine questions so the maximum score is 29 over 29, here is the count. So if you realize that many questions have three or four choices. If you put a monkey just Randomly pressing buttons on the screen you get a score of about ten. So you see Some Harvard students do barely better than a gorilla. (laughs) (Hestenes)I wanted my students to be in the last three bins that what I  expected  but this is the Beginning this is not the posttest after  I taught  I expected to be in the last three bins.

When I took the test I got twenty eight of twenty nine because I misreading a question I do a real mistake.

Hestenes said that Anybody scores below twenty three is still partially near Aristotelian  thinker connecting force and velocity rather than force and acceleration if you are a  near Aristotelian thinker Is really, really very hard to make any sense out of anything else in physics as you can see at Harvard  in spite of them having a five in advanced placement course some two thirds or maybe three quarters of the students scored below this cut off for Newtonian thinking. Well that occurs  before the gbeggining of the course so any responsibility for this data let’s see what happens after one semester of , you know high librated teaching by yours truly here.

SLIDE Education

Education is not just information transfer post test

Ready Here we go,  At least there was a gain (laughs) but is not as much as you see we still have you know gorilla level students still more than half below 23. Thaty Spots  a bit difference we can see The students have moved upright the distribution  on top

SLIDE EDUCATION overlay histogram

differently we can Compare with other universities  we put the initial class average scores we take the blue distribution and take the average  which is about seventy percent so right here on the results scale  and we put  the gain  the differences between the average of the red distribution and the average of the blue distribution on the vertical scale. Well They went from  seventy to seventy eight percent  so the gain is only eight percent, there is the datapoint.

SLIDE Education known vs gain

 

Where we Wanted the data point to be? as close as possible as perfect final score perfect posttest score So if you started  seventy you can gain thirty if you started twenty you can gain eighty and so forth. So This is the Harvard result the first one I obtained lets Compare with other schools from two year community colleges which are very different types of institution and large state universities and so on Here some data that were collected by a professor of Indiana university. Thousands of students here you seen Two year colleges some large state universities

Education Hake 1998

 

Notice at the Community colleges actually gain more than a Harvard students nearly twenty  percent, right versus eight percent for Harvard because they  Started much lower start at what  twenty eight per cent.

Harvard students began But Notice there is a trend here this were  all traditional taught courses. in all cases the data point a quarter of the way to the maximum score the gain about twenty when they could gain close to eighty,  they gained eight when they could gain close to thirty. It’s a quarter. Regardless of the setting, the traditional lecture course gets a gain of a quarter of the maximum possible normalized gain on this FCI.

SLIDE Education

Not transfer but assimilation of information is the key

Well so is not just the Transfer because this happen in all of this courses but the assimilation there off.

What did it happen for us ?

I often ask myself the question Where did I learn?

Where learning happen?

Did it Happen in the classroom  listening to the instructor and taking notes?,  think about for yourself  were the learning happen? I see many people shake, no, no.

Public Outside with the struggle with the material over the notes.

We were over the notes We become professors in the discipline  premedical students but can we Expect the same from our students especially let say premedical students expecting a career in medicine no probably not because they surely did not  put the same  efforts in to other disciplines  than we did in our own.

Now When I saw the results I thought for a moment “well maybe Eric  You are not a good teacher after all” but that  could obviously not be true, (laughs) right.  So I discarged right by the way.

I was trying have to explain how this poor results in the course I thought Maybe they are Dumb students but you know  Harvard selection. That can’t be true.

I thought a little bit my twisted mind came up looking for a perfect excuse put yourself in my position. Try to explain why  the gain was  so low. What You have concluded? can we hear Some suggestions?

They weren’t studying.   They were good students right.

The test was wrong. Of course there have to be something wrong with the test think about it

Heavy Truck  and light icar than on the light car intuitively  effect of the impact  the students were Confusing damage or putting in the hole idea of inertia maybe It was just a matter of Semantics after all if I ask them to solve  a problem involving Newton’s third laws  they applied fine.  So I thought Maybe matter of semantics. So I decided to do some testing on my own.

SLIDE Conventional problems misleading

And was then When I discovered another really important point namely that “The conventional way that we assess our students is very misleading”. Let me give you an expressive example and it will be the last physics I will show to you. And again is not important if you don´t understand the underlying physics. What I did was I decided first stay away from mechanics and Newtonic because we all have intuitive notions of Force, acceleration and so on right , but circuits, electrical circuits we have very little intuitive notions about .

What I did I decided first of all to take two problems One traditional Straight of the text book and another one which   of verbal quality. I show both to you.

This was the problem straight of the text book : three resistors these things who look lines and two batteries and students have to calculate the current in the resistor two batteries and the potential differences between the points P and Q.  If we asked to physicists, mechanical or electrical Engineers to solve this problem you know take ten minutes maybe Two thirds of the page equations but I am sure that if after ten minutes we compare answers  we find a range of answers because is very easy to make a mistake there is ample opportunity to make mistakes. I am sure many of ourselves  Including myself wrong.

 Well This was problem number five out the five in the midterm examination and in the same examination I put this question: Three identical Light bulbs A B C  are connected in series one after the other to a battery much like the lights in a Christmas tree and here There is the  switch S that’s shots out the last light bulb. Don’t do this at home tonight is dangerous OK

Bad ideas to you the question the students will ask short all the currents

No resistance or nearly cero resistance so nothing will go see anymore  all goes throw that short When the Switch is closed  

The students have to Answer the question When it is switched close What happen to the intensity in A and B? what happens to the intensity In C ?What happens to the current at the battery?  what is the potential difference across A B and C?  what happens to the total power dissipated?

The only thing they have to say is increases, decreases or  is the same.

No calculations I have  been told they have to give just justification. I did insist on justification, In fact is a bad idea because is actually the reason many words in the answer.

But Anyway if you understand these circuits Thirty seconds to answer the question of those twenty five seconds are spent on part E if you Answer the question, I mean you understand the basics.

In fact    I m tell you at the beginning of this talk the larger courses in Harvard  are taught by two faculty members turning weeks  to free the other  faculty  from interacting with the students and  that mean that in order to put my examination I have to convince my colleague that this was a good examination prompt . So I went to his office I show the draft of prompt he looked at me and said “Eric you have not in  your mind”

 I said “I don’t think so”

Semester FCI I was Told you So I told him I said I really annoying, you know : “I felt It was my prerogative to put this on the exam because  I have been  Teaching these circuits”.

He listen patiently and said “But Eric we have only five problems in the exam, we cannot give away twenty percent”(laughs) I said well I hope you are right but I am not sure”

Arguing for Half an hour and finally reluctantly agree to put on the exam. I we made Problem one call warm problem turns out Students Overheated on this problem. (laughs)

Prof Mazur said one student “problem one is the hardest problem of the exam. I don’t know how to Get started” What did you mean get started? if you get started you done, right? (laughs)

Anyway I said to the rest of the teachers let me grade problem number one why? Because They only has to write increase decrease stays the same one hundred students in the class I thought I could do it in two hours. quickly go straight Big mistake, big mistake. Some students, freak out and Written six pages in the examination booklets writing down everything they know about these circuits in hopes you know some of the right  thing.

To the physicist here some students   

My colleague said these Students Are C types (laughs)

SLIDE Califications conceptual vs. Conventional problem

I have to read all to see it was correct or no. So It took me a hole weekend to grade it.  By the time I finished the grading it was clear We have a problem on the left you see the conventional problem the textbook problem and the average of something seven out of ten and  the conceptual problem the light bulb problem the average was five out of ten.

SLIDE are the basic principles understood?

You see that huge peak to here just to amused the electrical engineers and physicists I tell You see comes from.

Most of the students in my class assumed or half of them at least when you closes the switch as here Nothing happens to the currents through A and B after  all it does not all the switch is closed the same current here What happens at the junction right Before the switch is open can only go straight when the switch is closed can go straight or take at right turn. (laughs)

 So the big question is how much is gonna to right turn to go straight? and how much take fifty percent of the Harvard students concluded?: Fifty-fifty, (laughs) already told you if you make it short fifty-fifty OK how would they the answer the question?

  Intensity of A or B OK How they will ask these questions nothing changes. Wrong because two light connected more current less resistance more current less resistance , so they loss two points there .

What about part C half current they wrote down decrease.  You know what? That’s they got the two points (laughs) Cause but I’d to be fair (laughs)

These are some of the nations Brightest primates some of they are going bypass surgery, Pretty scary. (laughs)

Any way This are the same students and the same days so let’s see the correlation between the performance on one type of the problem and the other

SLIDE Correlation

So Lets plot for each student the performance on the  conceptual problem versus the traditional problem.

To see the same thing well I am afraid I have to disappoint you there is no much correlation.

data points all  but if Line stretch the definition of the line a a little bit (laughs) you find that about fifty percent does equally poorly or equally  well in both types.

Look around that line or  band or wherever you call it .Forty percent lays below it, which  is not surprising concluded they do quite well in the traditional problem but not as well on the conceptual problem. But now  look at here is almost empty. No data points what does mean? It means that a student that understand basic concepts is likely do well the traditional problem  as well but  there  traditional one but do not understand the concepts. Who are those forty percent? What are the students. Why are they there.   How you will Classify the students?

Rote learners we have a term for they in the US, I don’t know if it’s the same plug and chug  see the problem they classify it and then just follows a procedure.

Lets  students  students in my class of premed

You don’t want physics  but you have to. You know is a high stakes course because winning a good grade in physics may mind you get in a more competitive medical school it may be the difference between getting in John Hopkings Medical school and you know the U of Nebraska medical school .

So in an exam you are nervous nothing but bad things Professor Mazur is Distributing the exam  you probably remember this days. You were Taking the exam you seat in one seat apart you still looking to your notes quickly sharpening your pens teaching staff isheading

Pens off

Printed side down one point, professor said: “put your notes away and you may begin”.  Silent

A hundred and fifty exams been turned around now  imagine you are one of the students and you look at the exam as you turn it around and you see this diagram, this one.

Conventional problem

 What is the first thought that crosses your mind? I want an answer from the electronic engineers or physicists.  What is the first though that crossed your mind?

 Kirchhoff’s laws are the laws, for those who does not know, that describe how currents distribute themselves.  why because  you are preprogrammed  if you see a diagram lines  Kirchhoff’s law problem read the test by the time a second  is the exam landed on his back  You already know  the problem number five is a  Kirchhoff’s law problem. How much physics is left? once you have made the Identification physicists for engineers, none is just a matter   taking the numbers conveniently presented in the diagram and  put in the right spot in Kirchhoff’s law  and then turning the algebra crunk.

Is Simple algebra  high school algebra clearly  most of the students in my class are pretty good at that. Do they understand  Kirchhoff’s law? For the reason I show, No. Because if they understood Kirchhoff’s law there were no problem with this questions so these students are likely to Became frustrated because they just Apply things by rote.

A Is not very intelectually stimulating  B if you don’t understand why you are applying  this procedures It is not very  interesting And plus you are bound to run in to problems  If you don’t understand you can run into problem don’t be able to apply it in a different context than applied. So The remarks I told you in the beginning: “physics is boring” not understanding physics and be frustrated Forty percent in my class is bound to have that experience.

SLIDE So what should we do?

So That s when I Finally ask Myself the question that I said in the beginning In my first confession that I should ask myself What should I do? The first think come up to me mind well was maybe I should Stop teaching graduate courses. (laughs)

To Give up

The solutions presented itself like most good things  in my life including black silicon in my lab they Happen by accident.

I told you I was afraid students will be offended by the simplicity of the FCI they weren’t offended at all, They were horrified by how poorly they done. They knew they Coming up. They asked me to organize a special session where we go over Every single question on the FCI .

So I book one of the classrooms in the center  Harvard room seven to ten we Spent three hours going to every single question. When I got to the question of the Heavy Truck and the light car I decided

OK  I gonna make a Diagram all the forces on the car there on the truck there

the force of the gravity the force of the road up I then do the force of the truck of the car on the truck and of the truck on the car and I said “by Newton  third law these two forces are equal in magnitude”.    

Whatelse you say. Turn around and I could See that There was a problem. I can see on the expressions of the students. So I thought does anybody have any question ?

They were so confused they couldn’t  articulate the question. You how it is articulate your own confusion you were that stage   this is the problem. So Better do this again so  And this time I rode On Acceleration, look even do  “The forces are equal in magnitude.  The truck is more heavier therefore the forces  gonna less effect in the truck is going to accelerate or decelerate less than the car. I thought it was a brilliant way of addressing the misconceptions I turn around only to see they are even look more confused.”

I was out of ammunition. They still could not Articulate any question. So But I know one thing I know that about half of them have answered  correctly on the FCI   So I did something I have never done in my class. I said Why you don’t discuss about this among yourselves?

The classroom as full as here maybe of people It exploded in total chaos. They forgot about me. I was something I never seen in a classroom. Always you know Lecture and , Stop and say. Does Anybody a question? Anybody? Students look down don’t make eye contact (laughs) person reluctantly no interation at all

Surprisingly  In two minutes out  they talk two minutes and they sorted it out . I spent five minutes unsuccessfully and they talk for two minutes and they figured out . But imagine two students sitting next John and Mary. Mary has the right answer because she understands it John has not. On average Mary is more likely to convince John  than the other way round  but and I think this is really the crux of the matter Mary is more like to convince John than  professor Mazur in front of the class there because she has only recently learned and still has the feeling for what the conceptual difficulties are that the beginning learner has . Professor Mazur learned at long time ago to him is So obvious that is becomes difficult to understand what problems the beginning learning can have .

You think the more you advance in your career the more qualified you become as a teacher. The opposite argument because the less qualified you become the harder becomes to understand the problems of beginning learners.

I don’t know  here In Harvard often the Graduate students who help teaching win the  biggest prices from the students and I think  is one of the reasons .

SLIDE GIVE THE students more responsibility tfor gathering information…

So I thought there is a way something We can made to Exploit this interactions I decided if we think education is a two steps process right. One transfer information, two assimilation of the information and  We ask themselves which of two parts is the hardest   we all agree Is the second part. Isn’t Ironic that in the traditional approach to teaching We put All of the effort on the easy step leaving the hard part to the students. should we do it especially in the information age should we do it in the other way around, leave the information gathering to the students: they can  watch the video, they can read the book, they can do so many things .

Take a lecture. What will be lost   by just filming the lecture?

Putin other way  Look at it. what will be lost?. You say The interaction is lost

I Argue is there hardly any interaction on line  you could have students Annotating  asking  questions and then put more questions that you would in  a normal classroom .

student look whenever is convenient to them  whenever they have the right mindset

education focus that that. I think you loose very little but Why get students responsibility for gathering information well

 we are  together

SLIDE Peer instruction

And a way do that is by assigning preclass reading. If you want  And in class I Focus in doubts not on coverage.

Is the reading what determines the coverage  not what I do in the classroom and I do that this by questioning rather by telling. Nothing new, Socrates over two thousand years ago teach by questioning not by telling. Here we are in the twenty first century mostly telling . So What is a concept test? I come in and Talk a few minutes and I project the question. Give the students  Some time to Think about it then They answer individually And then I tell them turn to neighbor see and find if has a different answer if not find another neighbiour andTry to convince that person  of your answer.

Then this chaotic discussion they answer again  Then  An explanation and I Repeat the process over and over. The question is to easy I put harder question  the question is  too hard I put one easier  question. If the topic of The concept is  understood  I switch to questions on a new topic .

So Basically when I walk in to the classroom I have Ten or twelve questions and I use  between six and eight.

Let me see you a video to see how this works in practice

Video

Magnetic field in the direction indicated bythese arrows. So the question is  what are  the 

Magnetic forces on the different sides of this loop five minutes take a minute to think about it and then your answers.

Don’t talk to each other they Committed an answer individually. Disagreement clearly here.

Turn to your neighbor and see if you can concivince one another of the Correct choice

(disccussion)

A hard moment

(second answer)

I know we have an absolute overwhelming majority for choice number two.

So What is happening here one Learning is active not passive, you cannot sleep throw class. Some students complain about that (laughs) every five minutes your neighbor start talking to you.

The other think there is Continuous feedback from the students to the instructor I find out then and where students don’t understand the concept when they talk to one another I don´t stand there I go and listen explanations   normally I find out  understand explanations

The other thing is that  is not the instructor the exam  has learned

The other thing is that is not only feedback   from the students to the instructor

They now were they stand in respect to the class They got continuous formative assessment of their own knowledge.

SLIDE Is it Any good?

Ok so Is it any good few minutes results Ok so  I gonna show you some results you know typical results

SLIDE First year of implementing PI

The word typical results is a cold word it means “the best data I have obtained in my entire career” (laughs)

Very Atypical results The first year I was implementing peer instruction

There is the pretest a very proud of these data is the most reproducible data ever obtained in my career amazing  a few percent variation

Harvard reproducible

So What about the posttests?

SLIDE Post tests

Here are the posttests. And you know still not the last three bins highest but It is significantly better than we have before we have some students below the  23 cut off but its significantly better than We show you at the beginning of my talk.

Did I teached to the test?. I was so excited with my results in 1991 American society of Physics Annual meeting in Vancouver. I submitted and abstract nobody  know me in education so all I got was a contributor talk eight minutes I spent four of my precious eight minutes documenting that I have not taught to the test . I want to be sure nobody accused to me say well you do this questions in class I used  none of the questions of the FCI or questions related to it to do  those question  in class or in the exams or in any assignment.

Anyway at the End of my talk someone raises the hand he said ”don’t worry about teaching to the test . we tried it, It doesn’t work.” (laughs)

So now we transferred a significantly larger fraction of population to higher grades.

SLIDE GAIN PLOT

 And if we look at that the another type of plot that I show you in the beginning instead a eight percent gain we now  double the gain sixteen percent. And in fact in the years

By simply having A better Selection of cuestions and The highest data point Was obtained when  I was on sabbatical has nothing to do with personality or anything else.

Other courses Implemented Interactive teaching also the best data here not the best data here but All this data together you see that  Simply implementing interactive teaching doubles the gain of the FCI.

SLIDE What About Problem solving?

Now the FCI only touches a very small part of the total  curriculum in a physics course is just you now Newton laws there is much more to physics plus you still to be able to solve problems after all some of they having gone medical some of they will Becoming Engineers still have to work out problems  right if a Engineer design a bridge a Beautiful brigde Conceptually  fine, but the first truck that goes over makes the  Bridge collapsed because you made and error factor one hundred in the load you can carry you are in trouble right is no partial credit for that (laughs) so I knew My colleagues gone and ask me “What about problem solving? How should I answer the question I thought about it one way it can be really conclusively answered that and that is  by repeating an exam.

I have Never ever repeated a single question in a exam. In Harvard we keeps thee years  of exams are found in a file the library

Exams students can go and look for them

Seven years of my exams very quickly concluded professor Mazur´s  never repeats an exam question

Meticulous data  I have entered individual question score in a data base im a data base jonkey when I did I have  Eight years of data to reanalyze. So I went back to 1985 I was able to Produce this histogram individual cuestions 

SLIDE 1985

I thought now Seven years later or six years later we can actually repeat this exam

The average was 60 now That year 1991 I prepared myself for actually performance  that was a little  worst than this  I have never ever done that year a single problem on the backboard. I decided that year that a student you derive very little benefit from seeing the back of the professor and watching a professor  solve problems In order to learning problem solving you have to do it. I mean You don’t learn to play the piano by popping CDs operas You gonna playing piano. You don’t train for marathon by seeing  in front of the TV eating popcorn watching DVDs on American runners, you have to do the running. The same is true for problem solving. You have to Struggle with the problem in order to learn problem solving.

The only thing that happens when a professor Solves problems on the board Students copy the solution in their note books and maybe look at it  later.

No problem solving and I said to myself you know as long as the problem solving ability do not  suffer too much I am happy because I know they understand the basics better.

SLIDE OVERLAY hystograms

You know what? that they actually doing better even no problem solbving in class  still problem solving   out of class but not in class.

The gain is not as big as the gain in conceptual learning but is statistically significant to a p value of  0,01. So you know is very solid is a more than a 10 percent in the gain in the performance.

What This tell us?

Slide Summary

So better understanding leads to better problem solving

(but “good”problem solving doesn’t necessarily indicates understanding!)

This tell us that Better understanding leads to better problem solving

Makes Sense actually right  but now Take the inverse of the statement  Good problem solving doesn’t necessarily indicate  understanding.

The Kirchoff’s law problem that I show you .

Student performance on exam problems doing By doing rote memorization of the procedure and applying the procedures. So I have been fooling myself for many years into believe I was a good teacher based on the student performance on exam problems because they were learning by rote. Let me show a very short clip what students have to say

Videos students

I feel the lectures very good  idea teaching each other .

You learn the material better when you have to convince  the person next to you of your answer.

Better When you have to teach it different ways  to explain the same thing

grasp the material better you have to explain it at someone else.

The nice thing is Committed to memory and still is  in reading text books  

The diference

Really help to me learn the material better

Didn’t have to go crazy before for the exam and you learn every thing

Is more relaxed studying

SLIDE SUMMARY

Traditional indicators of success misleading

Education is no longer about information

Ok, So in conclusion I think The traditional indicators of successful teaching end of a semester evaluation are limited assessment  is misleading we have a house of

Is very easy my collapse

 

Last thing I think Education is no longer about information

We live in a information age you know I think a hundred years ago We have to memorize a lot of things because  we to run to the library everything you

But nowadays if a not to know is No problem  because  I goggle And second later

I have More digits ever remember

I can still do my work. So one of the things I done in my class Is and I challenge you do to consider in your own classes.  Is to make all assessment open book. My students can bring whatever they want  to an exam, well they can not bring  another living person (laughs) dead person is OK. But you know they can bring Lab tops, they can bring cell phones thay can bring every they want.

Let In a very pragmatic way Later on in their career they have access to every information they want So Why are we examining them in an  environment that is completely artificial to that they have to operate?

Is not about memorizing information is about knowing how to use it and you know there are two things that happen when you take this steps. One is you send a strong messages to the students but Second The impact is bigger in you as a teacher when I reevaluate my exams I notice that a lot of the  problems I used to ask were simply Regurgitation from the book I cannot  ask these questions anymore because they has access to the book so they can goggle the answer

I have to actually  think harder on  Meaningful assessment in the information age and I think this  has big impact on the way I assess my students.

 So if you are interested.  I forget I about the top there

Remove this slide National For funding

You have A copy of the presentation  of my talk  

You can google Put my last name M- A- Z- U- R

Lucky bottom and then you have a copy of my talk.

Thank you very much for your attention. (Applauses)


Ċ
Alfredo Prieto Martín,
Mar 29, 2012, 6:38 AM
Ċ
Alfredo Prieto Martín,
Mar 29, 2012, 6:38 AM
Ċ
Alfredo Prieto Martín,
Mar 29, 2012, 2:14 AM
Ċ
Alfredo Prieto Martín,
Mar 29, 2012, 2:14 AM
Ċ
Alfredo Prieto Martín,
Mar 29, 2012, 2:14 AM
Comments