08.   OSCE in testing Communication and attitudes


If you can dream it, you can do it.
- Walt Disney

Contents:

Homepage - OSCE and OSPE

Compiled Responses

 The Beginning

 01.   Introduction and Basics of OSPE/OSCE

 02.   Is OSPE glorified spotting.. 

  03.   How to set up OSCE and OSPE stations..

 04.   How to collect the answer-sheets in OSCE/OSPE from students.

 05.   OSCE reliability

 06.   Levels of competencies in evaluation

 07.   Practical experiences of OSPE/OSCE

 08.   OSCE in testing Communication and attitudes

  09   Should knowledge be accessed in OSCE/OSPE

 10.   Simulated patients

 11. Examiner training...a must to do...

12.   Standard setting in OSCE

 13.   Disadvantages of OSCE

 

OSCE in testing Communication and attitudes

Anshu: Can you share with us examples from your own subjects where OSCE/OSPE
can be used to test :

1. Communication skills
2. Attitudes

Praveen: Although its a very known fact that it is very difficult to put up OSPE's
in Anatomy as in Anatomy we usually do not test the practical skills but we test the end of the process.
Still to try,

1) Communication skills : Examination of muscles of facial expression
2) Attitudes : Examining movement of eyeball of a patient with complaints
of diplopia,
3) Both Attitude & Communication skill : Although not entirely from anatomy but still; Explaining a couple with primary infertility, the anatomical basis of infertility in male & female

Anshu : Testing for communication skills is not impossible in Anatomy. Like if you ask a student to demonstrate Trendelenberg's sign or perform surface marking, you can see if he explains the procedure to the subject he is performing it on. Attitude and communication skills can be tested if you give him a written case scenario- e.g. asking him to convince someone for donating body to anatomy dept after death. Similarly physiologists can test communication when students are asked to demonstrate cranial nerves.
Do take a look at the examples (ppt) given in the link below:
http://...../practice_OSCE_for_finals.html

Anshu: One more. Does explaining to the parents of a child with a confusing karyotype fall into the realm of genetics/ anatomy or is it a paediatrician’s job?

Praveen: Thanks for very good link, it will be helpful in forming OSPE stations.
Actually what I gave you was a general opinion of Anatomist although I had added a few examples for the skills which you had asked for one of which was cranial nerve examination (VII th cranial nerve). I want your opinion for those ones also.

Regarding giving advice for Karyotyping it will mostly go with either the geneticist/paediatrician or if an anatomist well trained in cytogenetic lab is in charge of the cytogen-lab. Others are very nice including surface marking which I forgot but is Anatomy an ideal area for communication skill through OSPE.

One more point some of these are more of P.G. level OSPE’s otherwise per se mostly anatomy is tested for the factual recalls at U.G. level. We don't examine dissection skills, we examine the identification and recall of the areas dissected, same is for histology, we don't ask them to do slide preparation & staining but it is identification & viva on the slide.
Definitely we can put these at places where PBL & vertical integration is going where these anatomical skills will be placed alongside surgical clinical cases or correlation.
More of the communication skill & Attitude can be tested once the student is in 2nd year heading at least towards the clinic where he observes some form of communication. As also Sir had commented during the discussion on google group link of  MEU there should be no segregation of 1st, 2nd & 3rd year regarding anatomy or for that matter any subject..

Barathi: The first MBBS students learning anatomy are the prospective anatomist/geneticists/paediatrician of tomorrow therefore I think it can be taught to them This can be kept as a station to assess communication skills.

Bharti U: For Biochemistry OSPE can be used .We give a hypothetical exercise to the first year student after finishing topics like Regulation of blood glucose, Diabetes, Acid-Base Balance, Renal Function tests etc.
The students are given questions like to give the normal reference range of some blood/urine parameters, the reasons for the biochemical changes.
The communication skills can be tested here by making the scenario as to how they inform for collection of 24hrs urine sample, or simply how is information of overnight fasting given.
The practical aptitude is tested by observation as to how are they performing the practical

Anupama: Communication skills can be tested from internship level and beyond i.e. post graduation.
Till then, it should not be a part of OSPE.

Suman Singh : I think none of the field can be devoid of communication and attitudes as they are the most basic needs of existence. So definitely we can have wide verity of stations that can incorporate testing these two important domains in OSCE station in each of the subjects, which we are overlooking at present.

Talking about my subject i.e. microbiology we can test the students communication skill by providing them a subject and then tell them to get a particular sample e.g. urine collected for culture and see how he explains the things to the patient.
Counselling a patient for HIV testing or handling over a HIV positive report to a patient can very well judge both communication as well as attitude.
Like this we can have station targeted to objectively assess many components.

Bharti U: We may not need actual patient this can be done by fellow students taking turns as we used to do in Physiology for cranial nerve testing or in anatomy surface marking revision classes in the hostel rooms

Dr Tejinder Singh: However, remember that communication is not a generic skill- it happens in a context. So be aware of the context in which you are testing for communication. Proficiency in one setting may not translate into other. Moral- test it multiple times in different contexts. Also, avoid using your communication skill as the gold standard for assessing student's performance.

Barathi: Yes. We keep our skills as a control and compare the others. The risk categories are those who are better than us and those who are poor communicators than us.
But can we set guidelines as to describe our expected outcome from the task put forth on assessing the skills of communication.

Suman Singh P:That truly is remarkable. If my communication skill is poor I would find it difficult to evaluate the skill in students and if I am good I would expect the students to communicate the way I do but that can be overcome to a great extent by the thoughtfully prepared checklist of OSCE stations.
 Suppose I want to test the ability of a student to explain the test results of a widal test, will the following component suffice to objectively assess the communication skill-

 1. He greets the patient
2. He introduces himself
3 Explains the test result
4. Listens and answers to the patients queries with attention
5 Tells the patient to get in touch whenever required.

 Suppose a student does all the above steps but what will form the gold standard of the way he does all steps.
Etiquettes, mannerism, attitude all of them form a part of communication skill.
Under your directions we would like to be benefited by the guidelines that must be followed to judge these generic skills in a particular context

Dr Tejinder Singh: The question of reliability relates to the purpose of assessment. If you are using it for certifying examinations, then you need higher reliability. For formative only, one can work with a lower figure. The current emphasis on integrated assessment means that you should be able to test multiple competencies at each station. In other words, you do not have to have one station for history and one for examination. The same station can be used for history and examination of 2 different problems. Build communication in each station. This will solve your problem.

Suman Singh : Does this mean that we should have multiple stations trying to assess one or more objective.


Dr Tejinder Singh: Therein lies the importance of having multiples observations by multiple examiners (all need not be teachers- peers, colleagues, nurses, patients are some of the sources). The checklist that you have given should be agreed upon by the departmental group. There may be some bias during marking but multiple examiners and observations will tend to neutralize that part.

Navneet: I understand your statement that one should not use one's own communication skills as gold standard for assessing student performance but I really do not see how one can avoid doing that!
A checklist cannot cover aspects such as attitudes, mannerisms, language which have a direct bearing on communication and influence one's assessment

Dr Tejinder Singh: Pre-examination discussions amongst people who designed check list to define the acceptable behavior will help in overcoming the bias. Moreover, as I said yesterday, multiple observations by different examiners will help to improve reliability.

Chandrika: Nice discussion on communications. It is extremely important in each subject, be it anatomy or paediatrics, especially so in formation years or under graduation. We cannot teach medicine and then say `Talk medicine now` Even in anatomy the student and faculty need to communicate with their juniors and other students.
An interesting point was raised by Anshu in regards to communication in a child with abnormal karyotyping. The practice right now is that either the paediatrician who has referred the case for a karyotyping or a trained genetic counsellor (this could be anyone trained in that, including MSc genetics) counsels. Usually labs do not counsel patients of their reports.
Communication by students on HIV, family planning etc should be evaluated by a `standardised patient`. SP`s could be a PG or a nurse or a non medical person who has been trained for few hours on how to evaluate the communication. Communication could be observed on the side by any faculty with a checklist. Here the checklist is important and our personal skills or shortcomings should not evaluate the candidate. Certain validated checklists are available online, like ABIM score sheet etc. which may help in forming our OSCE checklist.

Suman Singh: At all the OSCE stations where patients, either real or standardized, are involved skills like communication, attitude, and empathy must be given weightage. This can be graded on a scale of five by the observer or can we take the opinion of the patient and see where he wants to put the student in terms of these skills?

Dr Tejinder Singh: We use the term attitudes very loosely- by definition, attitudes refer to our belief system. We know something- we believe it to be true- and we act on this. This middle component represents attitudes. We cannot observe attitudes. There are ways to measure them using various types of scales (Likert, Thurstone, Osgood etc). I think what you are referring to is professionalism, ethics and inter-personal skills. As said in my previous mail, we can have checklists to assess them but faking can be a very common problem. The best way to assess them is in an authentic environment. They are therefore best assessed on an ongoing basis. To counter subjectivity, remember the old formula- many observations by many people. While teachers can rate the students, use of self ratings, peer assessment, nurses' assessment or even from patients (360 feedback) can be a useful means to know about a student. The situations that you mentioned are best encountered in a natural setting. Mini CEX is another useful tool to assess some of these.

Praveen: You are very right Sir, it’s not easy to measure the attitude and for it various scales like likert scale etc are required and feedback from patient, nurses & peer group is very important.
I am herewith forwarding an abstract validating the statement.

Surveying Students' Attitudes During the OSCE
Authors
: Allen, Ruth1; Heard, Jeanne; Savidge, Mildred; Bittergle, Joseph; Cantrell, Mary; Huffmaster, Tim Source: Advances in Health Sciences Education, Volume 3, Number 3, November 1998 , pp. 197-206(10)

Publisher: Springer    Abstract:

This study assessed students' attitudes, students' level of confidence and the importance of faculty feedback during an Objective Structured Clinical Examination (OSCE). A survey was administered as one of 15 OSCE stations to 138 sophomore medical students. Eight of the twelve survey items were Likert type items with a follow-up open-ended question. The remaining four items required only an open-ended response. The five surveyors participated in the design of the questionnaire and in training to insure reliable information acquisition and recording. The OSCE was given in five identical tracks in a single day. Each surveyor interviewed either 27 or 28 students during the four-hour period of the track. The Likert responses to the eight items were analyzed and statistical comparisons made for each question. The open-ended responses were analyzed using qualitative summary techniques and compared to the Likert responses.
Statistical analyses of the Likert based items showed students had high levels of confidence in their ability to conduct physical examinations and histories, found faculty feedback helpful and informative, but were nervous throughout the examination. The qualitative analysis provided additional information regarding students' feelings about the OSCE, their levels of confidence, their stress levels and the importance of faculty feedback during the examination.
Results of the survey suggest curricular changes which could be made in preparation for the OSCE and for a new Introduction to Clinical Medicine course currently in its first year in the College of Medicine.

Keywords: faculty feedback; level of confidence; Objective Structured Clinical Exam; student attitudes; student surveys

Document Type: Research article

DOI: 10.1023/A:1009796201104

Affiliations: 1: Email: allenruthm@exchange.uams.edu

Dr Tejinder Singh: The current thinking is to have 'integrated assessment' meaning that you should not split clinical competence into parts to assess them. What would be more appropriate would be to give some weightage to communication at each station, unless your specific objective was to teach communication. Seeing how he communicates with an actual patient in a real setting brings authenticity. Putting a volunteer brings artificiality. A standardized patient with proper training may be appropriate but I am not sure, how a volunteer would help.

Anshu :I agree with the integrated assessment part. I am not happy with the 'introduces himself to patient and wishes him - 2 marks' kind of system. So feel global ratings would be better.
And yes, I would like to see how a student handles an argumentative patient, a worried tense mother or how (s)he treats a shy patient with dignity. Overall, it matters how a student treats patients while talking to them, undressing and examining them. These points need to be emphasized even while knowledge, interpretation etc are tested.
Any idea how one looks for attitudes in an OSCE? Can't they be faked for exam sake? Give me some concrete examples so that I understand.

Dr Tejinder Singh: It is a good idea to follow it by some questions. For example, regarding the history you just took, write T/F
The patient is a smoker
This ensures that history has been correctly taken.

Barathi Subramaniam : May be for communication skills can we use somewhere in OSCE? Grading is what would be problem

Dr Tejinder Singh : I am not clear on the questions- are you saying that we can ask the candidate to give commentary while on a communication skill? Or may be reframe your question.

Barathi Subramaniam: It is not in context with commentary. You had mentioned that OSCE
should not turn to be a viva station. Therefore I wanted to clarify using OSCE for communication skill. May be we can keep a question and a trained volunteer, in one station where the candidate would explain a
procedure /treatment.

Dr Tejinder Singh: Giving a running commentary has other problems as well. Suppose there is a mismatch between what a student says and what he does, and then what are we going to grade? I feel that OSCE should not turn into a viva station- a very common temptation, especially for new faculty.

 

 

 

Discussion Summary

Introduction

Origin and evolution of OSCE

What is OSCE OSPE..

Important terminologies OSCE and OSPE

Process - OSCE and OSPE

Competencies

Advantages OSCE and OSPE

Limitation OSCE and OSPE

The final words