Feedback in Medical Education

Additional Info

• Audio File:uab/ua168.mp3
• Doctors:Mullins, Haddon;Willig, James;Zinski, Anne;Harada, Caroline
• Featured Speaker:Haddon Mullins, MD | James Willig, MD, MSPH | Anne Zinski, PhD | Caroline Harada, MD
• CME Series:Quality and Outcomes
• Post Test URL:https://cmecourses.som.uab.edu/mod/quiz/view.php?id=4365
• Guest Bio:Haddon Mullins, MD is a General Surgery Resident. 
  
  James H. Willig, MD, MSPH, is the Associate Dean of Clinical Education in the School of Medicine. He attended Medical School at the Instituto Tecnologico de Santo Domingo (INTEC) and completed his residency at the University of Virginia Roanoke-Salem. At UAB, Willig has earned an M.S. in Public Health and completed an Infectious Diseases Fellowship. 
  
  
  Learn more about James Willig, MD, MSPH 
  
  After completing her doctorate, Dr. Zinski spent six years in the UAB Division of Infectious Diseases investigating the impact of behavioral interventions on patient self-care behaviors, including screening and retention in care. 
  
  Learn more about Anne Zinski, PhD 
  
  Caroline Harada joined the School of Medicine faculty in 2008. She completed her undergraduate work in biology at Brown University in 1996, and, after a year of living in Bolivia teaching an after-school program in an orphanage, began her medical education at the Yale School of Medicine, graduating in 2001. 
  
  Learn more about Caroline Harada, MD 
  
  Release Date: November 9, 2020
  Expiration Date: November 9, 2023
  
  Disclosure Information:
  
  Planners:
  James Willig, MD, MSPH
  Professor, Infectious Diseases
  
  C. Haddon Mullins, IV, BS, BA
  UAB Medicine
  
  Jill Deaver, MA, MLIS
  UAB Medicine
  
  Adam Roderick, M.ED.
  UAB Medicine
  
  Anne Zinski, PhD
  UAB Medicine
  
  Caroline Harada, MD
  Associate Professor, Geriatric Medicine
  
  Ronan O’Beirne, EdD, MBA
  Director, UAB Continuing Medical Education
  
  Katelyn Hiden
  Physician Marketing Manager, UAB Health System
  
  The planners have no commercial affiliations to disclose.
  
  Presenter:
  James Willig, MD, MSPH
  Professor, Infectious Diseases
  
  Dr. Willig have no financial relationships related to the content of this activity to disclose.
  
  There is no commercial support for this activity.
• Transcription:UAB Med Cast is an ongoing medical education podcast. The UAB division of continuing education designates that each episode of this enduring material is worth a maximum of .25 AMA PRA category 1 credit. To collect credit please visit www.uabmedicine.org/medcast and complete the episode’s posttest.
  
  Melanie Cole (Host):  Welcome to UAB Med Cast. I’m Melanie Cole and today, we’re talking about feedback in medical education. In this panel, are Dr. James Willig. He’s the Assistant Dean of Clinical Education at UAB Medicine. Dr. Haddon Mullins, he’s a General Surgery Resident at UAB Medicine. Dr. Anne Zinski, she’s an Assistant Professor in the Department of Medical Education at UAB Medicine. And Dr. Caroline Harada, she’s the Assistant Dean for Community Engaged Scholarship at UAB Medicine.
  
  Dr. James Willig, MD, MSPH (Guest):  Welcome to the forth chapter of bedside teaching for the medical educator podcast series. Today, we will focus on the provision of feedback. The ins and outs of how to do this difficult task effectively. We have some special guests with us today. Our first guest is –
  
  Anne Zinski, PhD (Guest):  Anne Zinski from the Department of Medical Education.
  
  Caroline Harada, MD (Guest):  I’m Caroline Harada. I’m Assistant Dean for Community Engaged Scholarship and a Geriatrician.
  
  Dr. Willig:  I’m James Willig. I’m on Infectious Disease or pus control as some call it. And we are here with our host –
  
  Haddon Mullins, MD (Guest):  I’m Haddon Mullins, third year medical student here the University of Alabama School of Medicine.
  
  Dr. Willig:  Take us away Haddon.
  
  Dr. Mullins:  So, the first question is going to deal with how and why do you give feedback. So, is your feedback theory-based, is it practical-based? Do you have a set of tools or an overall structure that you think about when you are giving feedback and how does that inform the feedback that you are giving?
  
  Dr. Harada:  So, for me, I don’t think there can be learning without feedback. And that’s pretty well grounded in adult learning theory that you can’t learn unless there’s information being fed back into the system to correct and improve a process. And so, when I teach on the wards or in my clinic, I try to set expectations really early about that. That feedback is going to be a part of every day and everything we do when I have an interaction with my learners. So that the learners know that it’s coming, and I find that the more often we have feedback, the more often I give feedback, the less scary and intimidating it becomes for the learners as well as for me. Because sometimes giving feedback is kind of scary and intimidating for me frankly. And so, if everybody knows it’s coming, and knows it’s coming frequently, then I think it tends to take away some of the negative feelings that often accompany feedback.
  
  Dr. Willig:  For me, I like to sort of make it very clear from the very beginning that my job is to help you get better. And really say that from the get go and say that that’s going to include – we’re going to talk about every encounter as it happens. I really move to where I used to really focus on doing a thorough mid-month evaluation and an end of rotation evaluation. I’ve moved forward to doing more feedback right after each clinical encounter throughout the day. And that really, I think sets the expectations and follow through with I’m invested in you, I’m going to try to make you better every day and here’s something that I think you can get better on today. That creates a good culture to really expect that conversation.
  
  And then when I do a mid-month, I use a structure that I learned from a gentleman called Donald Steinweg where he would sort of sit the learner down and really say heh, everyone of us is a combination of skills, knowledge and attitude. He would then use the RIME model, the reporter, interpreter, manager and educator model and sort of map out where I as a learner was supposed to be on that model and then he would say okay so, you’re an intern so I expect your reporting to pristine. I expect your interpretation to be improving and your management is sort of your aspirational goal where you are focusing on. So, the skills that I expect from you, are – and then he would sort of talk about the skills that are expected with where I was on the RIME as the background.
  
  Then he’d talk about the knowledge that he had seen me sort of express and where he thought the gaps were and what I needed to improve. And then he talked about attitude and sort of really talked about how empathy and how I related to families and patients and the greater healthcare team and really give me very direct feedback that was really based on observations that he had seen throughout the process. He really used to jot everything down as we were on rounds and it would be very much like last Tuesday, you did this when this other event happened. And it was incredibly helpful to get sort of that detailed feedback. But that’s how the mid-month would be and how the end of month would be.
  
  The thing that I’ve added is that throughout the course, I don’t get to mid-month and give you feedback, we get feedback throughout the whole way.
  
  Dr. Harada:  Yeah and I think that’s so important. It’s almost like more of a coaching model that we take in medical education where we say today, just like a sports coach would say, hold your bat a little bit to the right when you go for that swing. I say to my student, when you are checking proprioception, next time I saw where you put your fingers on the toe, that wasn’t exactly right. Next time I want you to do it this way. And just those little teeny pieces of information that you are feeding back into their system, that’s feedback and that’s super low stress and can be done like 12 times a day if you are working with a student in your clinic.
  
  Dr. Willig:  Dr. Zinski, you have a lot of insight into pedagogy and I wonder how you approach this question.
  
  Dr. Zinski:  Sure so, as a [00:05:41] in education, I’m always going to start with theory Haddon. So, I’m thinking about how the theory of experiential learning and so I’m thinking about very basic structure for something like feedback and so I’m thinking about having an experience, doing and then being able to reflect on it, reviewing and those are always the doing and reviewing are going to be the very first part of it as a learner. I’m going to do it and I’m going to review it. And I’m going to review it hopefully with someone who can help me make conclusions about what’s going on, so with a teacher. So, I’m doing and reviewing, I’m making conclusions with someone who can help me that’s my preceptor and then I plan ahead if I’m thinking in a cycle for what I’m going to do and review next. And so, that had a lot to do with Caroline’s adult learning theory but specifically, it’s Kolb’s experiential learning theory and it has a lot to do with the way you think about learning and giving feedback for learning.
  
  Dr. Willig:  So, just applying that to like the rounds, so it would be the student did a presentation. They have a chance to as a team, we review that presentation and then we reach some conclusions in terms of heh, this was really good, this was really bad, that we might give that feedback after that presentation. And then the charge to the learner is to start thinking about tomorrow’s presentation and sort of start planning on how tomorrow’s will be different.
  
  Dr. Zinski:  Absolutely. Or how it would be the same because you reinforce things that were done well during the review. So, components to keep, usually that starts with something like I noticed and you talk about a way that they met a goal or the way that they achieved something that was successful and then the next statement traditionally is I noticed something else that the student did that might not relate to the goal in the way that you had intended and you have another learning moment where you can say the way that it’s intended as a reminder is goal x, y, z and then you review together until you get understanding and plan for the next day.
  
  Dr. Mullins:  So, do you see the – it seemed like the way you approach it, it seems to be very learner-focused, would you that’s correct, and that the teacher is more of a facilitator to promoting self-reflection and providing self-feedback?
  
  Dr. Zinski:  So, I think it is – I believe in therapeutic alliance and so if you read about feedback in medical education there’s also a theory about therapeutic alliance and agreeing to what’s going to happen next. I also believe that there is someone who is – has more training than you in the room and so, that authority provides them an opportunity not so much to give advice but to be able to facilitate learning for a learner. And so, it’s learner-focused but I do understand that there’s an expert in the room.
  
  Dr. Mullins:  That’s a lot of what I read too. I think a big part kind of like Dr. Willig was saying is as an experienced clinician, you know what standard is and that standard should be communicated but then as the expert, you are able to observe where the student has gaps and the structure of feedback should be focused on closing those gaps. As experienced or practicing clinicians, how often do you see students engage in self-feedback or do you all try and promote that or is it more of a coaching model that’s more practical to use just on the wards in general.
  
  Dr. Harada:  I’d say that it varies widely. Some students are very insightful or very interested in self-assessment and they tend to learn much faster. Because they are constantly asking did I do that right when I presented the medication list? Is that the way I should do it or not? Those students, they get their feedback because they are asking for it. Other students are probably more like I was when I was a medical student, they are more shy, they don’t want to trouble the Attending by bothering them by asking and they feel like it’s selfish or self-centered of them to do that and so they might not ask. Or maybe they just don’t care, and they feel like they are doing it right and so they don’t even question it. so, I guess I would say the answer to your question is the level of self-reflectiveness varies widely among medical students but sort of as the educator, I fell like it’s my role to encourage everybody to be self-reflective and to ask them those questions even if they don’t bring them up themselves.
  
  Dr. Willig:  I’d like to pull in two threads of some of the things that you mentioned. The first is Dr. Zinski, when you mentioned that you should give also, highlight the practices that were done particularly well so that they can be sort of encouraged to come up again on subsequent days. I probably am guilty of not doing that enough but when I see something done well, I don’t necessarily mention it. I kind of think oh they got this. But I should probably – that’s an opportunity for feedback right there that I’m leaving on the table. That I can definitely see, you know, you did an amazing job when you did this particular thing in this presentation. And make sure that the student knows that that’s underscored to carry forward.
  
  The other thing, one of the things that Dr. Harada here mentioned, the learners that have different personalities and how they will interact with the Attendings, that’s very interesting to me because you see – I see that all the time. I see some learners that are very quiet, and the problem is that sometimes I might interpret that student that’s being quiet and respectful from their perspective, I might interpret that as being uninterested or not engaged. And by kind of setting the expectation of we’re all going to give each other feedback here, I want to hear – I’m okay with hearing feedback from you, that’s great. I need to get better too. And I’m going to give you feedback every encounter that even if it’s just pointing out something that was positive, I think that creates a climate where the learner’s personality won’t really hurt the learner there.
  
  One thing I will say about your question on self-reflection Haddon, is that very specifically at the mid-point, I think I would ask or and at the endpoint, what are your thoughts on how you have performed. And then sort of get the self-reflection of the student so you understand where they are and sometimes the students are really insightful and they really point out the things that they can get better and sometimes the students are way off base which let’s me know that that feedback conversation is probably going to be more extensive than I had initially thought depending on where the learner is. But I would hesitate to use that every day after every patient encounter is kind of thing so tell me how you think you did. It’s kind of the wards. It’s fast moving. Maybe it’s no tell me how you think you did; it’s let’s get this done.
  
  Dr. Mullins:  What about some more specific components of feedback and not being from what I’ve read, and you can agree or disagree that most importantly based on direct observation, accurate and timely, specific, and actionable. And [00:12:39] even goes as specific as saying, formative feedback should include more nouns than verbs as opposed to like a summative evaluation that’s adjective and adverb based. Do you need a specific code as that or does it need to be more natural or is there some kind of code or specific checklist that you hit when you want to give feedback?
  
  Dr. Zinski:  So, for me, I structure all of the feedback messages that I give on what I call 4G model and so I think about the goals of what we are doing, or an objective and so that’s the first G is my goals. And then I give an observation that relates specifically to that goal. So, not I noticed that something happened that has nothing to do with my goals. I noticed that something happened and then a description of how that relates to my goals. So, I’m giving an observation that’s based on behavior and a goal. And then I tell them where they glow, that’s one G. And then I tell them room to grow, that’s the other G and it all ties back to whatever that observation, however the observation related to the goal. And so, by the time I’m done, I’ve reminded them of where we’re trying to get to. I’ve talked to them specifically about the I noticed performance. So, I can never be wrong which is kind of nice. And then I talked about a way to get better and a way that they are doing something that I’d like them to maintain moving forward.
  
  And remembering that in a coaching model, there’s a lot of very specific but also advising type feedback happening. In an apprentice model, which I believe is more appropriate for something like a clinical setting. They’re going to be doing a lot of watching and then you have a high stakes situations where there is going to be a lot of learning going on. And so, they are going to be doing a lot of reading and a lot of teaching all at once. And so thinking about the four G’s is an easy way so that you don’t have to think about learning and think about teaching and think about patient care and keep changing your hats. Just have that framework for everything.
  
  Dr. Harada:  Anne, you’re really good when you talk about the I noticed that statement. And I think that’s something that really is key to giving feedback. Can you talk more about how you do that?
  
  Dr. Zinski:  So, it starts with being a great observer which might be a whole different podcast one day. But being able to observe someone’s behavior which tells them two things. The first thing is, you’re watching them. Which is beautiful the same way that my GPS is watching me. They are not always telling me a whole lot, but I know they are monitoring me because they show me my tiny little icon on my map and so they’re watching what I’m doing and then they make sure that they notice when I’m making correct and incorrect turns and they let me know. And they speak up when I’m on the right track or not.
  
  And so the same way my GPS might say I noticed, without me I noticed that you made a wrong turn and here’s what we’re going to talk about next. I like to look at a student or as much as I can, or as long as I can and notice what their behavior is. And it establishes trust is number one. And it gives you something to talk about is number two and that you are watching things that they actually did, again, you’re never wrong because you’re not trying to guess their intention and you’re not judging why they did it. you’re telling them what they did which gives them a beautiful learning moment.
  
  Dr. Willig:  I think that part is as important. You’re being very intentional about how you phrase this. I noticed that you did this. There’s no judgement. There’s no interpretation. There’s just fact. You did this.
  
  Dr. Zinski:  Absolutely.
  
  Dr. Willig:  And then, that’s the first G, so that’s –
  
  Dr. Zinski:  So, that’s going to be my given observation.
  
  Dr. Willig:  Given observation. Okay. So I give an observation. And then the next step would be –
  
  Dr. Zinski:  Tie it to an objective. And so, when I start, yeah, go ahead –
  
  Dr. Willig:  So, here, like heh I noticed let’s say give an example of when you did your abdominal exam, you forgot to do – you didn’t palpate the liver. Or you didn’t palpate the liver appropriately. So, just kind of carry me through that.
  
  Dr. Zinski:  So, I noticed when you gave your abdominal exam just now, you didn’t palpate the liver. You give them a minute, not a whole minute, like count to five to think about what you just said. And then from there –
  
  Dr. Willig:  One of our objectives is to know an abdominal exam.
  
  Dr. Zinski:  Or a complete abdominal exam includes. And review with them so that they have a sense of what the goal was what it is.
  
  Dr. Willig:  Okay so first you give an observation, and then goal tied to a goal or whether the objectives of the rotation, the next things are glow, what were they, glowed and then –
  
  Dr. Zinski:  Room to grow.
  
  Dr. Willig:  And room to grow. Glow and grow. So, the – where would you integrate the glow there in that abdominal exam example?
  
  Dr. Zinski:  So, sometimes the glow is there if I said, I noticed that you included all components of the abdominal exam and review what they are. So, that’s important because a complete exam will allow you to – and so you told them that they did well without having to high five or even add a judgement at that point. And if you were going to change it into a grow, you might say, I noticed that you didn’t palpate the liver when you gave your abdominal exam. Remember, that the abdominal exam includes these components and so next time we’re going to make sure that those four are in that.
  
  Dr. Willig:  I see. I see. So, you end in either the glow or the grow. But it dichotomizes the end. So, that’s great. So, I can give good feedback like this was great and this is why, or this is what I observed, that’s important because one of our objectives in this course is to find this and this is what was great about it, please keep doing it. Or gosh, I noticed you did not do this thing, that’s an issue because it’s an important part of what we learned in this course is to do these things and here’s how you can get better. So it kind of goes – even though it’s four G’s I use three at a time.
  
  Dr. Zinski:  That’s right, yes. That works and I think especially too, you can use it if you notice something that the student didn’t notice and so, I can also use the words I noticed that there was a little mark on your patient’s face. Once you’ve observed that they didn’t see the mark on the patient’s face. So, you can also notice when they didn’t see something. Or when – so if something’s missing, or if they did something extra. I noticed you went ahead and did a neurologic exam on someone with knee pain. And then you’d start that whole conversation. So, you can notice a lot of different things and you can notice when something is missing. You can notice when something is extraneous. But you are noticing facts and that gives you a lot more credibility.
  
  Dr. Harada:  Because, and that’s really important I think because I’ve had that situation where I give feedback to a student and they argue with me about it. They disagree with me. And that’s my fault because I haven’t been as objective and behaviorally based as I should have been. So, that’s something that I’ve really learned more recently is like nobody can argue with me if I say this is what I saw in the room. So, I think that’s a really useful important trick.
  
  Dr. Mullins:  So, let’s talk about that too a little bit more and from what I’ve read, is that a lot of the problems with giving feedback or when you run into students that really need it are defensive about feedback or reject feedback. A lot of times, that comes from a lack of feedback in the past. And that if you don’t deliver feedback on the frontend, what happens is students will start to create their own paradigms of giving themselves feedback and they start to either self-validate or self-criticize in ways that in the future make them less responsive to beneficial feedback. How would you all respond to a student to whom that has already occurred to where they haven’t received beneficial feedback in the past and now, they are with you. You are trying to give them constructive feedback. They are resistant to it.
  
  Dr. Harada:  I’ve had that happen so many times where students would say, but for the last three years, I’ve always done the abdominal exam that way and nobody has ever told me it was wrong before. How can I be wrong after all of this time? And that’s really tough and it speaks to the fact that we’re not doing as good a job as we could as an institution of medical educators. And so, what I usually say in that situation is I’m really sorry that nobody has told you this before, but this is – let’s tie it, as Anne says, tie it back to the learning objectives. We want you to be able to do an excellent abdominal exam by the end of this rotation and part of that is these are the five steps, and these are the three steps that you didn’t do or that you did incorrectly. As long as I keep it really behaviorally based, based on what I observed and tied to those objectives for the rotation, I think it’s easier for students to accept that maybe they have been just doing something wrong because it’s so easy to adopt that no news is good news philosophy where if nobody told me I was wrong, then I must be right. And that’s not actually true.
  
  Dr. Zinski:  I also think there’s a focus on grading and summative assessments and less of a focus in some educational settings on the observation part. So, there’s a lot of really well intended people who give advice about what they would do all the time, but they forget to give an observation, or they forget to do the observation of student skills or a certain kind of behavior. And so, once that’s lost, you lose your message. Because your message is about what the student did for a learning moment. And so, they may not have had that student who didn’t get great feedback before, might not have had someone who was a great observer watching their behavior. They may have been looking for a grade that says you’re fine or you passed versus looking for observations all the time.
  
  Dr. Willig:  Let me try to conceptualize that a little bit. Say that I have this great story about this mistake that I made in the abdominal exam and something that I learned when I was in training. If I walk out of the room and I tell you this story, you are suggesting that the impact of that story would be much greater if I could sort of say heh, when I saw you do your abdominal exam, I noticed that you did this. It’s important that we do this abdominal exam because it’s one of the things – goals for this course. Here's how you can do better. And then the reason I know that is because and then I tell my anecdote. That would be the correct way to frame that learning anecdote within this context.
  
  Dr. Zinski:  Sure, so there’s always room depending on your relationship with your learner, to add a personal anecdote, to add a piece of advice, to add an experience that you may have had in the past. But giving them the pieces of their learning moment are going to be important because they’re novices. Don’t forget that they are novices. And so sometimes we have to be incredibly literal and intentional about the messages that you’re giving and noticing their behavior and then relating it to the objectives because they might not have done a lot of reflection and they might not know the objectives by heart like you do. And so, putting that together for them is incredibly important in early stages of experiential learning.
  
  Dr. Willig:  and I think frankly I’m sharing the anecdote because I think it’s important that I took something away from it and if you’re telling me that if you put it in this framework, it’s going to be even more impactful, that’s a positive.
  
  Dr. Mullins:  And so what about the evaluation because I feel like I’ve seen from personal experience, students especially in the first two years, there really aren’t a lot of opportunities for feedback. It’s not like on the rounds or on the wards and so what happens is students fall back and they place maybe too much importance on things like standardized testing exams and quizzes and tests and evaluations. So, the evaluation becomes feedback. So, does the potential for evaluation affect learners’ perceptions of the feedback they are receiving?
  
  Dr. Zinski: So, I’ll start my answer by saying – by making the distinction that feedback at least when we’re talking about in a clinical setting or experiential type learning is going to be a different kind of feedback because it’s skill-based, experience-based versus when the assessment is going to be something that is knowledge-based or recall-based. And so, I’m going to look for that in maybe a written exam and look for a score on an exam if I’m testing recall. Whereas once I’m in a experiential learning environment, I will likely be testing skill and knowledge again and those will be together and most of the learning is done in that experiential place. And so, that’s a big area of distinction in medical education because you get a lot of classroom and small group learning and then you get this hard does of experiential learning and so the formats are [00:26:06] I’m being specific today to clinical setting.
  
  Dr. Mullins:  Would you all say that learners understand the difference between assessment of recall and assessment of skill based learning and how that feedback is or should be given?
  
  Dr. Zinski:  I don’t know if learners or faculty always understand the distinction, but I do believe that faculty know what they are trying to help students achieve. And so, I think a good faculty member knows the goals and objectives of their program or their rotation by heart and is trying to find the best ways to gauge those things with assessment.
  
  Dr. Harada:  Well what I was going to say is to your earlier point, I do think that students are so scared that when I give them feedback, it’s a sign that I’m going to give them a bad evaluation but sometimes, it makes it hard for them to learn from the feedback because they are – the other part of their brain is going oh my gosh, this means I’m getting a terrible evaluation. And so, what I try to explain early on, is actually this is a way I’m going to get you a good evaluation by the end of the rotation because if I can give you these small corrective pieces of feedback, as we go along, as well as, encourage and reinforce the good things you’re doing; that’s going to end you up at the end of the rotation getting a great evaluation if you can actually do the things I’m trying to teach you, you’ll be better. And so, I try to present that up front and early to try to allay student’s fears that any corrective feedback I give them is a sign that they’re going to get a bad evaluation. Because it’s not the case.
  
  Dr. Willig:  And I think one of the questions on our evaluations is was this learner responsive to feedback. And it’s interesting as I hear you speak, to sort of ponder how we are maybe making that issue worse by sort of building it up to oh it’s just going to be a test and it’s an evaluation. But I think to your point, I think a lot of learners don’t understand the difference between this is skill-based and we’re going to - repetition and iteration is how we’re going to get better. I’m going to give you feedback every day to get you to a point where you’re an expert as opposed to there is one final assessment at the end and it’s high stakes and that’s all there is. I think that type of anxiety and sort of I got to get this right, because it’s all my eggs are in this one basket. I think some of that translates over into the clinical setting and I think our learners need to hear us say, heh, over here, it’s iteration, it’s repetition, and we’re going to get better every day. It’s a different environment. It’s a different type of learning.
  
  Dr. Zinski:  And it’s formative. Remember that your tests and your exams and your summative score is meant to be summative, it’s meant to be a summary or a grade or a score or judgment about how someone did that’s the sum of a lot of our formative stuff is to shape our learners into something else.
  
  Dr. Mullins:  So, maybe make it clear that to a student, that might be so heavily evaluation based and it’s just been engrained in them for so long, make it clear that this formative instruction or formative feedback, the goal is to help you receive a better summative evaluation or assessment at the end.
  
  Dr. Zinski:  I’m going to say the goal is to help you learn.
  
  Dr. Harada:  To help you be a great doctor.
  
  Dr. Mullins:  Yeah, that’s –
  
  Dr. Willig:  Yeah, if you say – I always tell the students that if you set your bar at getting 100s in this rotation, that is probably slightly above the lowest bar which is passing the rotation. Your bar is I want to be a great clinician and I want to be a patient advocate like those are the stratospheric goals that you set for yourself. This evaluation, this is sort of – you have to pass to keep moving along in medical school but if this is the ceiling, of what you’re reaching for; you are way underselling your potential.
  
  Dr. Mullins:  So are there any specific or particular situations in which giving feedback is harder than it necessarily should be?
  
  Dr. Harada:  So the situations that I find the most difficult to give feedback in are the ones in which a student truly hasn’t been given any feedback for a long time and they are doing something like egregiously incorrectly. And often, it’s something about, something that involves sort of their personality or their just sort of intrinsic approach to their work. That’s the kind of thing that’s a lot more difficult to give feedback on compared to how you did the abdominal exam. And so, those are the situations that I struggle with and that I know a lot of my colleagues struggle with when you are giving feedback about you didn’t show up on time or you appear uninterested in your patient when you were interviewing them.
  
  Dr. Willig:  Or you judged your patient a certain way.
  
  Dr. Harada:  Right or you came out of the room and said something completely judgmental about your patient’s request for morphine.
  
  Dr. Zinski:  And so because it’s hard to describe that behavior?
  
  Dr. Harada:  Because it feels personal. Even though I’m describing it, it’s so obviously wrong in my eyes, that it feels like I’m saying that they are a terrible person and I’m worried that they will interpret that I am saying that they are a terrible person. And sometimes they do. And so it feels very – much more sensitive and much more personal when you start talking about a behavior like that.
  
  Dr. Zinski:  So, I would argue that telling someone that they were judgmental or dismissive or aloof is not an observation it’s a [00:31:41]. And so, it’s noticing a behavior that did that that’s very, very tricky and so you can get a gut feeling or I’ve had a gut feeling that a student was doing something that maybe didn’t have great intentions or even worse, annoyed me. But the thing that I have to do for them is watch them and be able to tell them what they did. Because if I start with a judgment, it is dismissed. Because I judged. And really, that’s not true. If I give a judgement if it’s negative, people don’t like that. If I give a judgment like great job, they will eat that up for dinner and lunch and so, the idea that I can not say you are aloof, but say I noticed that you looked at the screen and only made eye contact with the patient one time during the entire encounter. And be able to give them a learning moment and they might refute it and say no, I looked at them twice. And you’d say, so you only looked at the patient twice during the encounter. And so then you can have a conversation about the truth that just happened without saying aloof and it take five extra seconds and some great observation skills which again, is probably another podcast. But it’s the best part, it’s the truth part.
  
  Dr. Willig:  Tell us more about just regular skills-based feedback that where the student is not performing at a satisfactory level in terms of skills and we now have to provide feedback about that.
  
  Dr. Zinski:  So, a lot of the things that we’re talking about today, as far as giving observations all the time and doing something that’s timely and when I say timely, I mean within 24 hours because we tend to forget. And so, if I’m thinking about getting with a student or talking about a learners performance every 24 hours, I’m doing something much more preventive so that I have far fewer tough situations with learners.
  
  Dr. Mullins:  And you mentioned the therapeutic alliance earlier and I had read about modelling that on the educational alliance. And how important is the perception, the students’ perception of the feedback they are receiving?
  
  Dr. Zinski:  The feedback that is going to reach a learner in a way that makes sense for learning is going to be observation-based feedback. And so, a student isn’t always ready to hear that they didn’t do well the same way any adult learner is not ready to hear that they didn’t do well, but you can’t reflect on your performance all the time. We do it, all day long. And so, hearing that, with within a teaching moment, and then relating it back to the goals that I’m supposed to achieve as a learner; reminds that learner here’s truth about my performance and here’s what we’re all trying to get to. And so, that is rejected far less than something that was delivered with advice or judgment that is much more easily and personal and in learning situations, much more easily rejected.
  
  Dr. Mullins:  So, good feedback can in turn frame the perception of the student inherently.
  
  Dr. Zinski:  Absolutely. Good feedback when presented well, is something that helps the student learn. It reminds the student that you are helping them to get somewhere.
  
  Dr. Willig:  I always reflect that some of the things that Dr. Zinski has taught me is sort of the four – I would say the cardinal signs of good feedback and it’s got to be timely. The closer to the moment that the observation happened, the better. The more it’s impact on sort of helping that student learn and correct that behavior, the more you let things fester; it’s almost like an infection, you let it fester, it’s going to grow. It’s going to be a bigger problem. So, address it, acutely. So, timeliness.
  
  The second thing is it’s got to be tied to an objective. So, again, I know where I’m going, my job it to make you better – a better physical exam, a better history taker. You need to know better know these concepts for this illness, so I have be objective and the feedback then is tied to that objective.
  
  The third thing is it’s got to be specific. And I think we’ve heard about how important it is to really say I saw you do this, or I observed – this is my observation. It’s very much what exactly was seen by the teacher and we can go from there. And ultimately, it’s got to be actionable. My opinion about what you did, my judgment is not necessarily going to help you get better. What’s going to help you get better is you are here, specifically, and to get to the next rung up the ladder of skill, these are the things that you need to build on. So, those four things have helped me very much, really shaped my feedback messages and as I hear you speak today, I really appreciate the concept of timeliness in terms of how close to that – as immediately adjacent to the observation, the more the power of the feedback that you deliver.
  
  Dr. Zinski:  Absolutely. And I like to use the example, remember the dinner that cooked me three night ago? It wasn’t salty enough. And they say what, I don’t ahh. But if I said, oh I just tasted the meal that you’re cooking, do you think we could get some salt in there and so I would ask them if that was something they would do or I’d say I just tasted the meal that you cooked, I noticed that there’s no salt and then I have a moment that someone can make a decision about what they want to do next and how they might cook next without forgetting or judging that they’re a bad cook.
  
  Dr. Willig:  And frankly, if it’s timely, you have a chance to correct it and we see the tragedy of a student getting to the end of a rotation and hearing feedback that they could have corrected easily earlier on and now they are getting a summative evaluation of a certain kind because all the formative opportunities were left at the wayside and frankly had the student heard those things, they would have corrected them, they would have had time to correct them and they had the will and the interest and the desire to correct them, they just never had the chance to. That sad situation is one that we encounter still.
  
  Dr. Zinski:  Sure, this reminds me of the GPS example and the reason that GPS works for a few moments is because there is a GPS monitoring me via satellite watching what I’m doing. And so if I set a destination, together with the lady who helps me in my phone; to get to wherever I’m going, I set that destination and I know that – I trust that she’s monitoring me and I can see it because I can see my icon making turns and not making turns and she doesn’t wait until I’ve made five wrong turns to say you’re lost. And that sucks. You did a terrible job driving today. She tells me every time I need to make an adjustment. I’m watching you make a left turn when we are supposed to be continuing this way to get to wherever it is that we set together. And so that GPS is something that we’re used to, that’s a model that we’re used to. And when I make a wrong turn, she doesn’t call me a jerk, when I make a wrong turn, she lets me know – reminds me of the goal. And so that’s something easy to remember for GPS and teaching.
  
  Dr. Willig:  Well let me tell you a very difficult one that hearing you is making me very uncomfortable remembering a situation where I mean both the student and I – or in this case it was a resident, we judged this patient and we judged them in very different ways. And the issue on hand was one of us wanted to send the patient home and one of us felt that the patient was not ready to go home because it was going to – the temperature was going to drop and there was actually going to be an ice storm that day. And this was a homeless person. And we fell on very different sides of the spectrum on that issue. And the resident was visibly angry at my decision to keep the patient in-house and had some very strong thoughts on healthcare utilization, a lot of important like it’s great that you know about those things, but at the same time, what I felt like I had was a de4cade of experience at that point so, I think the best thing here, is to just wait until tomorrow.
  
  So, again, I could say that. I could – you know the student was visibly angry, he was aloof, he was all of those things but boy this became something where I really stumbled to give feedback and the only thing I could – I looked at the student and I said, they were literally trembling, clenching their fists when we walked out of the room. And I said, you seem really angry right now. Can you help me understand what’s going on? And it didn’t go well from that point. But let me – I mean that would be a situation where I want to give feedback and my feedback could be something if I passed my judgment like boy, you’re being a little heartless guy. But that would be the most terrible thing to do. And he sort of told me well you’re being completely foolish and disrespectful of healthcare utilization and all these things. It was a very fraught issue, but I think neither one of us was – an communicated effectively there. I wonder what you would do with that?
  
  Dr. Zinski:  So the difference of opinion is always going to come up because you will be a different levels of training and most of us just had graduated learners who got out of two years of book smarts, book studies and so, they might have a different kind of perspective that someone who has just had five or ten or twelve years of experience in the healthcare system. And so this is when it comes back to the really great truths of having goals and objectives that you care about and that you have memorized and then really being able to observe somebody’s behavior that relates to those goals. And so if you don’t have and I anticipate that you do have a goal about healthcare systems; you have a beautiful teaching moment about interacting with patients and learning about the healthcare system.
  
  And so, if you have an objective that’s based on learning the nuances of the healthcare system, I can say, I noticed that you advised the patient – I noticed that you advised the team et cetera, et cetera or I noticed that you made a recommendation about x, y, z. In this particular situation, when we’re thinking about healthcare systems and then go back to whatever your objective is. The nuance stuff is the stuff that gets us in trouble if we can’t get it back to a learning objective. It’s a difference of opinion then you can look at somebody and say I noticed that this happened. In my opinion, I would do something else and you let them know that you’re having a difference of opinion and this might not be a learning moment, this might be a moment where you are having a difference of opinion and you’re having a professional conversation that isn’t a feedback moment about someone’s skills and how they can get better.
  
  Dr. Willig:  So, that’s interesting. So, in that scenario, it wouldn’t go into – it wouldn’t be heh, here’s my thought on what we should do, my opinion versus your opinion; it would rather be – because I think there’s two principles that are sort of – relationships with patients is one of our objectives. To have good and empathy and those things. Another one of our objectives is to have good healthcare utilization. And I would argue that the student had a fantastic healthcare utilization case and I think I had a fantastic patient interaction or treatment of people case. But that wasn’t going to be resolved simply by the objectives. Because I think both of us were standing firmly on an objective so perhaps the path through would have been to say listen, we are having a difference of opinion. From my experience, and what I’ve seen, doing this job for this amount of time; this is what happens here. So, from what you’re saying and all that, I’m not arguing with you that that isn’t valid. I’m saying that that’s a different perspective. Ultimately, I have to make the decision here as kind of the senior person on the team and this time I’m going to decide this. When you are the senior person on the team, you may very well choose this side differently and that’s okay. We are having a difference of opinion here. Is that how you would frame it?
  
  Dr. Zinski:  I would maximize the learning moments that that person could have so, if there’s something to tie back into a learning moment for this person, then I would give that to them because that’s our job. And then I would make sure that yes, it’s very distinctively different to have an open conversation coming out of the teaching alliance and maybe more into a negotiation, a professional negotiation about the way that you would manage care.
  
  Dr. Mullins:  So, would say that like subjective opinion and formative feedback are not compatible?
  
  Dr. Zinski:  So, subjective opinion as far as sharing that with a student, sharing judgment is not something that you want to do as part of objective feedback ever. And we accept it because we accept nice job, and we accept atta girl and atta boy and nice work back there kind of things and we’ll take the subjective as long as they agree with something that I think is positive and the negative that was no good, that didn’t work or that was a mess is something that we reject because it’s judgment and we recognize the judgment when it’s negative. And so, good feedback is based on - good skill feedback is based on the doing and reviewing and the doing and reviewing and you have to be able to say, this is what you did, this is how we’re going to reflect about it, don’t forget, this is how it relates to your goals. That’s the learning way, if you care about learning outcomes, that’s [00:45:52].
  
  Dr. Willig:  Thank you. That was an uncomfortable situation I think to go through and even just re-discussing it, I think me feel just a little uncomfortable still. But I appreciate the addition to the tool set of just kind of taking that offline and sort of saying heh, we’re having a difference of opinion here. And that’s okay and taking it out of the feedback context because when I tried to give feedback about that, it really was my judgment kind of leaking through the feedback that I want to be was heh, you ought to be a little bit more like me pal and that’s completely wrong. That person is their person, they have their experiences and they are equally as valid as mine. So, it’s important not to insert that into the conversation, our personal judgments but rather keep it at the level of professional, objective feedback to make sure that learner is getting better and if we disagree with them, it’s okay. But this is not the forum to discuss that.
  
  Dr. Zinski:  Sure.
  
  Dr. Willig:  Thank you. That’s a great way to end that. Haddon, any closing words?
  
  Dr. Mullins:  I think that was the most informative session yet. So, thank you Dr. Zinski, Dr. Harada and I hope you all enjoy. Thank you.
  
  Dr. Zinski:  Goodbye everybody, thank you.
  
  Dr. Harada:  Thank you.
  
  Dr. Willig:  Thanks.
  
  Host:  And a community physician can refer a patient to UAB Medicine by calling the MIST line at 1-800-UAB-MIST. That concludes this episode of UAB Med Cast. To refer your patients or more information on resources available at UAB Medicine, please visit our website at www.uabmedicine.org/physician. Please also remember to subscribe, rate and review this podcast and all the other UAB Medicine podcasts. I’m Melanie Cole.
• Hosts:Melanie Cole, MS