Simulcast Journal Club is a monthly series heavily inspired by the ALiEM MEdIC Series. It aims to encourage simulation educators to explore and learn from publications on Healthcare Simulation Education. Each month we publish a case and link a paper with associated questions for discussion. We moderate and summarise the discussion at the end of the month, including exploring the opinions of experts from the field.
This month we review a seminal article in Simulation literature, and we hope to encourage new simulation fellows and educators to comment.
Title : “Hello old friend”
The Case :
Nitin closed the door of the debriefing room as his fellow trainees filed out. It had been the third debrief of his Simulation Fellowship and with his consultant Brad’s mentoring he was getting the hang of the basics, but today’s scenario had left him feeling frustrated.
“I’m annoyed at them.” He said, “They handled that case badly but they kept saying things went well! I don’t understand how they couldn’t see what a disastrous resus that was!”
Brad leaned back in his chair and looked at Nitin thoughtfully, “Who do you think let them off the hook?” he asked.
Nitin paused. “Luke was team leader. He should have had a better handle on things, but he kept defending the rest of them. Luke should know better, he’s got the same experience as me.”
“Who else let them off the hook?” Brad asked again, an enigmatic smile forming on one half of his lips.
Nitin frowned. “Sabrina made several errors but she just blamed the simulation environment.”.
“Nobody else?” Brad asked, and after a minute’s reflection Nitin’s heart sank as comprehension dawned.
“Oh crap.” He said. “It was me. I let them off the hook. They’re my colleagues, I didn’t want them to think I felt ‘above them’. So I didn’t acknowledge what I saw. I was too focused on being nice but in doing so failed to meet their learning objectives”. He sighed. There was still so much to learn.
Brad got up and walked over to the bookshelf in the corner of the room. He pulled out a dusty journal that must have been at least 10 years old. “There’s something you need to read.” he said.
Nitin was surprised by Brad’s expression. He was looking at the article like it was an old and dear friend.
The Article :
Rudolph, J; Simon, R; Dufresne, R; Raemer, D (2006)
Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare., 1(1):49-55, Spring 2006
Discussion :
This month we are looking at a seminal article in simulation education and an article that is likely at the top of many reading lists for new simulation educators.
First published in 2006 in Simulation in Healthcare’s Spring Issue, Rudolph et al’s article has had a profound effect on simulation education throughout the world.
For those of you who’ve never read it before, particularly new Simulation Fellows or Junior Educators, please let us know your thoughts!
For the senior educators in our field, it has been 11 years since this ground breaking article was published. What are your thoughts now on the article itself and where simulation education has evolved from 2006? Is advocacy and inquiry still the primary tool in your debriefing set?
References :
Rudolph, J; Simon, R; Dufresne, R; Raemer, D (2006)
Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare., 1(1):49-55, Spring 2006
Thanks Ben and Vic for choosing this article. It’s something I have wanted to read more about after hearing VIc talk about it at the Teaching Course last year in Melbourne.
I’m a novice medical educator and it is really useful for me to reflect on the content. I’m a big proponent of reflective practice and it is something I have been practising for a while after being formally introduced to it as a retrieval registrar. Now I’ve moved on I find that 5 minutes spent reflecting, purposefully, on my day makes it easier to go tot sleep at night and stop the mice running around the wheel in my brain.
When I am debriefing I have found it hard to move from the “Guess what I’m thinking” approach of judgemental debriefing, especially where there is an obvious knowledge gap. I’ve realized that the outcome of my debrief though is better patient care/improved workplace happiness/better communication and is really not about me or people not meeting my standards. In some recent courses I’ve been on I’ve seen such excellent role modelling of debriefing that have really impressed upon me how this approach works, especially with teams of mixed experience levels.
It is time I move away from conflict avoidance and figure out what the true purpose of the debrief is using the fundamental principle that all the learners intentions are good, they just may not be correct.
What I found most interesting in the article, though, was the suggestion of the effect of culture and background on the process. Rudolph et al. suggest that those that come from a culture of deference to higher authority may be less likely to disclose their viewpoint. I would also put it that there may well be other elements that come into play such as (unfortunately) gender – of either learner or de-briefer. If we take this out of the simulation context and move to the more general area of medical education, especially with regard to teamwork and communication, does anyone have suggestions as to how best to tackle the learner with lack of insight. I have concerns that debriefing with good judgement just doesn’t work when internal frames of reference are so disparate that the learner just cannot grasp the debriefers point of view due to cultural blindness?
Hi Andy, Ben, Rowan, Vic and journal club folks
Great conversation here. Love the comments shared so far.
Some thoughts
– effect on culture – yes, totally agree here. Hyun Soo Chung and colleagues published a nice paper in SIH journal a couple year back describing the influences of culture on debriefing. Those cultures that manifest a higher power distance (ie. the instructor is viewed as the holder of knowledge, and learners are present only to absorb knowledge shared by instructor) may have more difficultly implementing debriefing with good judgment. A few years ago I was doing a debriefing in Shanghai with a group of experienced clinicians. When trying to ask questions using A/I, I failed miserably at trying to uncover their frames. The learners, while experienced and well-reasoned in their actions, were reluctant to share their frames due to pre-existing cultural norms. Lesson learned … as Rudolph et al state, trying to communicate expectations ahead of time might help … or not…..
– learners with lack of insight – I view A/I as a technique that helps to verify a learner’s lack of insight. Once you’ve determined that by uncovering a frame that is inconsistent/illogical, then you likely need to use other techniques to address the issue. Validating and acknowledging their point of view (to a certain extent), broadening discussion to a related but different context, referring to existing guidelines, seeking input from other learners, are all strategies that can be used to manage poorly insightful learners.
– re: debriefer paralysis – such a common thing to hear …. and a very real struggle for debriefers of all levels. While scripts can help to provide various options for suggested questions, I generally advise people just to be themselves. Don’t try to do too much. Speak like you would normally speak. Try one new thing per debriefing (eg. one good inquiry question per debriefing?!?). And remember …. all of us know how to participate in discussion … sometimes getting someone to share their thoughts is as simple as previewing the topic (“I was hoping to spend some time discussing strategies to manage multiple medication orders”) and pairing it with an open ended question (“What are your thoughts on this matter”), and silence …..
Thanks Ben for getting this wonderful discussion going.
Andy, thanks for replying! Like you I’m in a similar boat : “conflict avoidance” and “guess what I’m thinking” are two big troubles I sometimes have in my debriefs. I agree that the article successfully reframes the educators perspective away from their own needs (i.e. herd acceptance and the blurring of the lines between ‘nice’ educator and ‘good’ educator) and refocuses on the outcomes of the participants instead. It’s a powerful lesson and I think avoiding those traps requires ‘constant vigilance’ to quote Alastor Moody. :p
When I watch some consultants give feedback (who haven’t had formal training in debriefing), I notice a lot of the traps described in the article come up. There is often an idea that “psych safety = being nice and not criticising too hard”, and sometimes it feels like a sim devolves into a self congratulatory exercise without a whole lot of actual growth. Everyone feels good afterwards, and walks out giving positive feedback, but in the end it’s a hollow exercise that didn’t achieve the learning objectives. Afterwards the consultants seem confused, “I kept asking them how they did? But they kept saying they did great!”.
Thanks again Ben and team for the case and paper, and Andy for your thoughts. So excited to see this paper discussed.
I read this article first only about a year or so ago and it was initially a lightbulb moment. I felt like I finally had some tools with which to tell people I disagreed with their decisions, in a way that didn’t put them into defense (or even ATTACK!) mode. That felt hugely enabling, and so, I suspect like many new debriefers, I tried it out on the group I most wanted to encourage to examine their own frames: my wife and daughters (at least one of whom is still pre-verbal) Needless to say, the debrief was ummmm challenging and largely ineffectual…
But my difficulties following first exposure to “debriefing with good judgement”, in a context with little mentoring from expert debriefers, seems to be repeating with each new group of instructors we expose to the techniques. Because the potential rewards of a great A-I question are so high, and the risk of blowing it are so real, plus the fact that the language is so new it seems as if many new debriefers (myself included) seem to get complete verbal paralysis mid-debrief. So, in practice, I have not found A-I to be the tool that has unlocked my debriefing outcomes.
But I guess, thats the whole point of the article. That A-I should not be the focus of your debrief, and that ‘debriefing with good judgement’ has three arms: the cognitive frames, the stance of curiosity and the A-I questions.
So actually other factors or key lessons have been far more enabling for me as a debriefer and have slowly made it easier to feel like I’m starting to encourage reflection, elicit frames, and slowly change them.
Firstly, strictly following a structure to the debrief has probably made the biggest difference. We use David Grant’s iTRUST (@davidgrantsim), but, as mentioned previously, its a PEARLS or reactions/analysis/discussion structure in another format. This has also made a big difference in my clinical debriefing attempts.
Secondly, I’ve really tried to focus on one of Walter Eppich’s goals stated in Simulcast 2: Eppich Debriefing; that he knows his debrief is going well when the participants are doing all the talking and they’re talking about how they’re going to do things differently at work on Monday (forgive the probably slightly incorrect paraphrase)
And the third enabler for me has been really trying to focus on the ‘stance of curiosity’, this time inspired by by Jenny Rudolph in Simulcast Episode 4. And the cognitive aid (you guys seeing the back reference?) that I use to try and get this right is to shamelessly copy the words that the Rudolphs, Eppichs, Kolbe’s of the world seem to use and start sentences or thoughts with: “I was wondering….” This really seems to focus me on trying to remain curious.
I suppose I’m curious, and I was also wondering, if some of the group who practice where there may be closer mentoring of novice debriefers by expert debriefers, or where the cultural norms are different, also experience the ‘paralysis’ that occurs with novices trying out the new conversation techniques that don’t come naturally? What are your strategies for overcoming this? What has helped us is frequently debriefing with a co-debriefer that allows one person to gather their thoughts while the other leads.
Once again, thanks everyone for your contributions, seriously inspiring and challenging conversations every month.
Hey Rowan, so great to hear your perspectives. The paper does provide one of those great ‘lightbulb’ moments and I vividly remember that ‘click’ when I read the paper the first time of realising this paper answered so many conundrums I had with my current debriefing style. I agree it can be really challenging to develop your own style when you don’t get the opportunity to experience role modelling from experts. I still can’t believe how much I learned from just watching one of Jenny Rudolph’s debriefs at the CMS course, despite it only being half an hour of watching her work there were so many useful moments I took away.
Having said that, I often hear some negative feedback about A & I too, which is that people feel frustrated being fixed into a particular series of phrases. i.e. “I noticed this…. that made me concerned because ….. what’s up with that?”. It can feel hokey and false. I think sometimes we misinterpret the basic AI structure with ‘this is the right way to speak’. Jenny may have the street cred to end an AI with ‘sup?’, but I just sound like a try hard loser if I throw that out there, and I think one of the longest journeys in debriefing is to just find your own natural style for exploring your students’ frames and being transparent about your own thought processes. Feedback I got on the course was often “Well why didn’t you just say that?”, (ie. instead of trying to structure a formal traditional AI question, speak from the heart). Some people might find that feedback frustrating, or a trap, (You told me to say it like that, now you tell me I shouldn’t!), but I think the heart of the matter is that you have to find your own style for being honest and open about your thoughts while inviting the learner to explore their own preconceptions. It doesn’t matter so much how you phrase it : it matters if you got the right outcomes -> curiosity, opportunity for learner reflection, frame exposure etc.
One technique I learned from watching other’s role modelling is making sure to openly negotiate learning objectives. If at the start, the group has agreed we will discuss the treatment of SVT for example, then sometimes I don’t even need an AI question. I can just open up the conversation : “So our next learning objective was to discuss the groups management of SVT….. *pause* “. That takes the pressure of me trying to ‘construct’ an AI, and instead allows me to focus on what I should be doing : listening to the learners.
Having said all that, I found once I’ve had a lot more practice with AI, it feels less hokey and I can come up with questions faster.
Thanks so much for the chat so far – including the frank confessions as to just how hard it is to do well.!
The concept is the strength of the article. As Andy said – it can be a revelation as to the possibilities of ‘direct AND nice’ – high expectation and high support.
The rewards of actually uncovering frames to discuss are worth the struggle in my book.
However as Rowan said – AI is a cognitive frame, a stance and a conversational approach. Sometimes different conversational approaches can be used with same curious stance and frame of the debriefer. And – if badly executed – the ‘AI questions’ (I’m sure Jenny would cringe at the term) don’t achieve the aim.
I also think the ‘hokey’ can be because of subtle conversational differences according to culture. All the authors are US based, and actually our language and conversation is a little different in Australia so our questioning probably would be too
Personally i had to truly take the ‘leap of stance’ for my questions to start to come out well. What this approach mainly changed for me is the ability to be honest in my opinion, but somehow project enormous interest as to why they did whatever they did that i didn’t like ;-).
Still on the learning curve, but use the technique judiciously in any given debrief, and also find application in meetings, teaching and feedback on the floor ( when i am feeling well behaved….)
Just in terms of article methodology and critique….
This was a ‘Concepts and Commentary’ article in SIH. Many other journals have similar ‘perspectives’ or ‘innovation report’ where articles related to ideas or experience can be published without data sets or true research frameworks. That said, the solid theoretical basis for the approach is well established in the literature cited, and the experience of the team is illustrated in their examples.
Figure 1 is a critical diagram (that i have drawn on whiteboards in many debriefing workshops)
Table 1,2 and 3 offer very practical/ tangible application of the concepts
What more woudl this article have if published in 2017?
I’d like to see the example given as actual videos embedded within the articles, so we can actually watch these interactions with all the nuances of tone and facial expression.
We look forward to material like that from CMS in their planned virtual Community of Practice
Thanks Vic. (*removes ‘A-I questions’ from his vocabulary*) I just wanted to support the idea of videos of debriefs being made available. I’d love to watch how the field leaders do it.
Vic – perhaps we can offer some examples of A/I questions on debrief2learn?
Would this article have been published in 2017? I would certainly hope so. If the reviewers/editors were blinded to the authors names?
As an ‘expert’ debriefer (I use inverted commas, because I very much feel I have more to learn), I can absolutely support and identify with the comments above. A/I is a unique skill, that takes a lot of practice. Even having had much practice, I find myself getting tongue tied or ‘framing’ my question in the worst possible way – setting up the participant to fail, or the ‘guess what I’m thinking question’.
A/I really does enable cutting to heart of the issue in a way that doesn’t cut down the participant. I think what really sets it apart is this idea that you are curious and advocating. You don’t have to use a tricky technique to do this; and so, when training novices in debriefing I will recommend that they choose a simple model to begin with. By choosing something like SET GO or SHARP, you allow a scaffolding appropriate to the individuals skills and expertise – just like we would in the clinical world. But, all you have to do is approach it from a frame of being curious and an advocate for the participant – this acknowledges that participants are well-trained, care about what they do and want to improve. This approach is not limited to A/I.
Hey all – fantastic discussion so far, and makes me remember why I look forward to reading these every month (even as a frequent silent observer…)
I don’t want to seem too hagiographic about this article, but I truthfully remember where I was when someone first introduced me to this article and to the concept of A-I. He had gone through the CMS course and was a faculty anesthesiologist; I was a trainee at the time. I asked him how he always managed to get a teaching point across without sounding so “lecture-y” and he handed me this article. It was, as Rowan described, a lightbulb moment. I’ve since been privileged enough to attend the course myself which I think was a pretty invaluable experience in both the theory and implementation. Even with all of that I still consider myself to be a novice on my best days, and a bumbling goon the other days.
The more I practice with this, the more I view it as a framework rather than a rigid formula. Personally, I do often use the “I saw / I think / I wonder” phrasing but whether that’s an American thing or just comfortable with my normal style I can’t say. What I’ve definitely noticed though are that there are times where that doesn’t work well and if I try to force it to be in that exact format it ends up coming out all kinds of awkward. And that usually leads me to flipping to a nonjudgmental style that is generally unhelpful. As others have mentioned, I think the real power A-I is not the language but rather the curiosity of it. I find that the times when I can genuinely get curious about trainees’ thinking, the words just sort of happen.
Along those same lines, I think our greatest power as educators of adult learners is not to teach them what to think, but rather how to think. Are there times where there is one clear treatment algorithm that has to be followed rigidly? Sure. But I think there are a lot more times where individual circumstances necessitate implementing a more nuanced approach that is tailored to the patient, resources, circumstances, etc. If I focus my debriefing on a “Do this, not that” level (I.e., actions), it does not translate well at all to slightly different sets of circumstances that our trainees may encounter some day. In fact, they might not even see any parallels. If I can focus on the “why this or that” level (i.e. frames), I hope that the learning will be much more widely applied.
I’ll wrap up this rambler by echoing what Ben said about honesty – I have found that honesty can cover a multitude of flubs with our trainees. I have generally become more and more transparent as I debrief, especially when I have some specific fact or data that I think is important to be conveyed. Instead of trying to construct what always seems to be an awkward A-I statement, I just say something along the lines of “I want to share something important with you that you need to memorize”. That way, there’s no guessing on their part and I make my intentions clear.
I love hearing what others wrote – I feel like you are speaking my experience when I read Andy and Rowan’s experiences! I’m lucky to work with some experienced sim people at my center, but sometimes the challenge is to help them remember what it was like to be a newbie. And of course the input from the senior folks on here is absolutely fantastic. Thanks for letting this American hang out!
Thanks for joining the conversation Jessica and Chris, I think what hit me most about your comments is that both of you emphasised that genuine curiosity is more important than ‘finding the right words’. I think sometimes it’s easy to forget this when you’re trying to do a debrief ‘correctly’.
Hello all,
What great comments on A-I and the influence of this article and I struggle to add something new. This article was the most profoundly influential one I have read in medical education. I see it as a companion piece (left bower/right bower, non-identical twins ) to the article “the Safe Container” and the general approach of psychological safety. The culture of curiosity sits on a bed of genuine respect and safety. One without the other fails.
Link:
http://journals.lww.com/simulationinhealthcare/Citation/2014/12000/Establishing_a_Safe_Container_for_Learning_in.2.aspx
When I got exposed to the good judgement article for the first time (pretty near when it was published) it was the relief of stating my own opinion that was my lightbulb moment. An expert was giving me permission to do what felt right. Hurrah!
The alternative? Building a structured platform of frustrating leading statements that I hoped would progress the learner to their own lightbulb moment (which was, of course, agreement with me.) And yes, it was frustrating because so often the problem with “guess what I am thinking” is that they don’t. Or the don’t agree. Or they just see the world from their own perspective. Blimey.
The clunkiness disappears with use, and when you disconnect specific trigger sentence and use as it as an overlay. Sentences becomes chunks or paragraphs with long sections of bounced discussion. What separates those who are learning the technique (and I have watched this in my own learning curve) is the disappearance of debriefing tools into the body of the discussion.
The criticisms of A-I have more to do with application and repetition and I would welcome the thoughts from the authors along the lines of cultural sentence structure discussed by Vic (Americans speak differently to Australians/others). I would also welcome the thoughts of the authors of how their background as non-clinicians shaped their approach and whether it works differently when used by debriefers with a senior role in their own clinical setting.
In settings where I am not an expert (trying to influence my family) A-I has a great role in understanding their world while making your concerns clear. Try using this the next time you are discussing things with your teenage child in moments of “domestic animation”.
Cheers Ben for posting this one. A dear old friend indeed
Ian
Hi Ben & Team
I’ve been a lurker on the list for a while but finally had to break my silence because I too feel like this article is “an old friend” of mine.
I’ve enjoying practising the A/I technique (sometimes successfully, sometimes less so) since I was first introduced to it when CMS first came over to Australia almost a decade ago. I’ve tried it on in a variety of settings: as an educationalist, manager, mentor, team member, learner, and father of teenage boys. I’ve sometimes been described as a ‘blunt instrument’ trying to wield a tool that’s far more refined than me 😉
In the end, I think it can be a great model for respectful communication.
It can also be very powerful. I’ve sometimes been surprised even by those colleagues who I would have initially (and prematurely) labelled as not having insight. On more than one occasion, I have seen real behaviour change in these individuals after a “good judgement” conversation. Maybe there’s no such thing as “lack of insight”. It might just take some of us longer to self reflect.
I also agree with your comments, Ben, about ‘psychological safety’. I think that if we brief learners (I’ve always believed that if a sim event goes poorly, most of the times it started with a substandard brief) that the debrief is intended to be challenging and honest, then we don’t challenge (to be nice), we are actually being dishonest. If we don’t follow through on our promises, we are neither respectful of their time or their emotional investment.
I’ll keep practising.
Keep up the great (and thought provoking) work, mate!
Thanks for joining us Jason (and for the record, Jason and I had a fairly similar conversation to the case study many moons ago when I was his simulation fellow). I look forward to having more conversations about it with you in the future :p
Thoughts for the day;
1.) To what extent does the addition of judgement (usually applied as instructor judgement) centre or at least bias the conversation around that instructors opinion? Is it possible to both use AI and be learner centred?
2.) Yes
3.) Ben has warned us that comments are closing and it is therefore possible that I will get the final word. Is it disturbing that this brings me a small amount of joy?
Hey Ian! I was thinking about a problem similar to your question (1) the other day. I agree that the addition of instructor judgment centres the conversation around the instructors opinion, however I think that if it is genuinely framed with an air of both curiosity and openness, it can still invite critique or disagreement. Indeed sometimes I find by the time the learners have bonded through a few scenarios over a course, their herd protective strategies can be such that they will defend each other against critique despite whatever ‘perceived authority’ the debriefer may have. I quite liked what they wrote in the end of the article that it is vital instructors “instead of treating their own judgments or concerns as the single “truth,” they [instructors] test their views against the trainees’ view of the same issue.”.
With regard to being learner centered and with good judgment, I have actually found it very difficult to get learners to submit topics for discussion at the start of the debrief IF I offer my suggestions first. While this is not a particularly revolutionary observation, it does remind me how subconsciously powerful that seniority structure is, no matter how cuddley and open I try and make my debriefs.
With regard to 3) Second to last word this time my friend :p
and yet….
the discussion then becomes about proving or disproving your theory, which takes away the potential of unfettered imagination-you might have missed the real issue (from a learner’s perspective) entirely.
But I appreciate your point about curiosity and that with the right hierarchy, safety, discussion rules and group dynamics it can be safe to challenge you, and quite good fun to be challenged.
If A-I/good judgment tends to fall down it’s when its either done badly or is thought of as the ONLY way to approach a debrief. Which is not the fault of the article, the principle or the fine teaching of the Harvard group- it’s just the fact that debriefing is actually quite a difficult skill, and if there was one answer to how to do it well it wouldn’t be so much fun.
Your turn….
Actually, I did want to ask you a question Ian. You made a brief comment earlier in the thread… “Would this article have been published in 2017? I would certainly hope so. If the reviewers/editors were blinded to the authors names?”, and I was wondering if you could explain your thoughts behind the question?
Ben,
The question raised “would this have ben published in 2017” actually came from Vic in the comments before me and I would have loved to have heard more from her about this.
I read her question and reflected on a simulation based article that I had just reviewed myself. Now, the authors names I didn’t know but I wondered what I would do with an article from someone I really admire like the authors of “good judgement” if i didn’t rate it highly or (yikes) if I just didn’t understand it.
So I’m going to pass this one back to Vic, if I may and ask:
1.) Have criteria for publication changed since 2005 to make you question if it would be published?
2.) How would this change based on the respect/position these authors hold?
3.) Should the process of review of journal article be open or blinded (either authors to reviewers, or reviewers to authors) and if it was, would this have any impact on the outcome of your question?
Ian
What an amazing conversation this month. Thanks to everyone and great choice in getting an old mate article out there. Reading through the comments I can’t help but feel the parallel with the evolution of sepsis management. Are the techniques described Debriefing with Good Judgement the Manny Rivers’ EGDT of simulation? By that I mean it has become part of the accepted truth of how we debrief, engrained in every simulation debriefing workshop/course, evolved and tweaked an to a point where ‘standard best practice’ looks and feels like the techniques described in this paper. The experts have moved on to contextualising the stance and principles to their own culture and resources, while novice practitioners may access AI questions like a script (or bundle).
No real substance to this comment, just a curious analogy.
Thanks again for the shared learning.
Yeah it’s been a whopper of a month, and so many great conversations. It’s understandable that there’s such a long delay for academic milestone to translate to clinical practice, but I’d have to say I agree with your suggestion that maybe the experts have potentially moved on to a more elaborate approach.
Which brings us to next month :p