Introduction :
Simulcast Journal Club is a monthly series that aims to encourage simulation educators to explore and learn from publications on Healthcare Simulation Education. Each month we publish a case and link a paper with associated questions for discussion. Inspired by the ALiEM MEdIC series, we moderate and summarise the discussion at the end of the month, including exploring the opinions of experts from the field.
The journal club relies heavily on your participation and comments and while it can be confronting to post your opinions on an article online, we hope we can generate a sense of “online psychological safety” enough to empower you to post! Your thoughts are highly valued and appreciated, however in depth or whatever your level of experience. We look forward to hearing from you.
The Article :
Kerrey, Benjamin, MD, MS, Boyd, Stephanie, et al. Developing a Profile of Procedural Expertise: A Simulation Study of Tracheal Intubation Using 3-Dimensional Motion Capture. Simul. healthc.. 2020;15(4):251-258. doi:10.1097/SIH.0000000000000423
The Case Study :
It had been a month since Harvey had been unable to intubate an unstable, septic neonate but the memory of desperately yet unsuccessfully manipulating the laryngoscope remained fresh in his mind. He remembered the looks of polite confusion from his junior staff when he failed, as if their perception of him had dropped dramatically but they didn’t know how to articulate it. Fortunately for him and the patient, it had been nothing an LMA couldn’t fix and the baby had recovered beautifully. Much faster, in fact, than Harvey had recovered from that particularly bruising identity threat.
He’d realised that he’d reached an alarming state of unconscious competence as a NICU registrar, to the point where his muscle memory intubated so well his brain had contributed very little. 3 years as a Consultant without intubating though, had atrophied those particular neurones, and skills wise he’d seemed left with nothing.
Expertise, he thought to himself, was a funny thing. Hard to define, hard to measure, and even harder to maintain.
Discussion :
This month we’re exploring procedural expertise, and simulation’s role in describing and identifying what procedural expertise actually looks like in practice.
As always, enjoy the paper and we look forward to your thoughts!
Hi Ben, fascinating article! I suspect this type of research will become more valid over time. I have a few thoughts about the study that I would love to get some feedback on.
On the intubation process: observing various simulated intubation attempts on various mannequins I have personally focused intubation learning on epiglottoscopy as the first step, and slowing the intubation attempt down until the epiglottis is located. My experience is that finding the tip of the epiglottis is the first most important step in the procedure. At least the way I am reading this article, they combined the entire laryngoscopy attempt into one process. I am a fan of and see the best results with more smaller steps including opening the jaw widely, controlling the tongue, finding the tip of the epiglottis, lifting the epiglottis, etc. My concern here is that the researchers combined what I consider to be different steps into one process, and described the acceleration of the laryngoscope during that entire process. I suspect we would find that the experts would perform the steps prior to lifting the epiglottis more slowly, I would be interested in learning more about data collected about each small step instead of the combined action.
On the mannequin selection: my experience with similar mannequins is that while they reproduce the shape of an airway, they do not reproduce adequately the physics of the airway. This is not an advertisement for any particular vendor, but once we purchased an intubation head from 7-Sigma (http://www.7-sigma.com/7-s3/) I saw a pretty significant difference in the need to open the jaw and control the tongue during intubation attempts in comparison to those practicing on basic airway heads similar to those used in this study. When the teeth are sharp, the tongue is floppy, the jaw moves like a human jaw, the head is heavy, etc the required skill in all of the steps of intubation become more important. The authors correctly noted this as a limitation. I would just like to point out that I believe for utility of the research it will be important moving forward to validate the physics of the mannequin being used for this type of research.
On laryngoscopic and arm movement as judgement of intubation expertise: what I’m trying to understand in this study is that the “theory of radical embodied cognition” is that cognition is exposed in our movement. I personally believe expertise in intubation requires a set of skills not measured by our movement such as decision making, leadership, patient selection, setting one up for success with plans A, B, C, stoping an intubation when unsuccessful, moving through the algorithm, etc. Those are not exposed in studying the movement of one’s arm during an intubation attempt. So I feel like a different term is needed for what this is attempting to reveal expertise in. Maybe the expertise is in laryngoscopy and placing an ETT, but that is different than what I consider expertise in intubation.
As always, I would love to hear everyone’s thoughts, and thank you for providing this fascinating and important study. I have to admit that I skimmed with disinterest it when I opened Simulation in Healthcare over the summer, but am happy for the opportunity to think more deeply about this type of research.
Hi Lon,
Thank you so much for such a considered critique and review of this paper. I think you have role modeled a really thoughtful approach to an article of this kind and certainly have helped me understand a more sophisticated approach to the data presented.
One of our reasons in selecting this article was that (as you mentioned) it’s a different type of simulation research, and one that we wanted to highlight as one with potential academic opportunity. To me your comments on mannequin selection highlight an important point : when we aim to quantify/describe procedural expertise through a simulation based study, we end up quantifying/describing procedural expertise on a mannequin. While I agree with the paper that this still adds to our understanding, I think it’s such a pivotal point to approaching this type of research. As I was driving to a course the other day, my colleagues described ‘needing to get a feel for the CPR mannequin’ when achieving > 90% success on their automated feedback scores, for example. We often consider the translation of conceptual learning from simulated education to clinical practice as relatively easy, but when it comes to mapping muscle memory, maybe we need to be more careful about how well that translates to genuine clinical expertise. While we are often somewhat fond of dumping on fidelity as a core tenet of simulation practice on Simulcast, here the functional task alignment is pivotal to our accurate understanding of procedural expertise.
Your second point about the ‘theory of radical embodied cognition’ is fascinating. Although I’m gonna take a little bit of a ‘both sides’ view her. I agree expertise in intubation cannot be exclusively mapped through our movement, although from personal experience and watching others I can also see that those with expertise in something tend to move more smoothly and directly in any procedure. How helpful that is to me teaching a procedure (as opposed to your beautiful illustration of breaking down intubation into more achievable, well described components), I’m not so sure…
Many thanks,
Ben
Thanks for sharing this incredibly interesting study. I really appreciated the details the authors described in measuring what I imagine would be very subtle movements. The outcomes describing faster and more efficient work of the experts using the intubation simulators made complete sense. Reading the paper I couldn’t help but reflect on a lot of cross over to measurability of movements found in other simulated procedural surgical work. VR laparoscopic trainers also measure time, path length of instruments, left and right handedness amongst other metrics. These tangible numbers have allowed ‘expert’ surgical performance to be defined and it has given our junior trainees learning on the VR trainers targets to aim for. The metrics also seem to help foster and encourage deliberate practice.
The questions I had reading the study somewhat echo the above comments – are the described differences on the intubation simulators realistic enough to translate to the real life patients? Would it be feasible to consider a similar study gaining metrics during actual intubation?
It would be hugely fascinating to follow-up this study with how many intubation attempts novice trainees require on simulators to achieve the expert metrics. Clearly by defining expertise – there would be opportunity to then study the learning curves of intubation and also credentialing for novice practitioners using simulators.
I very much look forward to reading comments from others. Thank you again for sharing such an interesting read.
Thanks Belinda, I was really looking forward to your comments given that this seems very much your wheelhouse. I hadn’t considered how the electronic accumulation of this data could allow for automated feedback loops in VR simulation.
Yay – procedural skills! What I LOVE about this paper is the premise – what objectively does a “great” intubation look like and how can we measure it? Sure, as Lon said, many other things are required for a safe intubation apart from the movement, but you do need to develop the right movement technique. If expert movement can be developed in simulation to a level of psychomotor automaticity then all the more cognitive space for learning about those other important things in the real world. I can see how this type of study can lead to AR/VR training models with automated feedback on performance both for both novice learners and those maintaining their skills.
The methods (3.75 pages long!) gave a very thorough explanation of the study protocol, and I admire the author’s efforts in standardizing the protocol and minimizing bias. This is truly an exemplar paper and I could palpate the effort they put into this! As a non-intubator, I was curious about only measuring the shoulder movement in the sagittal plane and the choice of right hand as a proxy for movement of the endotracheal tube – what are other’s thoughts on this, could these be important?
I must admit I quickly gave up all hope of sounding intelligent in this post following an aborted attempt to read some papers about radical embodied cognition. (Insert brain exploding emoji here!) After re-reading (x3) the authors’ explanation of radical embodied cognition, I’m still finding it difficult to be sold on the idea that “measurement of movement … can be a direct assessment of at least part of the underlying cognitive processes”. Yes, our thoughts initiate our body’s movements, but I feel like many things can moderate this before it is expressed as movement. While operating my mind knows exactly where I’d like to place that suture but it doesn’t always land where I intended it to. Why? So many reasons: an adrenaline surge is giving me a tremor, my hand can’t quite get to the right angle due to that retractor in the way, my muscles are tired and just don’t have the fine finesse needed at that point (the list of excuses goes on and on…). I think I would settle for movement as “indirect” assessment, but not really “direct” assessment of cognition.
I’m sorry I couldn’t offer more than these simple thoughts and questions. I hope others can produce some more intellectual commentary. (*just had to use synonyms function to find alternative word to intelligent – maybe I’ll never understand radical embodied cognition…)
Oh Sarah, I love how excited you sounded at the mention of a methods section 3.75 pages long! You’re the best! I would agree that the measurement of movement as a partial measurement of cognitive process is a big leap, although I feel like it instinctively makes some sense when I think of watching experts at something. I wonder if it’s more proportional to confidence or expertise?
Hey, thanks for a great discussion so far.
Its of interest to me as an ageing intubator :-), but also as one who trains others and has had high hopes for YEARS about better methods.
1. Its interesting that the motivation for the paper was to develop a profile of ‘what good looks like’. The approach of analysing what experts do to develop such a profile is a time honoured approach – have seen research on this in endoscopy, laparoscopic surgery and many other things – often a combination simulation and video reflexography.
The risk with this is that the experts may not have cracked the best method! I reckon the Fosbury flop in high jump is a great example of this…….. http://large.stanford.edu/courses/2018/ph240/williams-h2/ Merely analysing all the best high jumpers before Dick Fosbury would not have led to the world record he performed!
The hope is that some clever machine learning could be applied to actually tell us the best combination of movements?
2. I am sort of surprised we still need a motion capture laboratory for this kind of thing when we have all these VR and other motion tracking techniques, but i am certainly no expert
3. I do wonder about magnitude of impact of this kind of research on the procedural skill itself, versus finding ways to improve team and system performance (but then again i would say that). Of course we need both.
I am just dying to use ‘radical embodied cognition’ in a sentence. That will not indicate that i understand it 🙂
I may be showing my ignorance here but I had assumed “radical embodied cognition” was an evolution of “tubular embodied cognition” which I remember being banded around in the 90s.
He rode the bull with such radical embodied cognition that the audience could see it was not his first rodeo.
????
Love this
Imagine the cricket commentary…
“And that’s through the covers for 4…. , what beautiful technique, they hit that with radical embodied cognition”
Thanks for your comments Vic, Having just reread the article for summarising on the podcast, I’m a little intrigued about the level of teaching impact a number of the findings have…. is 2 degrees of variation in angular variability at the risk helpful feedback to a trainee? Like you say, mapping current experts and then teaching trainees to imitate those movements runs the risk of inhibiting innovation. In many ways I see us do that in the way we create mini clones of ourselves when trainees don’t move between institutions.
To me I start to get a bit skeptical about the level of clinical significance despite the statistical significance.
What I like most about this paper is that it identifies a gap in the literature and fills it effectively, without over reaching on its impact in isolation along a journey towards understanding procedural expertise better.
Thank you for highlighting a different take on simulation research, an interesting read.
Intuitively, I can see the benefits of this kind of data. It makes sense that being able to analyse movement and provide objective feedback could be a way of shifting curve from novice to expert quicker. Certainly remember playing sport (as a younger chap) and being filmed then watching it back (whilst cringing) with a critical eye helped to reflect and go round that kolb cycle. As other comments have said – maybe this could encourage deliberate self practice with objective/machine/AR feedback?
I’m impressed by how much effort the authors have put into the methods and making what they did as clear as possible. I thought the definitions of path length, start and end were interesting. If this is the start of building a larger, multicentred data set of experts these definitions need to be robust and reproducible. I wonder how they decided these definitions?
I think that as other comments have suggested non-technical, communications, planning and safety all have a big part to play in the overall intubation and it is important not to lose those aspect in the expert profile (I appreciate that the article is looking at specific aspect of movement).
Thanks Dan for your points, and in particular I loved how you highlighted the value of self reflection through video feedback / data on movement. I do find giving feedback on body language and room positioning etc very useful in traditional debriefing, but I’m genuinely unsure how helpful feedback at this level of detail is helpful as a teaching tool? I understand the academic benefits of defining expertise in an innovative way, I’m struggling to see how to translate that into feedback. But I think maybe that’s just because this is not my area of expertise?
Thanks for sending this article along Dr. Vic Brazil.
Interesting study regarding the mechanics of tracheal intubation. Quantifying movements is one aspect of airway management- trainees and I talk about being smooth with movements, recognizing structures, moving methodically, gently and safely.
Not all aspects of airway management expertise can be captured in one study- how to assess a patient anatomically and physiologically, knowing when and how to intubate that particular patient, along with a Plan B, C and FFF are all equally important.
IMHO first-pass success (FPS) is currently the best patient-centred metric we have to assess expertise in airway management. First pass failure has been associated with hypoxia, hypotension, increased end-organ dysfunction, and death. Louise Park and colleagues conducted an excellent systematic review in 2018 and came up with a minimum acceptable FPS of > 85% as being a minimum acceptable metric. https://onlinelibrary.wiley.com/doi/abs/10.1111/1742-6723.12704
It’s not necessarily exactly how you move (although the findings are very consistent with what I have observed, less movement with more experience) but the identification of structures that also determines expertise. Excellent article and discussion.