Introduction :
Simulcast Journal Club is a monthly series heavily inspired by the ALiEM MEdIC Series. It aims to encourage simulation educators to explore and learn from publications on Healthcare Simulation Education. Each month we publish a case and link a paper with associated questions for discussion. We moderate and summarise the discussion at the end of the month, including exploring the opinions of experts from the field.
Title : “Coffee, Checklists and Self Flagellation”
The Case :
Nimali and Cath sat on the hospital balcony and clutched their coffee flasks in the cold morning air as they had done together for the last 3 years. Handover was in 15 minutes but Nimali knew that with her upcoming conference presentation a week away this was likely the last chance she’d have to catch up with her friend before then. She had sensed that something was wrong.
“The truth is,” Cath sighed, “I’m ashamed.”
“We had this kid last week come in in asystole. I wasn’t team leading, but as the new consultant on the block I wanted to help out. I got asked to do CPR, so I did CPR… I did continuous chest compressions for 2 minutes, with great swaps with Brad, and together we kept good quality going CPR going for about 15 minutes.
But when ICU came down, they asked me whether the patient had a pulse. And I realised for the last 15 minutes we’d been giving CPR without even a pulse or rhythm check.”
She stared out at the traffic below for a while before continuing.
“I’m the consultant. I teach Advanced Paediatric Life Support. I teach on Paeds BASIC. I’ve been doing this for 10 years. The next kid who comes in with asystole might have me in charge, team leading. But when this really sick kid actually came in, everything I’d learned, everything I’d taught, it all went out the window. It’s made me think that deep down, maybe I’m just not that good at this.”
Nimali put her hand on Cath’s shoulder and gave a wry smile. It was company and a safe reflective space that her friend needed now, but the educator in her just couldn’t stay quiet.
“Have you read the Checklist Manifesto?” she asked.
The Article :
Marshall, S. D. (2017)
“Helping experts and expert teams perform under duress: an agenda for cognitive aid research.”
Anaesthesia, 72: 289–295. doi:10.1111/anae.13707
Discussion :
Cognitive Aids are often recommended by Simulation Educators as important tools in a resus, but uptake at many hospitals is scattered at best.
In his editorial in ‘Anaesthesia’ in November 2016, Dr Stuart Marshall provides an overview of current evidence around Cognitive Aids, identifies current knowledge gaps in the research and proposes a series of principles that should underline future research.
In doing so he provides not only important information for future researchers, but also a thorough overview of the principles behind Cognitive Aids in healthcare for clinical practitioners.
For our Journal Clubbers this month :
- If you are involved in clinical research, what points have you found pertinent in reflecting on your own study designs?
- If you are primarily a clinician, how have you found the implementation of cognitive aids in the workplace? What’s worked for you? Has this article changed your thoughts or approach to them?
References :
Marshall, S. D. (2017)
“Helping experts and expert teams perform under duress: an agenda for cognitive aid research.”
Anaesthesia, 72: 289–295. doi:10.1111/anae.13707
Thanks again for sharing another great case and article! Cognitive aids are a great topic for discussion, and Atul Gawande’s checklist manifesto is one of my all time favourites!
The article covers some key points about cognitive aids that Atul Gawande talks about – specifically that they are not useful at a script for novices, but more so as a check point for experts to offload some of their cognitive load. Other key points they talk about, are having them easily accessible, and also not having too many cognitive aids that the user would find it overwhelming.
Other key points to stress on are that they must be short, have efficient wording and have a clear font. Also very important is that it is used as a call-response – i.e one person calls out the item, and another person responds with a confirmation, to avoid the tendency to skip steps if one person does it themselves.
The article refers specifically to crisis situations, but in fact most of the evidence for the benefit of cognitive aids are in routine situations (Gawande’s study on the surgical safety checklist, and Provonost’s study on central line insertions spring to mind). In emergency medicine where I work, I think it is most useful not necessarily for the crisis situation, but for critical procedures involving multiple steps, where missing any of the important steps could lead to a bad outcome. The quintessential example is rapid sequence intubation. In a previous ED where I worked, I was involved in creating a new airway checklist to be used for intubations – and the things that helped with implementation was getting the buy-in from the group, by involving them with the final design, and testing it in simulations. Also, having it easily accessible (attached to the airway trolley), and making the considerations I mentioned before in the checklist design.
For the truly unexpected crisis situation, my personal belief is that checklists have a limited role, at least in the initial phases, and what is vitally important is to have a senior team leader who stands back and maintains situational awareness. For example in the case you described, Cath’s role was to provide CPR, and she was focussed on that task, and spending a significant amount of her “bandwidth” on it. The team leader who was standing back and taking in an overview of the situation, should have ensured the pulse checks were happening.
Although in the anaesthetic environment, which this article is targeted at, I can see that checklists would be more useful, as the type of crises that can occur are a little more predictable.
The other interesting thing is that checklists are more ubiquitous than I initially realised! Every time we fill out a drug chart, there is a check to ensure we have confirmed the patients allergies, and patient ID etc. Every time we request a CT on the radiology form, there is a check to make sure there is no contrast allergy, and the patients renal function has been recorded etc. Every clinical pathway or proforma we fill out is essentially a checklist!
Thanks again for sharing, and looking forward to the discussion!
Thanks for your comments Suneth! Checklist manifesto is one of my favourite books too, and it turned me into a convert pretty rapidly. So I really enjoyed having my new found ‘cognitive aid believer’ status challenged by Stu’s assertion that we have only limited evidence regarding their benefit. Has made me think about CA’s in a much more in depth way.
Thanks for highlighting an article I would not otherwise have read, Ben.
I, too, work as an emergency and retrieval physician and have found a simple airway checklist to be helpful – not usually for myself – but to help create a shared mental model for the multidisciplinary flash teams that I tend to work in. The challenge in introducing such a checklist into our retrieval service is finding something that all craft groups agree with. Whilst most emergency physicians would be happy with one approach, it can be a challenge to get intensivists and anaesthetists to agree with us. Creating a simple checklist then becomes burdensome.
When dealing with critically unwell children I always use a cognitive aid – this time it’s the Monash Paediatric emergency medication book. At times of high stress I don’t want to be calculating drug doses or trying to remember how to make up infusions so I think the 2 pages per weight is a great resource for the team to use (no COI to declare). There is a fine line between the resource being a textbook and a cognitive aid but it is not the sort of book you can read cover to cover. Compare this to the ubiquitous Frank Shann pocket guide – more of a vade mecum than cognitive aid (unless you want to cook roast potatoes).
Hello Ben, Stu, Suneth, Andy and others
There are some terrific points made here in reply and in the article. Great work Stu.
To my thoughts on introduction of a new aid: agree with the articles point about testing and implementation of the aid through simulation and we introduced the Sydney HEMS post RSI complication Emergency Action Cards in exactly this way. They are attached with Velcro to the resus wall above the ventilator (rip off and hold as needed) rather than taped. Repetition in use and familiarity is achieved by the same EAC set up in our daily airway drills room and ideally in ISS conducted where they are actually used. The point about the neglect of cognitive aids under the stress of a situation is valid, and ideally the prompter for use of an aid could be someone with a low cognitive/stress burden- the second ED senior/anaethetist or the scribe nurse. The biggest probelm with a CA is needing a CA to trigger use.
Love the linking of aid position to use eg malignant hyperthermia in the same box as the dantrolene or ventilator troubleshooting aid attached to the ventilator.
In design the main fault is the Homer Simpson car problem where everyone wants to add a bit, making the finished aid unusable. Great design is about simplicity. The aid should be tested and hold up under the gaze of the stressed mind, not designed to be read by an expert committee at their leisure.
Ian
Good design is about taking away
Thanks Ian, Andy and Suneth for your thoughts. Andy and Suneth it’s so nice to have you guys on board with JClub!
Ian I particularly love the ‘Homer Simpson Car Conundrum’ that you describe. The longer I’m a consultant the more I realise how hard it is to get a consensus agreement on ANYTHING, and cognitive aids would seem particularly predisposed to bloating and subsequent poor take up.
I really enjoyed this article because it deepened my thought processes behind CAs beyond “They’re so great and important and have you read anything from Atul Gawande?”, and I think if I’m ever designing a CA for something in Paediatric Emergency this article would be important pre-reading for the group involved because it :
– Gives a great overview of the current status of CA research
– Identifies practical clinical barriers and design flaws preventing CA uptake and use
– Identifies strategies to validate CAs using appropriate research based principles
– Challenges the preconception found in a lot of medical educators (such as myself) that CAs are by definition ‘great’. If they’re not actually making a clinical difference due to poor uptake or confusing design, then they’re actually not helping at all.
There were a number of take home messages for me, as well as some anecdotes that really hit home. Some quotes and points that come to mind include :
– “It is tempting to decorate the walls of the operating theatre with posters describing the management of every conceivable emergency, but these are still frequently forgotten in stressful situations.” (I’m sure I’m not alone in that temptation)
– The fact that there is a design specification for CAs (The CMAT) at http://onlinelibrary.wiley.com/doi/10.1111/anae.13015/full , which is well worth a read in and of itself.
– That aggressive streamlining and easy to read aesthetic design is likely to improve uptake (a philosophy that’s not easy to achieve by committee)
I think it would be easy to dismiss the importance of Table 3, an article highlight for me, which provides an overview of design principles for future research such as :
– “Research into cognitive aid use should target experienced practitioners and their performance.”
– “The purpose of a cognitive aid is to support experts in delivering the best care possible, not to provide a recipe book of detailed instructions for an inexperienced novice.”
– “Learning effects should be minimised and accounted for by study design.”
– “Testing should be undertaken in the actual environment that they work in or a close replica of it, with similar equipment and layout”
The points raised in that table give a great critique of current research regarding CAs, but also that there’s a lot of identified issues that have overlap into simulation research in general.
I appreciate this article isn’t super controversial, but I love the learning points. Thanks so much for writing it Stu!
In my area of general practice emergencies, I find doctors are keen to use aids, recognising that we see these emergencies rarely. Unlike hospital teams, GP teams are small and there is temptation for all to jump in and do something, rather than have one team leader maintaining situational awareness and using an aid. We discussed at our last debrief that next time I would actually just appoint a team leader as no one spontaneously does this, maybe seeing themselves as a flat hierarchy, or worried about unfamiliarity with these rare emergencies. So as the sim educator I will directly place the aid in the (rotating) team leader hands to ensure it get used. Too artificial?
Two related items stood out for me – the problems of physical and particularly cognitive accessibility. We have the posters on the walls, the laminated A4 aids hanging on the side of the trolley but even then the cognitive accessibility, prodding the team member to use it is tricky. We do have the anaphylaxis dosing on the adrenaline box so there is probably a place for me to tape a laminated ALS chart to the defib.
I really liked the focus on linear versus branching aids. The Vortex is unique in both its use of a 3 dimensional model creating the idea of progression and the excellent iconography. Tellingly it was created (and paid for) by largely two people rather than a committee.
Thanks to you all for the positive comments about the article. It’s very humbling as an author and someone who’s spent well over a decade thinking about these issues to get such good feedback.
Most of what I know about checklists relates to the emergency situation. In aviation, the idea of a ‘normal’ checklist (non-emergency) is much more familiar, however I disagree with you Suneth, I think the emergency situation is much more important to have a cognitive aid for.
Why?
Well, in ‘High Acuity Low Occurrence’ (HALO) situations like clinical emergencies we have limited capacity to think of the multitude of important items that we shouldn’t miss. We develop ‘cognitive tunnelling’, are at risk of confirmation biases and can’t effectively recall basic information under stress. Furthermore, we have to manage a team of people that we’ve often never met.
I agree that the urgent situation needs urgent action. So, while looking for the cognitive aid, someone needs to ‘fly the plane’. Pilots generally remember the first three items on their emergency checklist – I’d contest we don’t even need to do that most times – our first three items are generally A, B and C! Nick Chrimes (of Vortex fame) would disagree and cite his own ‘triad’ approach (http://triadapproach.org ) which essentially does the same thing but with more cognitive processing required.
Anyway, I agree it’s all about ‘bandwidth’ for the leader. A poorly designed cognitive aid will chew up more cognitive resources, a well designed one will assist with the normal processes and remind the user of commonly missed steps, doses or differential diagnoses. The problem is often the design doesn’t fit with how it will be used!
Thanks for joining the conversation Stu and Simon,
Simon I have heard the idea of the vortex touched on previously but had never invested time to look at it properly, so thanks for bringing it up! I agree that the excellent iconography helps hugely, although I also note that as with other cognitive aids it requires training, in that looking at the image on it’s own didn’t really explain to me ‘what to do’ on immediate glance. The subconscious effects of the 3D model are very effective however.
The discussion between Stu and Suneth regarding CAs for regular or rare events interests me, because particularly in Paediatric Emergency (where true critical care can be rare amidst the hordes of URTIs and Gastro patients), true emergencies of the ‘High Acuity Low Occurence’ type are exceptionally, exceptionally rare. To the point where we are NOT experts in the crisis at hand. (The case above is an adaptation of an error I made in a resus, and the reflection that after 10 years of training that patient was my second paediatric arrest was alarming. I will never be a true expert on cardiac arrest management, I don’t do it often enough.) As such sometimes we need a CA not as a prompt, but to help us deal with an occurrence in our clinical practice where we have stopped being experts and instead have become novices. I think this conflict of needs would be sometimes what results in CA bloat. Perhaps CAs as discussed in the article are not the answer there, or perhaps technology can assist in streaming a CA (i.e a phone app with a simple interface but increased information available by link for each step).
My final question for the group is “Why has CA development largely remained focused on RSI?”. I get it, it’s one of the most dangerous times for our patients, but we have plenty of other serious events in ED, and the CA development for them is poor. Culturally as well the EDs I have worked in have embraced the airway checklist. When it comes to CPR or something though, there’s no thought of getting some cognitive backup for that equally overwhelming task. Is it because we have been relying on ‘trickle down anaesthetics’ to drip feed CAs into our cultural psyche? If there’s a lesson I’ve learned from doing this journal club it’s that you can educate all you want, but until you achieve cultural workplace change, you’ve not helped as much as you think.
Thanks everyone – great discussion.
I agree with Ben – stretches thinking beyond “Yeah, yeah, checklists are good…” 🙂
Forced to realise that i use checklists a lot of preparation in avoiding crises eg RSI checklist, procedural sedation, but less often when really stretched with a crisis.
ACLS is exception, and a think the failed airway at least has a pre-formed algorithm in my head. But other things I experience as crisis in trauma or acute medical emergencies in ED – I’m not sure we really have the cognitive aids available until we reach the ‘final common pathway’ ie heart stops .
This editorial is just the tip of an iceberg of great work by Stu on this topic. Developing a good checklist is hard
– outcome measures are tricky to decide upon
– getting authentic teams and environment available for the volume of testing needed is challenging
– variations on the checklist may lead to change sin preference of providers but not performance
(‘Preference’ being far more commonly used as a source of expertise for checklist development in my experience)
I think in situ sim proponents should embrace in vivo testing of cognitive aids – but we look to Stu as to how we can develop and use those best outcome measures, AND how we can shape a debrief conversation to shift from preference to performance for cognitive aids.
Great topic – and another one for ideas for sim researchers.
I am personally interested in testing a new ventilator trouble shooting checklist our ED has developed. It make nice intuitive sense but keen to test with a bit more rigor than we have previously.
Thanks again for great chat
vb
Thanks again team
Realise I’m late to the party, and Ben, you may have begun collating your wrap-up already…apologies.
I have a similar concern to you Ben, that the list of important, life-threatening situations or conditions I encounter rarely is too large to maintain expertise in them through simulation or other means. I need to have a CA that works like a reminder and protocol. But I completely see how CA’s need to be drilled. That proficiency in the use of CA’s need to be maintained.
I wonder what Stu and the rest of the team think about the idea that all CA’s should follow a similar pattern and format so that you could pick any one of them up at the right time and ‘know’ how to use it, even if you were unfamiliar with the content?
We’re very much at the beginning steps of trying to develop and test a CA to treat Anaesthesia Induced Rhabdomyolysis. Even in our big paeds hospital, the handful of cases that have ever been managed are remembered by all the senior clinicians….thats how rare they are. I may never see one….but if it happens, I would really like to have something to help the team cope.
THanks again for all the work. Looking forward to the wrap.
You made it!
Hooray!
Hi Rowan,
Great comments, thanks.
You’re right, it’s difficult to remember details of all the cognitive aids so ideally all cognitive aids should have similar structure and familiar content even if you’re unfamiliar with it. Indeed, many of our existing cognitive aids do in the form of DRSABC (Mnemonics are in fact cognitive aids too) and arguably that’s what we all fall back on when we don’t know what to do. There have been a few examples of forcing people to think a different way in a crisis (COVER ABCD is perhaps the best example as noted in the article) but they just don’t seem to work.
Personally I think the trick with designing aids is to work out what effective people and teams do normally in a crisis and engineer that into all teams. Essentially it’s about creating a resilient system. There’s so much we don’t yet know about how we can do this such as the best team structure, the best type of communication and how to support them with cognitive aids. I’m sure this is going to be a growth area for research in the future.