Pre-Mortems – Looking at the Failure in a Critical Manner

A number of events occurred recently which reminded me of a concept I had read in Gary Klein’s book ‘Sources of Power‘ and were discussed in the Freakonomic’s podcast where Gary was a guest.  Pre-Mortems are the ability to critically examine a failed situation and prevent the future failure of that event.

The triggers in my case were:

1. I had undertaken a training course (TDI Heliotrox MOD 1) on a closed circuit rebreather (CCR) in December last year but due to a shortage of cash and time, I was not able to purchase my own rebreather until May this year. I had originally completed the course to expand my theoretical knowledge, but then realised that CCRs are likely to be the future of diving, so probably a good idea if I got one of my own to practice and develop on! So, having not dived one for 6 months, I decided to undertake a two-day refresher training course with my instructor (Kieran Hatton) to get back up to speed.  A number of people had said to me words along the lines of “How hard can it be? Breathe in, breathe out, repeat…”. The course was very good, certainly got me thinking in the right frame of mind again, and importantly reminded me of a number of things I had forgotten from the original course, especially assembly and simple servicing tasks.  One of the key reasons for doing the refresher was because I had projected myself into the future, imagining that I had not done the refresher but died on the first dive on the rebreather due to a simple mistake I had made, and thinking that people would be saying. “He qualified 6 months ago with only 14 hours on the unit, hadn’t dived since then and got in the water without a refresher. How stupid can you be?”

2. At a recent event at an inland dive site, I overheard a discussion about a series of dives which fortunately didn’t end in disaster but could have so easily done so. In essence, the divers had assumed that the additional cylinders of gas that would be there hanging over the side of the boat in the event of an emergency, weren’t there and when they did arrange for them to be there on subsequent dives, they hadn’t actually analysed the gas in them! (More detail here).  When I posted this account to a number of social media outlets, there was obviously significant criticism of the divers involved saying how stupid could they be. Furthermore, a number of divers didn’t get the concept of posting an account of a high-potential near miss so that others could learn from it – the thought being that as no-one has died or injured, why discuss it.

3. The recent fatality in the US Virgin Islands of Dr Guy Garman who, by all accounts, was undertaking a dive to 1200ft for which he was ill-prepared for.  Looking at the posts on social media (ScubaBoard, Facebook and here) it appears that there were significant group and social human factors at play which meant that it would have been very difficult for either the diver himself to say no, or anyone involved in supporting the dive to say no.  In hindsight, there was significant evidence that this attempt was very likely to end in disaster and even if it didn’t, then it was likely to be more about luck than judgement.

The latter two of these incidents and analyses involve significant hindsight bias, whereby the community or individual looks backs spotting all of the obvious issues which occurred but were unfortunately invisible to those at the time. Even if the issues were obvious to those involved, other biases such as optimism bias or outcome bias from previous events will have influenced their decision making processes.   Finally, conformity bias and the bystander effects are very powerful and so if no-one is able to break the silence, it will carry on regardless even if internally those taking part realise it is a mistake. (In this clip, someone does break the ‘silence’)

So, what is a Pre-Mortem and how can it help prevent future accidents?

As many project/programme managers and leaders will attest, it is normally very difficult to get to the truth about time/budget/production in progress meetings, even more so when the proejct isn’t going to plan. Steve Levitt (in the same podcast earlier) gives an example of a project whereby all seven of the key project managers gave a green light to the project within only 2 months to go saying they would complete on time and to budget, whereas those inside the company, the workers gave a 92% confidence it would fail. – the project failed!

Hardly a surprise given how often this happens in business. The problem is that when everyone is sitting at that meeting, looking at each other, with the boss asking those awkward questions, conformity bias means that no-one wants to stick their head above the parapet and be the bearer of bad news. They would rather just agree with the common view than question the status quo and hope they can resolve the issue before crunch day, or if not, hope someone else will carry the can.

Now consider the number of diving accident reports where it appears fairly obvious that it was going to end in disaster. Partly because information is now available after the fact, especially the impact of missing certain checks or diving with sub-optimal equipment or configuration.

If this is such a wide spread problem, how do we solve it?  One such way is to hold a pre-mortem! In effect, it is the discussion of the death of a project to find out why it died, with the fundamental difference that the analysis IS CARRIED OUT BEFORE THE PROJECT DIES OR THE ACCIDENT HAPPENS and so is not impacted by many of the biases ‘post-mortems’ are.

How to hold a Pre-Mortem.

The following is written from the perspective of a typical management-style project but the concept can easily apply to a diving expedition or serious dive.

At a point in time far enough back from the end of the project to realistically be able do something about the outcome of the Pre-Mortem, a meeting is called with the key stakeholders. This might be 6 months into a 12 month project, or 2 months before a major expedition sets off. Whilst the time is dependent on the specific scenario, the concept of what follows is the same.

The meeting starts with the facilitator, someone separate from the immediate outcome, describing the current position, stating that there are a number of clues that the project is heading for failure and the stakeholders need to recognise that unless something is done, the project is dead. They carry on and say that the stakeholders are to imagine it is now 4 months to go and these cues are getting stronger and there is a very high confidence the project will fail. It is now 2 months to go and there is no way that the project can be saved. Everyone is fighting over how to hide the bad news. At 1 month to go, there is limited communication between the stakeholders, no-one wants to discuss the project, just going through the motions, and looking for anyone else to blame for the failure. At project ‘go live’ date, the team is disbanded, the project is a complete failure with significant loss in revenue, reputation and personnel.  All through this commentary, the facilitator reinforces the failure of the project so that it is ‘real’ to the stakeholders.

At this point, the facilitator turns to the stakeholders and states they now have two minutes, in silence, to each write down as many reasons they can think of to explain how the project failed. Gary’s experience was that as the clock counted down, the writing intensified as people wanted to get their views into the discussion.  Most people can see where things are going wrong, but group dynamics prevents them from saying anything in a public format.

In his experience, pre-mortems identify the causes of the failures in the project that no-one wanted to talk about in a group situation because it would be seen as letting the team down and ‘breaking ranks’.  Furthermore, because the outcome is known (in the context of this situation), hindsight bias can be used as a positive tool when the ‘I told you this was going to happen’ comes to the fore.

The next stage once all the failure points are out is to go back to the stakeholders and say “what are you, personally, going to do to resolve this?” with regards to each point they have raised.  They have identified the failure path, so why not make them responsible for starting to resolve it?

In terms of diving, if you are diving as part of a team, by discussing the failure points and getting them into the open it makes it is easier to resolve issues, rather than relying on ‘hope’ as a means of getting past the issue. Further, if a number of people have spotted the same issue, then it is likely to be something that needs to be resolved as soon as possible and could have been a real risk during the dive.  If you are diving on your own, you could use the question “How would this all look if I ended up dying doing this?”

However, this isn’t simply a case of going through the motions, individuals and teams have to critically examine their plan, their equipment, their roles, their preparation, and be honest about the state of all of these. Only by being honest will the number of incidents and accidents be reduced.

4 responses to “Pre-Mortems – Looking at the Failure in a Critical Manner

  1. Like it, love it…. Risk Assessment is such a non-descriptive term and many times people are put of by the use of it! When I teach CCR Diving i train to the premise that “if it can hurt you it will”, and any mistake could be fatal…. So there is Zero room for error… Great piece!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s