Incompetent and Unaware: We Don’t Know What We Don’t Know

The simple premise is that we don’t know what we don’t know. Even worse, we don’t know that we don’t know!! This double hit in competency was highlighted by a couple of social researchers Dunning and Kruger when they undertook a series of experiments to assess what their subjects thought of their own knowledge. The exams covered logic, humour, and grammar and showed how prevalent the problem was.

The images below show the issue at hand. Pretty much everyone thought that they were above average (60-70% scores), but not exceptional. However, if you look at those who scored in the bottom quartile, the lowest scores were in the order of 12-15% even though they thought they scored 55-60%. The other noticeable part was that those who were well above average underestimated their abilities. However, those top quartile also couldn’t understand why those who were ‘incompetent’ couldn’t do the tasks at hand. They had excessive expectations of others, maybe fueled by the misplaced confidence in the others' abilities.

So what has this got to do with diving?

Those divers who are below the horizontal line have an over-inflated perception of their capabilities, and that includes me as the author. I will also have a perception of what is already known information!! It hits us all!! This over-inflated perception of knowledge and understanding means that we create models of the world which are not necessarily correct because they are based on our experiences. We have to have

This over-inflated perception of knowledge and understanding means that we create models of the world which are not necessarily correct because they are based on our experiences. We have to have experience to be able to develop credible but creative models of the ‘what ifs’ that we constantly face in diving. This mismatch between reality and the models is known as complacency.

Our brains have limited capacity to process information, actively this is around 7 +/- 2 pieces of information, passively it is in the order of several thousand bits of information, but our senses pick up millions of pieces of information! The brain sheds this information by filtering out what is not perceived to be relevant. Unfortunately, we don’t know what is relevant until after the event and something went wrong.

Therefore we have two massive biases which are working against us when trying to improve our behaviors. Outcome bias and hindsight bias.

Important: Watch this video to the end before moving down the page.

So, the wing suit flyer made it through the crack. Thoughts? Awesome experience? What if he had gone splat on the cliff face? What would have been your thoughts now? Probably fairly derogatory and negative. Now consider what happens when we read an accident or incident report from diving. How many times do we immediately jump to the conclusion that they were stupid? The problem is you know the outcome so you immediately think that they could have spotted what was going to happen and prevent it from happening.

Now watch this video. Every time there is a blank or fade think to yourself, what is going to happen.

Did you really think that that was going to happen? How many of you would have thought about checking the crucial piece of equipment? Now think about diving. How often do you check EVERYTHING that would prevent an accident or incident? How often do you analyze your gas? Every dive? What about pre-dive checks? Every dive? Have you ever completed a dive and found something that should have been checked and wasn’t? Don’t worry if you did, you are human!!

What you see is all there is…with hindsight we start to pick out information that is relevant because we know what the outcome is. At the time, you don’t know what is really important from the millions of pieces of information out there.

As Professor Sidney Dekker puts in his book Field Guide to Human Error (5th Ed) using these two brilliant cartoons.

To improve our decision making we need to have good situational awareness. Situational awareness is the ability to notice (not just see/hear/feel) what is going on around us, process that information to determine what it means to right now, and then project into the future. And the go around the loop around picking up changes.

This diagram is quite detailed but shows why experience [at the bottom] is so important. We need experience to determine relevance to the perception & processing stages. Beginners can see and understand something (SA Level 1 & 2) but are not able to project (SA Level 3). So when someone makes a mistake in a novel environment, it shouldn’t be unexpected. As long as they survive, they will probably learn from it and apply it to their next encounter. Therefore training has to be relevant and in the environment in which it will be used.

Checklists

An easy answer to limiting mistakes in aviation has been the application of checklists. These certainly have their place to improve situational awareness by ensuring that at the point in time immediately before an ‘operation’ is executed things are as they are supposed to be - they baseline the model. However, for checklists to be effective, they have to be easy to use, they need to be no more than 7-9 items in length, a team needs to be formed, and communication skills need to be employed to ensure two-way communication takes place. As Ken Catchpole, Professor of Human Factors Medical University of South Carolina stated in his paper 'The Problem with Checklists - “…a checklist reliant on teamwork for success may fail despite all the items being followed, because those team skills were insufficient."

Safety has been improved in aviation not directly because of checklists, but because checklists are part of a safety system where crews are taught leadership, communications and assertion skills, teamwork and the limitations of human performance. Checklists provide the structure for effective communication and decision making.

Decision Making

One of the reasons checklists can help is because of the way we make decisions. Daniel Kanheman in his book “Thinking. Fast and Slow” describes the two main ways our brains work.

System 1: fast, without apparent rationale, little or no effort and
System 2: logical, thoughtful, slow, effort required.

The problem is that for humans to operate at the pace we do, we need to operate primarily in the System 1 thinking zone. This means we take lots of short-cuts because that is efficient. The negative outcome occurs when we don’t have complete Situational Awareness (due to perceived irrelevance, distraction, experience etc) because we can make poor decisions as a consequence. Watch this video to see what I mean.

The problem with this decision making process in safety situations is that if the adverse event that could happen doesn’t, we create a new baseline as to what is ‘safe’, and not until something bad really does happen do we realize how far we have come from the original baseline. This is known as ‘normalization of deviance’. Unfortunately, humans are known to migrate to high risk situations and therefore it shouldn’t be unexpected when incidents happen or people break 'rules'. This is why checklists can help.

Given that we only get feedback when someone adverse happens, we don’t know where the unacceptable boundary is until we fall over the edge, so the best we can do is keep ourselves in the safe area with good teamwork which keeps us accountable, checklists to limit deviation, communications skills to challenge when things aren’t going to plan.

Using statements like “If you don’t follow this checklist (or safety message) you could die.” and then you don’t get hurt or a friend doesn’t die, you reduce the value or weight applied to that statement and you start drifting… If you open up an aircraft operating manual it doesn’t say on page one “Caution, not following the instructions in this manual could get you killed.” I understand the reasons for the including such statements (liability limiting) but it is unlikely to have the intended effect.

Safety is an emergent property which comes from valid risk perception and acceptance. However, you can’t perceive the credible risk unless you have experience, and you can only accept the risk you see. This is why experienced divers and engineers sometimes use statements like ‘they [meaning the user] have to accept the risk’ but if you are in the 4th quartile (top right of the graph at the top) you have a different (more complete) view of the risk compared to someone in the bottom left and therefore they are unable to understand the full risk they are accepting.

Moving Forward

I have mentioned many issues that make our diving less safe. What can do about it, especially if we are unable to see our own failures or what knowledge and skills we lack?

First off, we need to learn to talk honestly about failure. Failure is normal. That is how we ‘grow up’. But we can only talk about it if the community recognizes that we will make mistakes, even obvious ones!! If people open up and talk about how their incident or event occurred, ‘shouting’ at them for being stupid is going to achieve 3 things: they won’t talk about future events because they are scared, they may think they are stupid and can’t dive and leave the sport, others who are observing won’t report their incidents. Training cannot provide for every eventuality which means we have to learn from others’ mistakes and errors. In addition, we have to learn about human fallibility and the cognitive (mental) biases we suffer from. Hindsight and counterfactuals are great if you want to see what someone should have done, but it doesn’t help identity why it made sense for them at the time. The vast majority of people have an innate sense of self-preservation, therefore they don’t choose to do something that would end up with them dead. They make constant risk assessments and judgements based on previous experiences and behaviors of theirs, especially those in positions of authority. This is one reason role model behavior is so important.

Resources

Dekker’s videos on Just Culture - this short course by world expert Sidney Dekker should be considered essential watching for anyone involved in incidents (investigator, subject, or those reading about incidents)

For a reading list of additional information, please visit https://www.humanfactors.academy/pages/reading-list.

Human Factors Skills in Diving Online Class - Introduction to Human Error and the Human Factors Skills (Improved Situational Awareness, Decision Making, Communications, Leadership/Teamwork/Followership and effects of stress and fatigue). More than 2.5 hrs of theory with some practical exercises and detailed case study. 50% off discount code for Scubaboarders - SBHFSD2016-50 - valid until 31 May 16. Enter it on the right of the checkout screen.

Human Factors Skills in Diving Two-day Class - In-depth practical and theory classroom-based course using computer-based simulation to develop these skills in a practical manner. It is done in a classroom because it would be impossible to impart the same knowledge transfer in a diving environment in two-days. The interactions, communications, and stress levels are far more easily controlled in the classroom too! The class is currently aimed at Instructor Trainers, Instructors and those divers with an interest in Human Factors and Human Performance. Course dates can be found here: Details of the class dates are here

21 Responses

  1. Awesome article.  Some of the comments would be particularly beneficial for people participating in A & I threads!  Thanks
  2. Can you fix the second sentence? [quote]The simple premise is that we don’t know what we don’t know. Even worse, we don’t that we don’t know!![/quote]<br />-----<br />[FONT=Verdana]Ganbatte kudasai - Please do your best/Good Luck[/FONT] PADI IDC Staff Instructor PADI #324403
  3. Never mind, I thought there was a word missing. I apologize.<br />-----<br />[FONT=Verdana]Ganbatte kudasai - Please do your best/Good Luck[/FONT] PADI IDC Staff Instructor PADI #324403
  4. Gareth, this is brilliant and could be applied to lots of other areas besides diving - I'm thinking this would be great to show new graduate nurses.<br />-----<br />[URL="http://www.dukedivemedicine.org"]www.dukedivemedicine.org[/URL] [URL]http://hyperbaric.mc.duke.edu/[/URL] [I]Information provided is for educational purposes only, is not intended to replace the advice of your own health care practitioner, and should not be construed as a practitioner/patient relationship. Duke Dive Medicine does not condone the placement of "Skimwords" advertisements and does not endorse any of the products or services advertised. [/I]
  5. It may be my pettiness, but I love how you start that article by quoting Kruger and Dunning.<br />-----<br />Photography geek suffering from GAS and limited talent. [URL='https://www.flickr.com/photos/storker_moe/sets/72157630879703498/detail/'][COLOR=#0000ff]My underwater photos[/COLOR][/URL]. My club's [URL='http://www.pingvindykk.no/?page_id=345'][COLOR=#0000ff]webpage[/COLOR][/URL].
  6. Excellent.  Passed it on to our QA person.  I work in a hospital laboratory.
  7. Thanks for all the comments. These issues aren't just diving related, but applicable to so many walks of life. The healthcare area is a domain I am looking at too and have just finished two sessions with part of an ICU team looking to develop their teamwork and communications skills. Fundamentally, without a Just Culture we will not learn. Sidney Dekker (and many others) make it quite clear: there are two reasons to undertake an investigation after an adverse event - you can either look to blame someone, or you can look to learn from the event. Unfortunately the two are mutually exclusive!!If you want any more information, or have questions, just ask away! I love to help people!!RegardsGareth
  8. Useful article post. Here waiting new post from you and  Some of the comments would be particularly beneficial for people participating  Thanks in advance
  9. Nice article.  The principles like checklists and learning from failture with feedback is also useful for our normal  life and daily work.
  10. Nice article.The principles like checklists and learning from failture with feedback is also useful for our normal life.
  11. Nice article  Very nice indeed. I like where this is going...There is something else to wiggle into the conversation the future:  reasoning from deductive, inductive to abductive learning & reasoning models. Sure we learn, massively I might add, from failure analysis.  Human learning or repetitive adaptation comes through many forms.   In the video did the guy who merely turned because of the crowd actually learn something?  Or did we as the observer?  Field experience is great when managing failures or risky situations and especially team debriefing during success and failure are emphasized. I am passionate about conducting failure analysis beyond talking (arm chairing the situation) about but actually reheasrsing what went wrong and replaying if we would have responded accordingly (or not)?  Just my four cents worth...btw: this is good stuff!!Thanks for the opportunity to pipe up on a relevant subject.Peace.
  12. I wonder - what are [U][B]you[/B][/U] unaware of? In other words, why should I think that you have [U][B]ALL[/B][/U] the answers/knowledge? The title is obnoxious - Having not thought of absolutely every eventuality does [U][B]NOT[/B][/U] equate to incompetance. I prefer a competance that allows me to evaluate an unexpected situation in a calm state of mind and solve the problem.<br />-----<br />Abandon all hope ...
  13. Kharon, apologies if the title appears obnoxious. The title comes from a phrase within a research paper looking at this specific issue. Incompetent is taken in the technical sense, not the emotional sense. Competent - able to do something well. Incompetent - the opposite.I do not propose that I have ALL the knowledge - the point I was making was that in many cases there is a big difference between what is known and what is thought to be known. I am not sure if you read the article but if you look at the research paper I cited, the majority of people thought their level of competency across a number of different tasks was around the 65% make, but those who were incompetent (could not do the task) had much lower scores.  This is no different in diving, there are a significant number of inexperience and 'incompetent' divers out there who think they are above average...it is this mismatch that causes the problems.Your competence is a valid expectation, but if you look at Endsley's model of situational awareness and decision making, it does not work like that in the real world. Furthermore, if we have plenty of time to diagnose, create options, decide on a course of action, then it is unlikely to be a stressful situation. However, that decision making is still informed by previous experiences. See the bottom of Endsley's model.We can never learn everything that might go wrong, but if we understand our own limitations and learn from others, we are less likely to put ourselves into situations where we are unable to problem solve effectively in the time available to execute the mitigation.If you have more questions, please ask...
  14. [QUOTE="GLOC, post: 7703891, member: 148284"]... there are a significant number of inexperience and 'incompetent' divers out there who think they are above average...it is this mismatch that causes the problems.[/QUOTE] +++1 on this - it's this type of diver that is a danger to everyone in the water with them and one of the reasons I feel far safer diving solo. And thanks for the clarification. I was put off by the title. Guess I'll go back and read the [U]whole[/U] article. I only scanned the first few sentences - my bad.<br />-----<br />Abandon all hope ...
  15. Kharon, that is indeed a very conservative attitude but risk perception and acceptance are very much at a personal level. Your comment about solo vs team diving is interesting because it significantly changes how you go about incident prevention and problem resolution. A team is always more effective than a solo diver, unless we are talking about sump diving or places where two people cannot see or touch each other. The problem is that we are rarely taught how to be an effective team member, or how to communicate effectively, or a number of other situations. Therefore people have a bad experience and think that solo is safer than team diving. In terms of errors, pilots make in the order of 3-6 errors per hour! But they are trapped before they become critical. Making an error is part of human behaviour. The problem is we normally classify something as an error after the event once we know what the outcome was... As you can tell, I am very passionate about this subject. The world is not black and white like lawyers want us to believe, and the context and environment are incredibly important when it comes to looking at decision making. Regards<br />-----<br />
  16. [QUOTE="GLOC, post: 7718445, member: 148284"]... A team is always more effective than a solo diver, unless we are talking about sump diving or places where two people cannot see or touch each other. The problem is that we are rarely taught how to be an effective team member, or how to communicate effectively, or a number of other situations. Therefore people have a bad experience and think that solo is safer than team diving. ...[/QUOTE] You make my case for me: " [I][U][B]The problem is that we are rarely taught how to be an effective team member, or how to communicate effectively, or a number of other situations.[/B][/U][/I]" Putting my faith in an ineffective team is far more dangerous (for me) than diving solo - with way more than 100 solo dives, fully equipped to be completely self-sufficient, with the skill to use that equipment, and the cold knowledge that in my entire life I have never experienced panic - even in seriously life threatening situations. I'll take solo over diving with muppets every time. It's safer [U]for me[/U]. I enjoy the hell out of the freedom from babysitting, and I can focus completely on where I am and what I am doing without worring about what my "buddy" might or might not do.<br />-----<br />Abandon all hope ...
  17. [QUOTE="Grumpy Fish, post: 7864464, member: 482540"]Until the day you panic, alone, for an unforeseen reason and, die. If that's a risk you are comfortable with then that is your prerogative. A simpler safer solution would be to find a good buddy..... Or are you so good that no one is your equal? Safe diving[/QUOTE] I like the "we are all solo" attitude, not because no one is my equal, but because it kicks me out of any type of complacency that "I didn't know I was being complacent about"<br />-----<br />
  18. [QUOTE="Francesea, post: 7864471, member: 480055"]I like the "we are all solo" attitude, not because no one is my equal, but because it kicks me out of any type of complacency that "I didn't know I was being complacent about"[/QUOTE] Interesting comment. Whilst having a buddy introduces other cognitive biases such as assumptions that someone is going to doing something based on an assumed shared model (he is thinking what I am thinking, therefore I expect them to do something) it does mean that you have one brain and attention which can be succumbed rather than having two to spot the changes. As a consequence I am not sure what 'complacency' you are referring to here Francesca. [URL='https://www.humanfactors.academy/blog/complacency-the-silent-killer-but-it-s-not-that-simple']Complacency is normal[/URL]. It is normally attributed to an event where something negative happened when full attention wasn't paid and the assumption that the model of the world we have constructed behaved in the manner it was supposed to, but didn't. If it all went well, nothing adverse happened, and we achieved our goals more quickly, we often call that efficiency! Regards<br />-----<br />
  19. [QUOTE="GLOC, post: 7864731, member: 148284"]. As a consequence I am not sure what 'complacency' you are referring to here Francesca. Regards[/QUOTE] You don't, hmm? Well Neither do I, which is where the added phrase "did not know what I was being complacent about" comes in. I NEVER plan to go solo, that is not fun for me. But by looking at my dive setup as though I was solo, i realized there are some gear and habits I need to update. In addiction, next time I go diving I can rethink my approach to increase self sufficiency. Pulease don't ask me to confess my many deficiencies, and then say I should have been better to begin with. The point is that just looking at things as though I am on my own in various potential emergencies is a helpful boost of adrenaline plus dive budget elasticity to buy safety gear.<br />-----<br />
  20. [QUOTE="GLOC, post: 7718445, member: 148284"]Kharon, that is indeed a very conservative attitude but risk perception and acceptance are very much at a personal level. Your comment about solo vs team diving is interesting because it significantly changes how you go about incident prevention and problem resolution. A team is always more effective than a solo diver, ... The problem is that we are rarely taught how to be an effective team member, or how to communicate effectively, or a number of other situations. Therefore people have a bad experience and think that solo is safer than team diving. As you can tell, I am very passionate about this subject. The world is not black and white like lawyers want us to believe, and the context and environment are incredibly important when it comes to looking at decision making.[/QUOTE] Yep, Yep, Nope - eh maybe Yep with qualification, and me too. 1st: Yep, it's personal and I am very conservative. I intend to enjoy diving for many more years. Can't do that if I'm dead. 2nd: Solo does indeed change the paradigm re. prevention and resolution. You must be prepared to deal with anything on your own - or else die - which I am [U][B]not[/B][/U] prepared to do easily. 3rd: A team [U][B]can[/B][/U] be more effective, [U][B]but[/B][/U] your statement of the problem is spot on. Ain't many people that qualify. I no longer have access to any team person, that I can trust completely, diving at home, and certainly not on trips. On the boat dives I have taken I have been totally appalled at what the DM's allowed. Numbers of those people were putting my life at risk with being drunk/hungover, leaking regs, non-functional BC's, abysmal skills, totally lacking in situational awarness, and just doing really stupid stuff. And finally, I am passionate as well - about continuing to live. I love diving and I will suffer no muppets that could possibly kill me because of their own lack of skill or plain stupidity. I will never allow myself the attitude that I can depend on [U]anyone[/U] in the water beside [U]myself[/U]. If I get myself killed, so be it. At least a muppet didn't do me in.<br />-----<br />Abandon all hope ...
  21. Loved this article. I too believe that we need to talk more openly at when we screw up. You learn something, and passing on each "oops" just might help someone else avoid it! Or you can be a scientist like me and mess it up several times just to make sure it's relevant.<br />-----<br />
  22. [quote="Kharon, post: 7718582"][QUOTE="GLOC, post: 7718445, member: 148284"]... A team is always more effective than a solo diver, unless we are talking about sump diving or places where two people cannot see or touch each other. The problem is that we are rarely taught how to be an effective team member, or how to communicate effectively, or a number of other situations. Therefore people have a bad experience and think that solo is safer than team diving. ...[/QUOTE] You make my case for me: " [I][U][B]The problem is that we are rarely taught how to be an effective team member, or how to communicate effectively, or a number of other situations.[/B][/U][/I]" Putting my faith in an ineffective team is far more dangerous (for me) than diving solo - with way, way more than 100 solo dives under all kinds of conditions, fully equipped to be completely self-sufficient, with the skill to use that equipment, and the cold knowledge that in my entire life I have never experienced panic - even in seriously life threatening situations. I'll take solo over diving with muppets every time. It's safer [U]for me[/U]. I enjoy the hell out of the freedom from babysitting, and I can focus completely on where I am and what I am doing without worring about what my "buddy" might or might not do.[/quote]Until the day you panic, alone, for an unforeseen reason and, die. If that's a risk you are comfortable with then that is your prerogative. A simpler safer solution would be to find a good buddy..... Or are you so good that no one is your equal? Safe diving<br />-----<br />
  23. [QUOTE="Kharon, post: 7704529, member: 409808"]+++1 on this - it's this type of diver that is a danger to everyone in the water with them and one of the reasons I feel far safer diving solo. And thanks for the clarification. I was put off by the title. Guess I'll go back and read the [U]whole[/U] article. I only scanned the first few sentences - my bad. A few minutes later: OK - read the whole article. Good stuff and it gave me insight into why I do things the way I do. Some examples (but not exhaustive): no one but me touches my equipment, if someone "helps" me gearing up I break everything down and start over, my log sheet has a place for any errors and for gear needing attention, I dive very conservatively, and I rigidly follow my "rules": [LIST=1] [*][B]You are [/B][U][B]always[/B][/U][B] diving solo[/B] no matter how many divers are in the water, or how many buddies you have, or how much experience they have or what you discussed during the dive plan. [*][B]It is always absolutely necessary to take [/B][U][B]at least one[/B][/U][B] bearing before every dive.[/B] [*][B]If you forget something non-trivial during gear up that dive is aborted.[/B] If you aren’t focused enough to gear up perfectly you aren’t focused enough to dive solo or otherwise. [*][B]If you forget something non-trivial during gear up on two dives the diving day is over.[/B] Two mistakes in one day means you are way off and need a time out. [*][B]If you make a non-trivial judgment error on a dive at least the next dive is skipped.[/B] You need to think about it. [*][B]If you make two non-trivial judgment errors in a days diving the next days dives are skipped.[/B] You [U]really[/U] need to think about it. Note: non-trivial includes pretty much everything more serious than mistaking a coronet fish for a trumpet fish. [/LIST] [/QUOTE] Number 2 is particularly amusing. I have been on dives where yo9u are so far of shore that all you see is the horizon. What are you taking a bearing on then? hope?<br />-----<br />
  24. [QUOTE="Grumpy Fish, post: 8041295, member: 482540"]Number 2 is particularly amusing. I have been on dives where yo9u are so far of shore that all you see is the horizon. What are you taking a bearing on then? hope? if you have no team you can rely on perhaps you should go about forming one around you. You seem to have a pretty high opinion of your own skill ( which in itself, in diving can be very dangerous) so perhaps you should for a team of similarly minded divers and train them to your standard. Personally I train very hard and pick my buddies diligently making sure that are all my equals or better ( usually the latter as I am always trying to learn something from every dive), you sir, are exactly the sort of diver I would always gracefully decline from partnering up with.[/QUOTE] Added to Ignore list.<br />-----<br />Abandon all hope ...
  25. [QUOTE="Kharon, post: 8041777, member: 409808"]Added to Ignore list.[/QUOTE] Great so I don't have to read your silliness any longer<br />-----<br />

Leave a comment