Risk Management as a decision issue

The term “risk management” has been around for a long time in financial, technical and medical practice. It is a term that is very loosely used and I want to dive in with a decision-centric view just to further muddy the waters.

A good place to begin is with a formal definition of “risk”. If you enter “risk definition” into Google you will get over twenty-five definitions; some are redundant, and there is little consistency. Regardless of the definition, risk traditionally amounts to answering three questions:

1. What can go wrong? Some event.
2. How likely is it to happen? Probability that the event happens based past statistics, an analytical model of the event, or best guess based on experience.
3. What are the consequences? Money, time, and possibly even lives are wasted.

The above set of questions describes event risk. They are the basis of technical risk evaluation. However, most managers are concerned with decision risk rather than event risk. For decision risk, the questions managers really want answered are:

1. What can go wrong if I choose alternative X?
2. How likely is it?
3. What is the impact?
   

(Brian Seitz of Microsoft articulated these as cleanly as shown here)

Note that these are the same three questions as with event risk, just slightly tweaked. Regardless of whether focused on an event or a decision, risk management, by definition needs to be effort directed at some or all of these questions.

The first question raises the issue of how we know our choice was poor. In the book Why Decisions Fail, the definition of a poor decision is one that had no positive impact after two years. There are other measures of a failed decision. For example, our time is up and no satisfactory alternative has been developed. Or not everyone on the team agrees with the choice made and some team members feel disenfranchised. More often, the result of a poor choice is not known until much later. For example, if we choose a bad restaurant, we will not know until we have eaten there. Or if an engineer makes a poor decision on what material to use for a product, this may not be evident until the customers have used the product for a number of years.

One thing is consistent in this discussion: Without uncertainty there is no risk. A corollary is that the more uncertainty, the higher the risk of making a poor decision. “What can go wrong?” is that one or more of the criteria are not satisfied. “How likely is it?” is directly dependent on our certainty during the alternative evaluation. We may know from past experience or data that the probability of something failing is XX%. But, this probability may be compounded by other uncertainties such as lack of knowledge, disagreement amongst team members or incomplete data. And “What is the impact?” is that the alternative chosen no longer is as good as it was originally thought.

In order to manage risk during decision making:

• You must address risk during decision making, not as a task to complete after you have selected an alternative. This is because decision risk is a measure of your lack of knowledge, as well as other uncertainties you need to consider when you make a decision.
• There is risk associated with every feature of every alternative. Traditional risk assessment separately addresses financial risk, performance risk, and schedule risk. But when including risk as a part of the decision-making process, you must integrate the uncertainty inherent in all the features at once, because it is the combination of them that drives your decision.
• You can get a good assessment of uncertainty, and thus risk, by fusing the evaluations of all the members of your team.

Event Risk vs Decision Risk

I was just gave a presentation at the IG's (Inspector General's) office. They are concerned about the risks involved when they make decisions. What they don't realize is that there are two kinds of risk they have to worry about, event risk and decision risk.

Event risk is what most people mean they talk about "risk". It is the expected value of an event, its undesirable consequences and probability of its occurrence. Determining risk amounts to answering:

1. What can go wrong? –An event occurs that may have bad consequences
2. How likely is it? – Probability dependent on past statistics and model results
3. What are the consequences? –Money, time and possibly lives are wasted

NASA and others have entire handbooks on assessing event risk.

During decision-making, risks are inherent in uncertain knowledge, information and models. Uncertainty creates the risk that a poor decision will be made. This doesn't say that the alternative chosen will fail, that is even risk. Drawing analogy to event risk, decision risk focuses on:

1. What can go wrong? – A poor choice is made
2. How likely is it? – Probability dependent on uncertain knowledge, and the fusion of the team’s interpretation of information and models
3. What are the consequences? –Money, time and possibly lives are wasted

One problem the IG wants to address is selecting new employees. Clearly the risk here is a decision risk - they want to ensure that they don't make a poor hiring choice. They also want to manage their portfolio of projects. Here the risk that the project can go wrong affects the risk that they make a poor decision. The higher the event risk associated with an option, the higher the decision risk may be.

Both types of risk are based on probabilities. However, traditional probability methods (often called frequentist methods) are good fro event risk, but are not capable of managing knowledge uncertainty. Rather, Bayesian probability methods are specifically designed to integrate accumulating, uncertain, incomplete and conflicting knowledge.

Can I convince the IG folks of this? We will see.