7 Popular Methods for Making Decisions (None of which are advised)

Organizations use many different strategies to make decisions. The following is a list of strategies seen in practice. They all lead to decisions, but they all run the risk of not making good use of the available people and information. Have you ever experienced these?

1. Decision by Running Out of Time: This is the most common form of decision-making. There may be some effort to develop criteria and alternatives, but often time runs out before there is any effort to ensure that a robust decision is made.

2. Decision by chaos: The president of the company says, “I want our new product at the Atlanta trade show in two weeks”. The show is in two weeks. There is no rational way to prepare for the show and make robust decisions. The decisions made in the chaos may need revisiting after the show and work will need to be redone. Some people prefer to work in a chaotic environment, and when in positions of power, they will manufacture chaos as the working environment.

3. Decision by Fiat (or Decision by Authority): This is a very common style in autocratic organizations where a manager or someone else in authority decrees that a certain alternative is his/her favorite. It is often seen when the boss’s idea is chosen in order to preserve the relationship with him/her. This is more justification than decision-making.

4. Decision by Coercion: A champion for one alternative pressures his/her colleagues into submission. Often the loudest voice wins, the others having given up. One colleague referred to this style as “hijacking the process.

5. Decision by Competition: Here concern for who wins is most important, as instanced in most sports. This is often a win-lose situation and the relationship among individual team members is not important.

6. Decision by Voting: Democracy works, but does not often make the best possible choice. This decision making process is a weak form of compromise. Think how most products and businesses would operate if they were designed the same way we elect a president.

7. Decision by Inertia: This style is based on “We did it that way before” which may result in a robust decision, if the previous one was. But, not much progress or innovation is made using this style. Sometimes the tough decision is knowing when to innovate, and when to keep the cruise control on.


I have seen these all. Do you know of other dysfunctional decision making practices, ones that you have experienced?

What can you do to defuse these styles? The first step, like any good program is to realize that there is a problem and that put a name to it. Hopefully, this list can get you through that step. The second is to get general agreement that decision-making is a process. But, both of these steps may be hard as some managers don't want any rationality in the process. There are many ideas in my other blogs and on my web site. Do you have ways of managing these?

Decision-making maturity

I just spent a day judging a contest of pre-teens making design decisions. I learned a lot about decision-making. My day was in support of the US First Lego League (FLL) local competition. The theme this year’s event was “Climate Connections Challenge”. As part of the event the 10 -14 year-old students built robots out of Legos to compete in a simulated world. They also had to make a presentation on climate change and their community. Finally, they prove they could work as a team.

This year I was the chief Teamwork judge. To show teamwork each team (between 4 and 8 students) was given a simple task and 5 minutes to solve it. On entering the room, I welcomed them and had them gather around a table. I then put on the table a thin plywood disk and a small bag of large sized paper clips. I read them directions which were something like: “You are to build a structure that will hold the disk off the table. You can bend the paper clips in any form you like. The paper clips must be connected in some way. You can not use the plastic bag. You can not harm the plywood disk. You have 5 minutes. Go!”

Two other judges and I then observed them solving the problem. We were not interested in the solution, but in the team-work displayed during the solution. We had rubrics to guide our assessment and had time to ask questions at the end.

It dawned on me part way through the day that I had 21 samples of conceptual design decision-making by pre-teens to observe. This became clear to me too late to allow me to take formal data, but I did make the following anecdotal observations. In each, I juxtapose them to what I have observed in adult decision makers.

• As soon as I stopped reading, the kids grabbed the clips and started bending and talking at the same time. One team was so eager I feared for my safety as I jumped out of the way (slight exaggeration). There was no planning or reflection by any of the teams! I did an experiment in the late 1980s where I gave 6 professional engineers a design problem and five hours to work it. I video taped the sessions. Most of these mature engineers read the requirement over several times before beginning to generate ideas. One subject however dove right in like these pre-teens. He pursued his first and only idea and proceeded to patch on it (see next item), never leading to a reasonable solution of the problem. My conclusion from these experiences is that it takes maturity to do the up front work necessary to make decisions.
• Once the kids jumped in, they patched their way to a solution. Patching means to try different ideas until you stumble onto a solution (I discuss this further in The Mechanical Design Process). What is bad about patching is that it is usually random with no structured method to explore the design space. I saw the same with the engineer described above and other immature designers.
• When they asked clarification questions we only responded with “Read the problem description”. Few did (less than 1/3). They took our response to mean “No”. to whatever was asked. I was surprised at this and after the first few times, I made a big deal of laying down the problem description (in big font on green paper) in the same location as the wooden disk and paper clips. Only one team (out of 21) reread the description together to clarify their issue after we prompted them. My generalization of this is that, for the most part people don’t use all the information available to them.
• On the whole most of the teams did a good job of including most of the members on their team. This is a credit to FLL and the mentors who worked with the teams. I have had many college level teams that were not as inclusive.

By the way, there were three classes of solutions, 1) bend the clips so they sat on the table and the disk was supported by upward extending wire legs (most did this with great patching), 2) bend the clips so that they fastened onto the disk with downward extending legs (only saw this done once successfully), and 3) lay two clips flat on the table and set the disk on top. The last was the most elegant and simple. One team discovered this 40 sec into the five minutes. To see more teamwork, I quickly said “Great, now make the support as tall as possible” (whew!). Most teams never found this solution, but with their dive right in approach and random patching, I am not surprised.

Decision Thinking – its time is coming

I was trained as a mechanical engineer. I know how to model systems, take data and develop an understanding for physical things. Most (nearly all) technical university courses are about how to analyze things. In my case these were physical things, but my education could have been in any engineering discipline, in business or in the sciences, and the courses still would have been primarily “thing” focused.

In the 1980s I was teaching mechanical engineering design at a university and began to appreciate that things come into being through a process. My thinking moved from being thing focused to being process focused. Process thinking was not new to some fields. In fact, in engineering there are control processes, chemical processes, fluid thermal processes etc. But these are nothing but “process” things. What I became interested in was the process of developing these things.

In 1990 I began work on a text book for mechanical engineers so that they could study the process of how things mature from need to a final, working object. I agonized over the title as the term “process” was not really a part of the engineering lexicon except for process things. I finally chose the title “The Mechanical Design Process” and the book was published in 1992. This title turned out to be a good choice and the 4th edition of the book is out in January 2009.

When I began to research the design process in about 1984, my goal was to understand how objects evolve with eye toward developing methods and tools to better support this evolution. I don’t mean CAD and solid modeling tools as these are representation tools for what is being designed, not tools that actually support the design process. In fact, I have argued in a paper I wrote in 1990 that CAD can be detrimental to the design process ( Ullman and Wood, 1990). Since writing this paper, CAD has evolved into solid modeling which is much better, but the arguments in the paper still hold.

Anyway, I was especially interested in developing a tool that could record the evolution and rationale for product evolution. This design rationale system would be able to capture how a product came into being and could be reused, queried and vetted to form a permanent record of its birth, life and death. When I began, I focused on the evolution of the assemblies, parts, and features. I quickly realized that this wouldn’t work because these things evolve during the design process and thus, focusing on them missed all the birthing – all the interesting, creative engineering.

My thinking turned to capturing the process through which things evolved. Process thinking encapsulates thing thinking as the process is about the evolving things. This shift in my thinking coincided with the text book mentioned above. However, it became evident by the mid 90s that process thinking, although much better than thing thinking, was not the best way to develop a design rationale system. What became evident was that decisions are the punctuation marks in the process and that my approach had to make yet another shift, one to decision thinking.

Thus, by the late 1990s my thinking had matured from thing, to process, to the decisions made during the process to develop things. Specifically, I wanted to understand and support the decision-making process. My research showed that on macro level, decisions were made at gates (in a stage-gate process) a countably few times during the evolution of a product. On the micro level – the cognitive level - they occur about 1 decision per minute (see Stauffer,and Ullman 1991). Somewhere between these extremes there is much need for decision making support.

Before we go on, a definition of decision thinking - decision thinking is focusing on the decision-making process used during technical or business development. “Focusing” implies understanding and supporting individual decision makers and teams of people making decisions when information is incomplete, evolving and conflicting so that the decisions are robust.

I am not alone in this evolution in thinking. First, when I chose the tile “The Mechanical Design Process”, the word “process” was problematic as it was not commonly used in industry. Now, the product development industry makes good use of process thinking. Then, when I started talking about decision thinking in the late 1990s few in industry knew what I was talking about. Now there is much evidence that companies and the government are beginning to realize the importance of decision-making in their processes. My evidence for this is not firm, but many of my contacts seem to understand what I am talking about and few did five years ago, and the number of hits on the Robust Decisions web site continues to climb. Their thinking is maturing through the process to the decisions made during the process.

Second, the CAD industry matured into PLM (Product Lifecycle Management) during the 1990s. Where CAD and solid modeling is about things, PLM is about processes that manage things and document things. I have tried to interest the PLM vendors in decision thinking for about eight years. Initially they told me there was no customer pull (see the previous paragraph). Recently, I have gotten their attention. Now that they have the process under control, they too are maturing toward decision thinking.

Ullman, D.G., S. Wood, D. Craig, "The Importance of Drawing in the Mechanical Design Process," Computers and Graphics, Special Issue on Features and Geometric Reasoning, Vol. 14, No. 2, 1990, pp. 263-274.

Stauffer, L.A., D.G. Ullman, "Fundamental Processes of Mechanical Designers Based on Empirical Data," Journal of Engineering Design, Vol. 2, No. 2, 1991, pp. 113-126.

Decision making maturity

I just returned from an ASME (American Society of Mechanical Engineers) meeting in NYC. During my time there I discussed the concept of decision making maturity with three different groups and thought it worth writing about. Best to do in context with my career.

When I was trained as an engineer, I focused on how components and assemblies were shaped, manufactured and functioned. This element-centric view of the world is not at all unique to engineers. Businesses focus on documents, e.g. POs, recepts, memos, contracts, etc)

By the 1980s I became fascinated with the process of developing the elements. This process-centric view is recent. Sure engineers have studied and developed chemical and manufacturing processes for years, but the concept of product design processes and business processes is recent. As evidence of this, in 1990, I wrote an engineering text book about how to progress from customer need to produced product. I debated long and hard about what to title it. I finally landed on "The Mechanical Design Process", a title that proved to be right on the mark (note that its 4th edition will be out in January 2009). The use of the word "process" in the title was problematic because it was only beginning to be used to discuss product evolution.

In the early 1990s my research was about how to capture and manage the evolution of products, the rationale for form and function. This was process-centric, but not getting anywhere. In about 1995, it dawned on me that "design is the evolution of information, punctuated by decisions". That started me on a decision-centric view of the business and technical worlds.

My maturity from element through process to decision is being taken by industry. While in NYC I met with a PLM (Product Lifecycle Management) guru. PLM grew out of CAD (Computer Aided Design) which was all about parts and assemblies - element-centric. PLmis currently focused on the process, i.e. the lifecycle of products. I have been trying to sell decision-centric capabilities into PLM since 2001 with no success. The push back has always been "we aren't ready for that yet" and they weren't. Now there is the beginning of interest. The product development industry is maturing through process to realize that business and technology progress is punctuated by decisions and it is the quality of those decisions that determine the product and business success.

Further, when I talked about this decision-centric view of the world five years ago to industires, audiences had no idea what I was talking about. Now I get good awareness and it is building. There is yet hope for "evolution punctuated by decisions".

Most estimates suck

The rebuilding of the World Trade Center (WTC) is behind schedule and over budget. According to the executive director of the Port Authority, Christopher Ward "The schedule and cost estimates for the rebuilding effort that have been communicated to the public are not realistic," (from AP article). I can only say "Duh!"

All decisions are based on estimates of the murky future. In most public works projects honest estimates would never get them funded. It is much easier to get something approved with optimistic time and cost estimates and then, once its started ask for more. However, even if you are not playing politics, making estimates is hard work that is not well recognized for its impact of organizations

Let's get this straight. Even if you are trying to be as accurate and honest as possible, making an estimate is a highly uncertain undertaking. And, most decisions are based on many, interdependent estimates. In a simple experiment, I asked hundreds of people to estimate the length of time needed to clean some dishes. I gave them a detailed list of the dishes, and a photo of them, the sink, and cleaning materials. I asked them to estimate how long to clean the dishes. Depending on how I worded the question, the average time estimated was anywhere from 17 to 32 minutes and standard deviation was as high 10 minutes. In other words, asking for an estimate for this simple, daily task isn't a whole lot better than using a random number generator set to give an estimate in the range of 10 -40 min.

If this simple estimate is so bad, think of what happens with complexity, or for tasks that have not been done before.

What is to be done then. The only solution I know is to use methods that account for uncertainty. For planning , the old PERT system made an effort at this (much improved method is Critical Chain). For decision making, I have been pushing people for years to include uncertainty in their decisions making process.

In decision making, it is not only the estimates that are uncertain, it is the targets that you are trying to achieve that are uncertain. More on this in a later blog.

Back to the World Trade Center, I believe that if there had been a non-political effort at making decisions, and uncertainty had been included in the estimates, the plan would have been far less grandiose. However, the WTC should be grandiose, shouldn't it? This leads me to believe that honest estimates may not be the bast for all situations. I would hate to run or work for a business with that in mind.

Cognitive problems in making decisions

I have been reading "How Doctors Think" by Jerome Groopman. It is one of two books with the same title. Basically, reading this will make you scared to go to the doctor as doctors are human and make lousy decisions, just like all the rest of us. He has a good chapter titled "The Uncertainty of the Expert" which focuses on my favorite topic, decision making with uncertain information. All of Groopman's comments come from cognitive psychology and apply to business and technical decisions as well.

The key point in the chapter is captured in one sentence about half way in: "Physicians, like everyone else, display certain psychological characteristics when they act in the face of uncertainty". Uncertainty arises from 1) incomplete mastery of the available knowledge, 2) limitations of the knowledge, 3) difficulty distinguishing between the 1 and 2, and 4) (he leaves this one out) the variability of the nature of things.

In the face of uncertainty we all tend to:

1. Focus on the positive rather than the negative (except engineers who are pessimists by nature).
2. Ignore uncertainty. This is very evident in how we all talk about technical issue in terms of single values. Luckily we have terms like "about", "near to"
3. Go with what's been done before even if it is based on an unknown and unproven orthodoxy. Of course risk aversion is good and in medicine often leads to the correct diagnosis, whenever the problem is by-the-books. However, in business, technology this aversion can lead to being swamped by the competition.
4. Have a confirmation bias. This means that we look for support for our favorite alternative or hypothesis, at the expense of work on other possible options and discounting the negative (see item 1).

In the medical profession these characteristics are supported by the lack of time, risk aversion and an old-boys-club attitude. It is interesting watching the decision making on the TV program House. Here an arrogant, troubled doctor who hates the establishment diagnoses rare conditions and displays items 1, 2, and 3. He is really good at #3. But, most of us are.

Bayesian Brain and Bayesian Teams

An interesting article "Is This a Unified Theory of the Brain?" appeared in the New Scientist (full text) on May 28th. This is basically a discussion of the work and theories of neuroscientist Karl Friston of the University College in London. His work is based on an earlier theories that the brain makes decisions by trying to make sense out of the uncertainties in the outside world. Basically, you make hypotheses about reality and compare sensory inputs to them, updating the hypotheses and the belief in them as you gather more information. In essence you are constantly updating the probabilities that the hypotheses are true. In fact the wiring in the brain is continuously changing to is suppress prediction errors.

What makes this really fascinating is that 1) this is Bayesian updating and 2) this applies to team decision making also. The first observation has spawned the Bayesian Brain camp of neuroscientists. They believe we are all Bayesian thinkers, updating the probabilities that your hypotheses are correct as you solve small problems (e.g. if I turn the knob, the door will open) and large ones (e.g. If I choose to study the Bayesian brain, I will better understand team decision-making).

This brings us to applying what the neuroscientists are doing on the individual, to what happens in a team making business or technical decisions. These decisions are generally about what courses of action to take to address a current situation (obscured by its immediacy) or the future (clouded by uncertainty). A team that is functioning well will develop alternative courses of action or hypotheses and then gather an communicate information to increase its belief that one is better than the other, or to update the options based on the new information. No different than a single brain, just much more complicated.

What makes it harder is not only the communication of information amongst the team members (a clear focus of Information Management, Business Intelligence and the Webex's of the world), but developing a shared vision of the information. This is not to imply that everyone needs to understand all the information, but that there is some common understanding of the important bits. This is a topic I beat on in Chapter 4 of Making Robust Decisions "Team Don't Make Decisions, But...."

When we were first developing Accord software, an effort to support the team decision making process, we assumed that we could help teams by making what occurs inside one person's head transparent for the entire team. Maybe we should become neuro-scioscientists and study "Bayesian Team Dynamics".

Decision Making for KM and BI

A June 10th article "Knowledge Management and Business Intelligence" tries to tease apart KM and BI. In the piece the author, Richard Herschel, refers to Gartner's definitions of the two terms:

• BI= a set of all technologies that gather and analyze data to improve decision making
• KM = a systematic process of finding, selecting, organizing, distilling and presenting information in a way that improves an employee's comprehension in a specific area of interest Specific knowledge management activities help focus the organization on acquiring, storing and utilizing knowledge for such things as problem solving, dynamic learning, strategic planning and decision making.

What has always amazed me about these and other fields such as Analysis of Alternatives (AOA), Data Mining and even most Decision Support Systems (DSS) is that they are all about developing and managing information to support and improve decision making, yet none of them actually support the decision making process.

The decision making process requires 1) framing the problem, 2) evaluating the alternatives, 3)fusing the evaluation and 4) deciding what to do next. All of the methods listed above help gather and organize information that is vital to good decisions, but all stop short. Decision making requires more than the availability of database information. In fact, as mentioned in the article "up to 80% of business information is not quantitative or structured in a way that can be captured in a relational database". This non-quantitative information is the basis on which many business and technical decisions are made.

Two cases support this. First, a manufacturer of rocket engines uses our Accord software because many of their key early decisions are qualitative and Accord can manage fusing the qualitative and quantitative evaluations. Second, a Fortune 100 company received 20 proposals in response to an RFP for a piece of electronic equipment. In the RFP were over 60 quantitative specifications for size, functionality, reliability, etc. When they reviewed the proposals, they sorted them into two piles, those that met the specs and those that didn't. Then they began to use the unstated, usually qualitative measures to differentiate the acceptable proposals so they could decide how to award the contract.

The point is, most best practices focus on generating and managing information so decisions can be made. They don't spend enough effort on framing, fusing and managing what to do next. Framing and evaluation fusion are the social interactions occur that develop buy-in, accountability and robust decisions. It is these areas that are the hard part, that are not covered in school and are not well supported.

Stuck in 'Analysis Paralysis'?

Unresolved decisions can be very damaging to an organization. You know something needs to be done but can't decide what to do. Deliberation continues among your staff without an actionable conclusion. You have 'analysis paralysis'. Your operation is not moving forward, but you're using resources and your competition is not standing still.

Symptoms of 'Stuck' decision making include the following:

• Holding more than 3 meetings on a single issue
• Continually gathering more information
• Endlessly discussing and deliberating but not coming to a conclusion

'Stuck' deliberation is often resolved at the twelfth hour and by a manager who may be forced to act arbitrarily, if not irrationally. For any organization this is a highly risky way of operating. Here are some pointers for avoiding analysis paralysis:

1. Build a Shared Understanding: Information about the alternatives and criteria in a decision are understood only from each team member's own perspective. Team members may not have a complete picture of the situation, nor the knowledge or time to develop a full, long-term view. The key here is to set up environments that support the sharing of pertinent information needed for the decision.


2. Work to separate Goals from Importance: Separate what is to be achieved (i.e. goals, targets) from how important it is to achieve it. It is easier to agree on goals than what is important.Evaluation uncertainty may swamp out the differences in importance, but only if this goal/importance separation is made explicit.


3. Acknowledge and Manage Uncertainty: The future is always uncertain. The extent of this uncertainty needs to be identified as objectively as possible during decision-making. Engineers and financial analysts in particular are prone to giving single, deterministic values for information that is really a distribution. Push back on them to find the distribution, even if it is in terms like "very sure", "about" or "sort-of". Early in the development of a system all estimates are uncertain and need to be managed as such.Information that is highly uncertain needs to be discounted or its uncertainty reduced if-and-only-if it is important (see item 2)


4. Develop Multiple Alternatives: Always consider multiple courses of action that can be itemized. If the choice is to do A or "do nothing", then make an effort to develop alternatives B and C. Develop methods within your organization that encourage creative options.Find ways to help the champions of each idea compare and contrast their alternatives with others.


5. Define a Decision-making Strategy: Make sure there is an agreed to decision-making strategy. Decision-making by stirring and restirring existing information is not beneficial.


6. Build collaboration: Collaboration means that all stakeholders' opinions are heard during deliberation. Then, even those whose first choice is not chosen will more likely buy in to the outcome. This also gives the whole team a sense of accountability for the final decision.


7. Be Aware of Diminishing Analytical Returns: Analysis is expensive and is likely to postpone resolution. Over-analysis is the risk-averse activity of trying to drive out all uncertainty. When the fidelity of simulation is greater than the uncertainty of the information on which the simulation is based, time and money are being wasted.


8. Reuse History: Work toward learning from past decisions. Evaluating your success requires keeping track of past choices; the actions as well as the results.


9. Find a Platform to manage and fuse uncertain team evaluations: Use proven methods and tools that help your organization reduce risk and avoid deliberative quagmires.

A Justification is Not a Decision

There are two kinds of decision-making: justification and selection. Justification occurs when the result is a foregone conclusion — the choice is made in advance of any argument or consideration. This often happens when the boss already has his favorite option in mind and wants information to "prove" that it is the right choice. I have seen this during my career in product design and the nation saw it in President Bush's decisions about Iraq. In Maureen Dowd's NY times Op-Ed on June 1 2008, "Cult of Deception" she says that "our president is a one-man refutation of Malcolm Gladwell's best seller Blink, about the value of trusting your gut."

In an earlier blog, I discussed Blink and how trusting your gut can be the wrong approach for complex decisions, because you can't include sufficient information or study alternative courses of action. However, managers that get that warm fuzzy feeling when they know the best course of action, before they have sufficient information, can be dangerous. Of course, they may be right. If they usually are, then they clearly have sufficient information and a good decision making style. But if these justifications often end up with later fire-fighting, then justification is not working in lieu of decision-making.

Actually, the CIA has a method that is supposed to short circuit justification-thinking. It is called Analysis of Competing Hypotheses and is in a downloadable book titled "Psychology of Intelligence Analysis." Basically, ACH prescribes the following steps:

1. Identify the possible hypotheses to be considered.
2. List the significant evidence and assumptions for and against each hypothesis.
3. Draw tentative conclusions about the relative likelihood of each hypothesis.
4. Analyze sensitivity of the conclusion to critical items of evidence.
5. Identify future observations that would confirm one of the hypotheses or eliminate others.

This differs some from the process I develop in Making Robust Decisions, but both begin with basic fact that — YOU MUST HAVE ALTERNATIVES TO MAKE A DECISION. I put this in bold because it seems evident that many Washington decision makers don't follow this basic truth. In fact, many business leaders don't follow this either. In the book Why Decisions Fail, Paul Nutt gives three basic blunders. One of these, "Premature Commitments" is the the same as the justifications we are discussing here.

So what do you do to stop this kind of ineffective decision-making. The only thing to do is to set up an environment that forces multiple decisions to be considered. Sometimes this is difficult from below, but if you are a manager, insist on multiple alternatives to consider.