I want to compare two approaches to hypothesis-driven thinking. The first is in a paper by Jeanne Liedtka, of the University of Virginia, Darden School of Business, "Using Hypothesis-Driven Thinking in Strategy Consulting". The second is Richards Heuer's work for the CIA called Analysis of Competing Hypotheses (ACH) (Chapter 8 of his book "The Psychology of Intelligence Analysis.")
Liedtka tunes her process to consulting and states "The traditional decision-making process that we are most familiar with in business involves a linear method of thinking in which the problem is defined, a comprehensive range of alternative solutions is generated and evaluated, and the optimal one is selected. In contrast, the hypothesis driven approach, associated with the scientific method selects the most promising hypothetical solution early in the process and seeks to confirm or refute it" She goes on to out line the following steps:
- Define the problem
- Gather preliminary data that allow construction of initial hypotheses about the causes of the problem
- Develop a set of competing hypotheses about the causes
- Select the most promising
- Identify analyses to study these
- Collect data and test the hypotheses
- Reformulate as needed
Heuer on the other hand suggests that you should:
- Identify possible hypotheses
- Make a list of significant evidence for/against
- Prepare a Hypothesis X Evidence matrix
- Refine matrix. Delete evidence and arguments that have no diagnosticity
- Draw tentative conclusions about relative likelihoods. Try to disprove hypotheses
- Analyze sensitivity to critical evidential items
- Report conclusions.Identify milestones for future observations
These two processes are very similar. Where Liedtka, in the quotation beats up on developing multiple hypotheses, her method encourages more than one to be tested at a time. Heuer, on the other hand begins with "identify multiple hypotheses". He never states that this list needs to be complete, just that there needs to be more than one.
My research into how people develop products has shown, that designers that only develop a single alternative for a problem usually get into trouble because they have nothing to compare their efforts to and no secondary course of action, if the first idea hits a road block. When hypothesis driven, it is somewhat the same. You can develop a single hypotheses and test it. If it fails, you can develop a second one, maybe even using some of the information developed during the first effort. However, this is fragile as you may be headed in the wrong direction.
A better approach is more like what Heuer suggests. Develop multiple hypotheses and then use measures (i.e evidence) to try to disprove them. Heuer never says that the the list of hypotheses needs to be extensive or complete, just multiple.
One additional thought- as evidence is generally uncertain then you just take great care in using it as sufficient to eliminate a hypothesis. This is why multiple evidence is generally best. The good news is this is the way we naturally operate. For example, I was approached last week by a man with a business idea close to something I was already interest in. My initial hypotheses about him (in retrospect) were, 1) he had a good idea, was credible and I should invest time with him, and 2) he was a flake and I should waste little time with him. I began with hypothesis #1 and began to collect evidence by talking with him. After a reasonable time, I decided he passed the "sniff test" and modified my hypotheses so I could dig deeper into the business proposition. However, I have not yet totally eliminated hypothesis #2.