I have begun to work on intelligence agency decision-making and, in doing so, realized there were two different types. I have never seen this decision-making dichotomy written up anywhere and cant find it in the literature (any literature).
The first decision-making formulation is the standard process where the goal is to choose the best alternative from a list. This is facilitated by comparing each alternative to a set criteria. Each is measured relative to each criterion and its success in meeting it is combined (either formally or informally) to find the overall satisfaction with the alternative. For example, say you are buying a new car. One criteria is that the car should accelerate for 0 to 60 in some fast time and a second is that it should get better than xx miles/gallon. Information with which to evaluate each of these may come from different sources. I may actually measure the acceleration of one car but rely on the test figure in a magazine for another. In this way the evaluations for acceleration are independent from one alternative to the next. The mathematics for this is referred to as Multi-Attribute Utility Theory (MAUT). Methods based on MAUT that support decision-making are the decision matrix, Pugh's method, Expert Choice and Accord.
The second formulation is what is seen in medical and system diagnosis, and business and government intelligence. This has not been formalized as best I can tell. Here the goal is to choose the most likely hypothesis. A hypothesis is like an alternative. The big difference here is that , as each piece of evidence is gathered and analyzed, it either supports, denys or has no import on each of the hypotheses. This differs from the first formulation in that a single evaluation potentially adds information to all the hypotheses (instead of independent). For example, supposed we want to determine Iran's nuclear intentions. Hypotheses include: 1) generate power only, 2) develop a nuclear weapon for ground delivery, or 3) develop a nuclear weapon for air delivery. A piece of evidence, say a satellite photo of activity at an airbase, contributes information to all three of the hypotheses.
The standard MAUT formulation does not support this second type of decision-making and neither do the tools listed above. An aside, the medical profession uses the the term "evidence based" to mean making clinical data available to practitioners in a usable manner so that the doctor trying to figure out why your finger is rotting off has all the best clinical data on which to base her diagnosis. This does not really support choosing which of the hypothetical diseases you actually have, but supplies information on which to evaluate the evidence.
Situations like what I have just described can be modeled with methods like Bayes Nets, but there is no known method for facilitating the process as with the decision matirx, Pugh's and Accord. I have spent months working on this and have found little in the literature. Do you have any leads?
The first decision-making formulation is the standard process where the goal is to choose the best alternative from a list. This is facilitated by comparing each alternative to a set criteria. Each is measured relative to each criterion and its success in meeting it is combined (either formally or informally) to find the overall satisfaction with the alternative. For example, say you are buying a new car. One criteria is that the car should accelerate for 0 to 60 in some fast time and a second is that it should get better than xx miles/gallon. Information with which to evaluate each of these may come from different sources. I may actually measure the acceleration of one car but rely on the test figure in a magazine for another. In this way the evaluations for acceleration are independent from one alternative to the next. The mathematics for this is referred to as Multi-Attribute Utility Theory (MAUT). Methods based on MAUT that support decision-making are the decision matrix, Pugh's method, Expert Choice and Accord.
The second formulation is what is seen in medical and system diagnosis, and business and government intelligence. This has not been formalized as best I can tell. Here the goal is to choose the most likely hypothesis. A hypothesis is like an alternative. The big difference here is that , as each piece of evidence is gathered and analyzed, it either supports, denys or has no import on each of the hypotheses. This differs from the first formulation in that a single evaluation potentially adds information to all the hypotheses (instead of independent). For example, supposed we want to determine Iran's nuclear intentions. Hypotheses include: 1) generate power only, 2) develop a nuclear weapon for ground delivery, or 3) develop a nuclear weapon for air delivery. A piece of evidence, say a satellite photo of activity at an airbase, contributes information to all three of the hypotheses.
The standard MAUT formulation does not support this second type of decision-making and neither do the tools listed above. An aside, the medical profession uses the the term "evidence based" to mean making clinical data available to practitioners in a usable manner so that the doctor trying to figure out why your finger is rotting off has all the best clinical data on which to base her diagnosis. This does not really support choosing which of the hypothetical diseases you actually have, but supplies information on which to evaluate the evidence.
Situations like what I have just described can be modeled with methods like Bayes Nets, but there is no known method for facilitating the process as with the decision matirx, Pugh's and Accord. I have spent months working on this and have found little in the literature. Do you have any leads?
Labels: decision making, evidence based decisions, MAUT
Posted by David G. Ullman at 10:51 AM
del.icio.us
digg this
3 Comments:
Mr. Ullman,
I am working in a similar vein; I have always found good information in reading:
"Financial Speculation Theories"
and the related field of:
"Behavioural Finance" or "Behavioral Economics"
http://introduction.behaviouralfinance.net/
As to theories of speculation, given your problem, consider this recent quote from Professor Antal E. Fekete, which makes one think a bit:
"Theory of speculation
Speculation is man’s main tool to deal with risks and future uncertainties. Mainstream economics fails to make a distinction between risks created by nature and risks created by man. This distinction is fundamental. Speculation can effectively confront the former, while it will only aggravate the latter. "
Mr. Ullman,
I just read How Doctors Think by Jerome Goopman, MD. He suggest that doctors use alogrithms to confirm/deny findings in an evidence-based context (no remarks about who creates the alogriths - I am left thinking insurance companies and institutions.) He goes on to suggest that experience fuels the development of alternative ideas. He is interesting on the topic of heuristics and "emotional temperature" suggesting an inventory of emotions that might add momentum to a wrong diagnosis.
Cheers,
David Fish
www.trial-tools.com
Thanks for the note. I have read How Doctors Make Decisions in my effort to better understand the basics of evidenced based decision making. In fact, a big thing in medicine is Evidence Based Medicine. However, what this means is the organization and delivery of clinical information to the practitioner, not the use of evidence to make decisions. So, in the Goopman sense, EBM tries to supply sufficient info for the Drs to get the best possible results from their algorithms. The algorithms (diagnostic and prescriptive logic) come from medical school training. It is interesting to watch the House TV show and tease out the algorithms. The logic used in this show is very flawed and formulaic (which is why I don’t watch it anymore) but makes good drama.
What is your interest in this area?
Post a Comment
<< Making Robust Decisions - A Blog for Decision Makers