Most estimates suck

The rebuilding of the World Trade Center (WTC) is behind schedule and over budget. According to the executive director of the Port Authority, Christopher Ward "The schedule and cost estimates for the rebuilding effort that have been communicated to the public are not realistic," (from AP article). I can only say "Duh!"

All decisions are based on estimates of the murky future. In most public works projects honest estimates would never get them funded. It is much easier to get something approved with optimistic time and cost estimates and then, once its started ask for more. However, even if you are not playing politics, making estimates is hard work that is not well recognized for its impact of organizations

Let's get this straight. Even if you are trying to be as accurate and honest as possible, making an estimate is a highly uncertain undertaking. And, most decisions are based on many, interdependent estimates. In a simple experiment, I asked hundreds of people to estimate the length of time needed to clean some dishes. I gave them a detailed list of the dishes, and a photo of them, the sink, and cleaning materials. I asked them to estimate how long to clean the dishes. Depending on how I worded the question, the average time estimated was anywhere from 17 to 32 minutes and standard deviation was as high 10 minutes. In other words, asking for an estimate for this simple, daily task isn't a whole lot better than using a random number generator set to give an estimate in the range of 10 -40 min.

If this simple estimate is so bad, think of what happens with complexity, or for tasks that have not been done before.

What is to be done then. The only solution I know is to use methods that account for uncertainty. For planning , the old PERT system made an effort at this (much improved method is Critical Chain). For decision making, I have been pushing people for years to include uncertainty in their decisions making process.

In decision making, it is not only the estimates that are uncertain, it is the targets that you are trying to achieve that are uncertain. More on this in a later blog.

Back to the World Trade Center, I believe that if there had been a non-political effort at making decisions, and uncertainty had been included in the estimates, the plan would have been far less grandiose. However, the WTC should be grandiose, shouldn't it? This leads me to believe that honest estimates may not be the bast for all situations. I would hate to run or work for a business with that in mind.

Bayesian Brain and Bayesian Teams

An interesting article "Is This a Unified Theory of the Brain?" appeared in the New Scientist (full text) on May 28th. This is basically a discussion of the work and theories of neuroscientist Karl Friston of the University College in London. His work is based on an earlier theories that the brain makes decisions by trying to make sense out of the uncertainties in the outside world. Basically, you make hypotheses about reality and compare sensory inputs to them, updating the hypotheses and the belief in them as you gather more information. In essence you are constantly updating the probabilities that the hypotheses are true. In fact the wiring in the brain is continuously changing to is suppress prediction errors.

What makes this really fascinating is that 1) this is Bayesian updating and 2) this applies to team decision making also. The first observation has spawned the Bayesian Brain camp of neuroscientists. They believe we are all Bayesian thinkers, updating the probabilities that your hypotheses are correct as you solve small problems (e.g. if I turn the knob, the door will open) and large ones (e.g. If I choose to study the Bayesian brain, I will better understand team decision-making).

This brings us to applying what the neuroscientists are doing on the individual, to what happens in a team making business or technical decisions. These decisions are generally about what courses of action to take to address a current situation (obscured by its immediacy) or the future (clouded by uncertainty). A team that is functioning well will develop alternative courses of action or hypotheses and then gather an communicate information to increase its belief that one is better than the other, or to update the options based on the new information. No different than a single brain, just much more complicated.

What makes it harder is not only the communication of information amongst the team members (a clear focus of Information Management, Business Intelligence and the Webex's of the world), but developing a shared vision of the information. This is not to imply that everyone needs to understand all the information, but that there is some common understanding of the important bits. This is a topic I beat on in Chapter 4 of Making Robust Decisions "Team Don't Make Decisions, But...."

When we were first developing Accord software, an effort to support the team decision making process, we assumed that we could help teams by making what occurs inside one person's head transparent for the entire team. Maybe we should become neuro-scioscientists and study "Bayesian Team Dynamics".