Smart Heuristics

April 8, 2003 by Gerd Gigerenzer

Many people are ill-equipped to handle uncertainty. But the study of smart heuristics shows that there are strategies people actually use to make good decisions that deal openly with uncertainties, rather than denying their existence.

Originally published on Edge, March 31, 2003. Published on KurzweilAI.net April 8, 2003.

At the beginning of the 20th century, the father of modern science fiction, Herbert George Wells, said in his writings on politics, "If we want to have an educated citizenship in a modern technological society, we need to teach them three things: reading, writing, and statistical thinking." At the beginning of the 21st century, how far have we gotten with this program? In our society, we teach most citizens reading and writing from the time they are children, but not statistical thinking. John Alan Paulos has called this phenomenon innumeracy.

There are many stories documenting this problem. For instance, there was the weather forecaster who announced on American TV that if the probability that it will rain on Saturday is 50 percent and the probability that it will rain on Sunday is 50 percent, the probability that it will rain over the weekend is 100 percent. In another recent case reported by New Scientist, an inspector in the Food and Drug Administration visited a restaurant in Salt Lake City famous for its quiches made from four fresh eggs. She told the owner that according to FDA research, every fourth egg has salmonella bacteria, so the restaurant should only use three eggs in a quiche. We can laugh about these examples because we easily understand the mistakes involved, but there are more serious issues. When it comes to medical and legal issues, we need exactly the kind of education that H. G. Wells was asking for, and we haven’t gotten it.

What interests me is the question of how humans learn to live with uncertainty. Before the scientific revolution, determinism was a strong ideal. Religion brought about a denial of uncertainty, and many people knew that their kin or their race was exactly the one that God had favored. They also thought they were entitled to get rid of competing ideas and the people that propagated them. How does a society change from this condition into one in which we understand that there is this fundamental uncertainty? How do we avoid the illusion of certainty to produce the understanding that everything, whether it be a medical test or deciding on the best cure for a particular kind of cancer, has a fundamental element of uncertainty?

For instance, I’ve worked with physicians and physician-patient associations to try to teach the acceptance of uncertainty and the reasonable way to deal with it. Take HIV testing as an example. Brochures published by the Illinois Department of Health say that testing positive for HIV means that you have the virus. Thus, if you are an average person who is not in a particular risk group but test positive for HIV, this might lead you to choose to commit suicide, or move to California, or do something else quite drastic. But AIDS information in many countries is running on the illusion of certainty. The actual situation is rather like this: If you have about 10,000 people who are in no risk group, one of them will have the virus, and will test positive with practical certainty. Among the other 9,999, another one will test positive, but it’s a false positive. In this case we have two who test positive, although only one of them actually has the virus. Knowing about these very simple things can prevent serious disasters, of which there is unfortunately a record.

Still, medical societies, individual doctors, and individual patients either produce the illusion of certainty or want it. Everyone knows Benjamin Franklin’s adage that there is nothing certain in this world except death and taxes, but the doctors I interviewed tell me something different. They say, "If I would tell my patients what we don’t know, they would get very nervous, so it’s better not to tell them." Thus, this is one important area in which there is a need to get people—including individual doctors or lawyers in court—to be mature citizens and to help them understand and communicate risks.

Representation of information is important. In the case of many so-called cognitive illusions, the problem results from difficulties that arise from getting along with probabilities. The problem largely disappears the moment you give the person the information in natural frequencies. You basically put the mind back in a situation where it’s much easier to understand these probabilities. We can prove that natural frequencies can facilitate actual computations, and have known for a long time that representations—whether they be probabilities, frequencies or odds—have an impact on the human mind. There are very few theories about how this works.

I’ll give you a couple of examples relating to medical care. In the US and many European countries, women who are 40 years old are told to participate in mammography screening. Say that a woman takes her first mammogram and it comes out positive. She might ask the physician, "What does that mean? Do I have breast cancer? Or are my chances of having it 99%, 95%, or 90% or only 50%? What do we know at this point?" I have put the same question to radiologists who have done mammography screening for 20 or 25 years, including chiefs of departments. A third said they would tell this woman that, given a positive mammogram, her chance of having breast cancer is 90%.

However, what happens when they get additional relevant information? The chance that a woman in this age group has cancer is roughly 1%. If a woman has breast cancer, the probability that she will test positive on a mammogram is 90%. If a woman does not have breast cancer, the probability that she nevertheless tests positive is some 9%. In technical terms, you have a base rate of 1%, a sensitivity or hit rate of 90%, and a false positive rate of about 9%. So, how do you answer this woman who’s just tested positive for cancer? As I just said, about a third of the physicians thinks it’s 90%, another third thinks the answer should be something between 50% and 80%, and another third thinks the answer is between 1% and 10%. Again, these are professionals with many years of experience. It’s hard to imagine a larger variability in physicians’ judgments— between 1% and 90%—and if patients knew about this variability, they would not be very happy. This situation is typical of what we know from laboratory experiments: namely, that when people encounter probabilities—which are technically conditional probabilities—their minds are clouded when they try to make an inference.

What we do is to teach these physicians tools that change the representation so that they can see through the problem. We don’t send them to a statistics course, since they wouldn’t have the time to go in the first place, and most likely they wouldn’t understand it because they would be taught probabilities again. But how can we help them to understand the situation?

Let’s change the representation using natural frequencies, as if the physician would have observed these patients him- or herself. One can communicate the same information in the following, much more simple way. Think about 100 women. One of them has breast cancer. This was the 1%. She likely tests positive; that’s the 90%. Out of 99 who do not have breast cancer another 9 or 10 will test positive. So we have one in 9 or 10 who tests positive. How many of them actually has cancer? One out of ten. That’s not 90%, that’s not 50%, that’s one out of ten.

Here we have a method that enables physicians to see through the fog just by changing the representation, turning their innumeracy into insight. Many of these physicians have carried this innumeracy around for decades and have tried to hide it. When we interview them, they obviously admit it, saying, "I don’t know what to do with these numbers. I always confuse these things." Here we have a chance to use very simple tools to help those patients and physicians to understand what the risks are and which enable them to have a reasonable reaction to what to do. If you take the perspective of a patient—that this test means that there is a 90% chance you have cancer—you can imagine what emotions set in, emotions that do not help her to reason the right way. But informing her that only one out of ten women who tests positive actually has cancer would help her to have a cooler attitude and to make more reasonable decisions.

[Continued on Edge.org.]

© 2003 Edge Foundation, Inc. Reprinted with permission.