By the Numbers

A conversation about understanding medical statistics and what they mean for health care decisions

Medical consumers face an increasingly dense thicket of statistical claims about medicines, procedures, and tests. According to research conducted by Lisa Schwartz, MD, and Steven Woloshin, MD, much of this information is not only misleading but can encourage poor health decisions.

Focus article photo

Lisa Schwartz, MD, and Steven Woloshin, MD, researchers at The Dartmouth Institute and Norris Cotton Cancer Center. Healthy skepticism "doesn’t mean disbelieving everything, it means you want to know something about the science behind the claim."

The couple spoke with Focus about their work, and their recent book, Know Your Chances: Understanding Health Statistics (University of California Press), which expands on the points they made in conversation.

In a media environment where medical consumers are barraged with numbers, statistics, and hype, how can we gather facts to help us make good medical decisions for ourselves and our families?

Steven Woloshin, MD: It's important to approach advertisements, public service announcements, and the medical news with healthy skepticism. That doesn't mean disbelieving everything, it means you want to know something about the science behind the claim—can I believe what I am hearing? Is the claim based on good science like a randomized trial of large numbers of people? Then if you think the claim is credible, ask: What's really being measured? When they say the drug reduces such and such by 48 percent, ask, "Does the drug reduce the chance of dying or feeling sick—or does it just make some lab tests look better?" If it just makes a lab test look better, you should be much more skeptical.

Lisa Schwartz, MD: You really need to get the numbers. How big is the benefit? How big are the harms? Asking your doctor is a great place to start. Sometimes you'll need to do some research on your own. For example, if you hear that a new drug reduces the chance of fractures from osteoporosis by 68%, ask, "68% of what?" If the chance of fracture is very low to start, a 68% reduction might not mean much. It's no different from how you'd approach a sale. If a store across town were having a "68% off" sale on selected items, you'd ask what items. You'd be more likely to schlep across town if big-ticket items like flat screen TVs were on sale than low-ticket items like a pack of gum.

What's a common way cancer statistics are misused or misunderstood?

Dr. Schwartz: No question, survival statistics. These are used all the time to convince you to get screening tests to detect cancer early. The PSA test for prostate cancer is a familiar example. You may hear statements like: "Most prostate cancers detected by PSA screening are at a very early stage—nearly 100% have survived 5 years." This makes screening sound like a no-brainer. But the argument is an illusion. Screening lets people know they have cancer earlier than if they waited for symptoms. So survival from the time of diagnosis has to increase. But it is possible that they don't live even one day longer. We just can't know from these kinds of statistics. The only way to know is from the results of a randomized trial showing that fewer people who are screened die from cancer.

Dr. Woloshin: Survival statistics also play well in the press. Every disease, every kind of cancer, has a politician who champions it, and they use survival statistics all the time to justify one thing or another because they sound impressive. But this is where healthy skepticism is so important. What do these statistics really mean? Often they don't mean anything.

Dr. Schwartz: We all want to believe there are things we can do to avoid dying from cancer. That's why we're so vulnerable to the power of survival statistics. The way they're used, they make you think that if you have this or that test or take this or that drug, you're going to live longer. It's very difficult to resist information like that. I think it comes down to being honest about what we know—and what we don't know—about what will really change our chances of death.

The "drug box" that you developed, which is similar to the "Nutrition Facts" panel consumers are already familiar with on food labels, offers consumers a way to better understand research statistics. Describe how the box informs consumers.

Dr. Woloshin: It gives the actual data from research in a way that's easily understandable. Consumers will be able to see for themselves whether the drug is actually "100% better" or whatever the claim is. The information is balanced and accessible. But the real point of the drug box is to make the discussion between doctor and patient possible—by making the information easily available.

Dr. Schwartz: If you don't see the actual numbers, you haven't given yourself a chance to fully understand the information. You should have the information you need to make an informed decision, and that's what the drug box provides.

Dr. Woloshin: We've been working with the U.S. Food and Drug Administration on this for several years, gradually working our way up the ladder. The FDA has now invited us to help them. They recognize that the problem of misinformation isn't just in the drug ads, it goes back to the label (the package insert—also called the prescribing information for doctors).

Dr. Schwartz: The drug box was actually included in the health-reform bill that passed the House and was signed by President Obama—half of page 1,113 of the bill describes the drug box. Now the Health and Human Services Department must study the drug box, determine its effectiveness, and implement the box if it's deemed effective. We've become the law!

July 10, 2010