HOW TO EVALUATE MEDICAL EVIDENCE

Imagine you’re at your daughter’s soccer game and you overhear two other parents discussing a recent report they heard on TV about the dangers of a blood pressure medication for which your doctor just gave you a prescription. Should you ask the two people about what they heard? Should you still fill your prescription at the pharmacy? How do you evaluate medical information in order to help you make good medical decisions for yourself with your physician?

We are all inundated with medical information from our family, friends, acquaintances, co-workers, TV programs and advertisements, radio, newspapers, magazines, billboards, web sites, and even advertisements from lawyers seeking people who may have been injured by a certain medication. How do we decide which information is accurate and which is not? In many ways, it’s important for our health for each of us to be a Sherlock Holmes in order to privately investigate the evidence about the medical treatments we are considering for ourselves and our families.

It is important for you to get information about medical studies from reputable sources such as highly regarded newspapers, magazines or web sites such as WebMD and Medline Plus, the National Library of Medicine’s website for consumer health information. Information from people, print media or web sites which you don’t fully trust may not be providing you with accurate information. You should not discontinue any treatments without first consulting your physician, particularly if the information you are using is from sources which may not be reliable.

WHERE DOES THE BEST MEDICAL INFORMATION COME FROM?

Randomized, controlled trials are generally felt to provide the most valuable medical information. These are trials in which a group of people who all have one disease or problem are split up randomly (randomized) into two groups, one of which has a given intervention and the other of which does not receive the intervention (control group). The idea is to make the two groups as similar as possible, so the study will tell if a certain intervention is beneficial or not. For instance, if one is looking at whether a certain medication is lowering the risk of heart attacks, it would be important for the control and intervention groups to have equal proportions of smokers. If the intervention group has a higher proportion of smokers than the control group, then any possible benefit from the medication may be offset by the damage from the higher rate of smoking in the group. Randomized trials are particularly valuable if the control group is taking a placebo [pluh-see-boh] (a pill which does not contain any effective medication). In double-blind placebo controlled trials, individuals in the intervention and control groups do not know if they are taking the active treatment or placebo. However, it is not possible to do trials with a placebo if the intervention group has surgical therapy.

Other types of studies such as observational or case/control trials are felt to provide much weaker evidence as compared to randomized controlled trials. In observational studies, a large group of people are followed over time to see if certain traits are associated with more or less of a particular disease. For example, many observational studies of estrogen users before 2002 showed a strong association of estrogen use in post-menopausal women and reduced incidence of heart attacks. However, randomized controlled trials which have been completed over the last 10 years have shown no benefit on reducing heart attacks in postmenopausal women who took hormone replacement.The apparent benefit of hormones in the prior observational studies was due to the fact that women who chose to take hormone replacement also tended to lead a healthier lifestyle. It wasn’t the estrogen therapy which prevented the heart disease in these women, but rather the healthier lifestyle choices they made.

Another type of study is called a meta-analysis. This type of study tries to combine many prior studies of a similar type. For example, the large meta-analysis of type 2 diabetes treatment drug Avandia® and heart disease risk published in the New England Journal of Medicine in 2007 combined many randomized controlled trials of Avandia® versus placebo in type 2 diabetes. Meta-analyses do not provide any original research and, as such, are generally felt to be inappropriate to use to in making definitive conclusions about any medical intervention.

It is important to look at the number of subjects in any trial in order to understand its importance. The Heart Protection Study (HPS), which studied the use of statins (a medication that lowers cholesterol) and vitamin supplementation in patients at risk of cardiovascular disease and the JUPITER Study, aimed at evaluating whether statins reduce heart attacks and strokes in people with normal cholesterol levels, each contained around 20,000 participants, which makes them two of the largest trials of statin drugs. One interesting finding from these two studies was that statin users did not have more muscle symptoms than the control group members who were taking a placebo. We are all bombarded with information about the risks of statins on the muscles, yet in these two large randomized placebo-controlled studies statin users did not appear to suffer muscle pain due to the statin.

The duration of trials is important to consider. The media was full of information about the two studies published this year demonstrating the benefits of bariatric surgery on type 2 diabetes. Yet these trials were only of one-totwo-years duration. It would be important to know if the diabetes comes back three, four or more years after the surgery when people begin to regain some of the weight they have lost after bariatric surgery.

The characteristics of the study populations are important to understand in order to see if the study pertains to you. Although the HPS and JUPITER studies did not demonstrate muscle side effects from a statin, the people in the studies did not have liver or kidney disease and were not taking certain medications that may interact with statins. We know that individuals with kidney disease or those on certain medications are more likely to suffer muscle problems while on statins than are healthier individuals. Although bariatric surgery has been shown in randomized controlled trials to cause type 2 diabetes to go into remission, it will not have the same benefit for you if you have type 1 rather than type 2 diabetes.

Some studies use surrogate or “soft” endpoints, whereas others use “hard” endpoints. For example, if you are evaluating medications for cholesterol, it is important to choose one which will not only reduce your cholesterol level (a soft endpoint), but more importantly reduce your risk of having a heart attack (a hard endpoint). Many studies have shown that coronary stents open blocked arteries, but what is more important is the hard endpoint of reducing major heart attacks and death. The COURAGE Study, which examined the effectiveness of heart stents and angioplasty to drug treatment in individuals with stable coronary heart disease, demonstrated that coronary stents plus medical therapy did not prevent more hard cardiac endpoints than did medical therapy alone.

When looking at the benefits of any medical therapy in a randomized controlled trial, you should look at the difference between the intervention and the placebo group. Many diseases have a strong “placebo effect” where the group receiving a placebo demonstrates significant improvement. The true benefit of the intervention in the study is the difference between the results in the intervention group minus the placebo effect. For example, if people on a placebo lose 10 pounds of weight and those on a medication lose two pounds, then the benefit of the drug is only 12 minus 10 or a two-pound weight loss.

It is important for the results of any study to be replicated by other similar studies. One major study in a hospital in Belgium demonstrated that extremely tight control of blood pressure after surgery saved lives in the ICU. However, several other studies involving multiple sites demonstrated differing results.

Finally, after you have tried to investigate as carefully as possible, you should consult your own physician and discuss the question with her or him as an informed patient. This joint medical decision making will be much more productive for you because of your effort to evaluate the evidence.

Dr. Richard A. Haas is a board-certified endocrinologist in private practice in Worcester, Massachusetts. He completed his undergraduate studies at the Massachusetts Institute of Technology and medical school at the University of California San Francisco. Dr. Haas completed his residency in internal medicine at the Mount Sinai Medical Center in New York City and his fellowship in endocrinology, diabetes and metabolism at the Columbia College of Physicians and Surgeons. He is an Assistant Professor of Medicine at the University of Massachusetts Medical School and a member of the AACE Board of Directors.