Here you'll find the answers to a wealth of health and wellness questions posed by Healthy Talk fans. Listen in because what you know helps ensure healthy choices you can live with. Today on Healthy Talk, you wanted to know:
How can we better evaluate a clinical study and determine whether it's a valid study or not?
If you're reading an article, or watching the morning or evening news, most headlines are meant to draw your attention and sensationalize the subject matter. Even though you want to trust what the main news outlets are telling you, it's SO important to do your own research. It's also important to ask yourself questions like, who conducted the study? Who participated in the study? What are they specifically studying? How are they studying this/gathering information?
A great website that Dr. Mike likes to use as a reference is PubMed.
If you have a health question or concern, Dr. Mike encourages you to write him at This email address is being protected from spambots. You need JavaScript enabled to view it.
so he can provide you with support and helpful advice.
Selected Podcast
Ask Dr. Mike: How Can You Better Evaluate a Clinical Study?
Transcription:
RadioMD Presents:Healthy Talk | Original Air Date: February 13, 2015
Host: Michael Smith, MD
You’re listening to RadioMD. It’s time to “Ask Dr. Mike” on Healthy Talk. Call or email to ask your questions now. Email: This email address is being protected from spambots. You need JavaScript enabled to view it. or call: 877-711-5211. The lines are open.
This first question, from a listener, I think is really important because, you know, more and more we’re seeing a lot of anti-supplement studies and some of them aren’t all that great as far as how they’re designed. There’s definitely some biasness in there, so this question comes from Sam. He says, “Dear Dr. Mike: How can we as laypeople better evaluate a clinical study? You know, whether it’s a valid study or not?”
Well, you know, that’s a great question and, you know, maybe this is a whole segment, as a matter of fact, but let me just try to give you a few things that you can ask yourself when you’re reading a headline or you’re watching the morning news or, you know, Good Morning, America, whatever it is you watch, and they mention some study briefly, usually with a sensational headline.
You have to do some of your own research, right? You know, Sam, this is a great question. I think it’s important, but there’s not an easy answer. You do have to do some work. Don’t forget: PubMed.com is a great site, right? When you go to PubMed.com, it literally is a huge database of all of the publications in medicine—in science. I mean, it’s all right there and you just search. You know, let’s say you’re watching Good Morning, America and the headline is “Grape Seeds Cause Cancer. Stop Taking Grape Seed Extract,” or something like that. I just totally made that up, right? That’s not a headline. So, you would go to PubMed.com, right? Don’t believe—I mean, listen to the show, whatever. Take it with a grain of salt, but do your due diligence. You know? Be a smart consumer, right? Go to PubMed.com and type in “grape seed extract” and “cancer”. Just type that in and you’ll see, you know, what’s out there; what’s going on. That one study might even pop up first, right? You may have to play around with the search terms a little bit on PubMed. Maybe it’s just “grape seed” and “cancer” or “grape seed cancer”. Whatever. You’ve got to play around with it. So, you do have to do some work. By the way, all good websites, all good shows, you know, like RadioMD.com and all the shows on this wonderful network, we provide you the exact reference and if we don’t it’s just by mistake. But we, 99% of the time, we will tell you exactly the journal, what year. That’s even easier. You plug that into PubMed and that one study just pops up. That’s it. There are no choices. So, good solid companies, institutions, networks—they provide you with that. It’s not hard to find. So, it’s always these mainstream media shows that make it hard to find some of the references, but anyway, PubMed.com. That’s your “go to”. So, here’s just some things.
So, when you see that headline, just a few things to remember. Mainstream media, number one, sensationalizes everything, alright? And that’s just the first thing to remember. Positive or negative. Even if it’s a positive study about grape seeds, right? Mainstream media—those morning shows—are going to make it seem even too good to be true or too bad to be true. So, just remember that, okay?
So, number two, you know, in most cases, Sam, these anti-supplement studies, they are not what the study was actually designed to look at. In most anti-supplement studies that have been published over the last, say, five or six years, what’s happened is there was a study about, let’s say, cancer and a chemotherapy drug and how nutrition affects the chemotherapy drug or whatever. And then, they decided to do a subanalysis out of that on just grape seeds, grape seed extract. So, most anti-supplement studies are borrowing from another study that was designed to do and investigate something totally different. That’s number two. So, the headlines are sensationalized and it’s not an original study. They’re borrowing from another study. And that’s not necessarily a bad thing to do, but as a doctor, I know that when I look at a subanalysis from another study looking at a different issue, I should never draw any definite conclusions. That’s the point. That’s the problem is you have these researchers who are biased against supplements; they’re borrowing from another study; they’re doing a subanalysis and then, they act like they’ve got some definitive conclusion. You should never do that off a subanalysis. Never. So, that’s number two.
Number three: ask the question, “Who?” What I mean by that is who was in this study? Who were the participants? What you will find in most cases, when it’s anti-supplement, is the “who” were sick people? People who already had established disease, in most cases. Not all, but most cases, and in these studies, some of these people the conventional medicines weren’t even helping. There was one a couple years ago that was looking at fish oil and heart disease and the conclusion was that “fish oils do nothing for your heart”. Well, it turns out the people they were studying were people who already had heart attacks. Well, I’ve got news for you. There are not many conventional medicines that are helpful in those cases. I mean, there are some that reduce the risk of a second heart attack, but not all that great. So ask, “Who?” In most cases, it’s already people who have established disease or, the other side of it is people who are living, you know, the average American lifestyle which is pretty unhealthy. Okay. So, that’s number three.
Number four: ask the question, “What?” What are they studying? What is this nutrient? Okay, since I’ve been using grape seed extract, let’s just go with that one. What about this grape seed extract? Where did they get it from? Is it the right dose? Is it the right quality? Are they even asking those questions? What you’re going to find is the next question--the “How?”—the how are they studying this? Sam, it’s mostly surveys. So, when you ask questions like, “Well, how much grape seed extract were these people taking? What was the quality of it?” They don’t have any of those answers because the way—the process for gathering; the how did they gather that information—was simply through surveys where all they do is they simply ask the people in this study about something totally different, by the way. “Hey, who’s taking grape seed extract?” “Oh, I am.” And maybe that person took it once in their life. Maybe they took it three times. Maybe they do take it every day, but they lump all of those people as grape seed extract users. They lump them all together, so the “what”—the dose of the grape seed, the quality of the grape seed—it’s never even asked because they don’t ask that question. All they do in the “how”—how are they studying this—it’s with a survey. They’re just lumping you into two different groups. People who have used grape seed extract at some point in their life and people who have never used it. Or, at least that’s what they say, based on these surveys. So, if you just remember: headlines are sensationalized, okay? In most of these anti-supplement studies, they are using a subanalysis approach. So, they’re borrowing from an original study that was studying something totally different, right? Number three: in many cases, the “who” are already sick people or unhealthy people—unhealthy lifestyles. The “what”—the quality of the extract, the dose, is not even answered because they’re using “how”? Surveys.
So, look for those things. Look for was this a new study or an old study? Who are they studying? What are they studying? How are they studying it? In most cases, you can answer it with sick people, you know, surveys and an old study. So, again, it’s not that you can’t do that. It’s not that you can’t study things that way, you just can’t draw definitive conclusions.
Kevin, how am I doing on time? I know you said in my ear, but I…Okay. Thirty seconds.
I got so wrapped up in all of that because this is important. We’re going to see more and more anti-supplement studies come out there. So, be a smart consumer. Use PubMed, ask the “who” the “what” and the “how” before you believe those headlines.
This is Healthy Talk on RadioMD. I’m Dr. Mike. Stay well.
RadioMD Presents:Healthy Talk | Original Air Date: February 13, 2015
Host: Michael Smith, MD
You’re listening to RadioMD. It’s time to “Ask Dr. Mike” on Healthy Talk. Call or email to ask your questions now. Email: This email address is being protected from spambots. You need JavaScript enabled to view it. or call: 877-711-5211. The lines are open.
This first question, from a listener, I think is really important because, you know, more and more we’re seeing a lot of anti-supplement studies and some of them aren’t all that great as far as how they’re designed. There’s definitely some biasness in there, so this question comes from Sam. He says, “Dear Dr. Mike: How can we as laypeople better evaluate a clinical study? You know, whether it’s a valid study or not?”
Well, you know, that’s a great question and, you know, maybe this is a whole segment, as a matter of fact, but let me just try to give you a few things that you can ask yourself when you’re reading a headline or you’re watching the morning news or, you know, Good Morning, America, whatever it is you watch, and they mention some study briefly, usually with a sensational headline.
You have to do some of your own research, right? You know, Sam, this is a great question. I think it’s important, but there’s not an easy answer. You do have to do some work. Don’t forget: PubMed.com is a great site, right? When you go to PubMed.com, it literally is a huge database of all of the publications in medicine—in science. I mean, it’s all right there and you just search. You know, let’s say you’re watching Good Morning, America and the headline is “Grape Seeds Cause Cancer. Stop Taking Grape Seed Extract,” or something like that. I just totally made that up, right? That’s not a headline. So, you would go to PubMed.com, right? Don’t believe—I mean, listen to the show, whatever. Take it with a grain of salt, but do your due diligence. You know? Be a smart consumer, right? Go to PubMed.com and type in “grape seed extract” and “cancer”. Just type that in and you’ll see, you know, what’s out there; what’s going on. That one study might even pop up first, right? You may have to play around with the search terms a little bit on PubMed. Maybe it’s just “grape seed” and “cancer” or “grape seed cancer”. Whatever. You’ve got to play around with it. So, you do have to do some work. By the way, all good websites, all good shows, you know, like RadioMD.com and all the shows on this wonderful network, we provide you the exact reference and if we don’t it’s just by mistake. But we, 99% of the time, we will tell you exactly the journal, what year. That’s even easier. You plug that into PubMed and that one study just pops up. That’s it. There are no choices. So, good solid companies, institutions, networks—they provide you with that. It’s not hard to find. So, it’s always these mainstream media shows that make it hard to find some of the references, but anyway, PubMed.com. That’s your “go to”. So, here’s just some things.
So, when you see that headline, just a few things to remember. Mainstream media, number one, sensationalizes everything, alright? And that’s just the first thing to remember. Positive or negative. Even if it’s a positive study about grape seeds, right? Mainstream media—those morning shows—are going to make it seem even too good to be true or too bad to be true. So, just remember that, okay?
So, number two, you know, in most cases, Sam, these anti-supplement studies, they are not what the study was actually designed to look at. In most anti-supplement studies that have been published over the last, say, five or six years, what’s happened is there was a study about, let’s say, cancer and a chemotherapy drug and how nutrition affects the chemotherapy drug or whatever. And then, they decided to do a subanalysis out of that on just grape seeds, grape seed extract. So, most anti-supplement studies are borrowing from another study that was designed to do and investigate something totally different. That’s number two. So, the headlines are sensationalized and it’s not an original study. They’re borrowing from another study. And that’s not necessarily a bad thing to do, but as a doctor, I know that when I look at a subanalysis from another study looking at a different issue, I should never draw any definite conclusions. That’s the point. That’s the problem is you have these researchers who are biased against supplements; they’re borrowing from another study; they’re doing a subanalysis and then, they act like they’ve got some definitive conclusion. You should never do that off a subanalysis. Never. So, that’s number two.
Number three: ask the question, “Who?” What I mean by that is who was in this study? Who were the participants? What you will find in most cases, when it’s anti-supplement, is the “who” were sick people? People who already had established disease, in most cases. Not all, but most cases, and in these studies, some of these people the conventional medicines weren’t even helping. There was one a couple years ago that was looking at fish oil and heart disease and the conclusion was that “fish oils do nothing for your heart”. Well, it turns out the people they were studying were people who already had heart attacks. Well, I’ve got news for you. There are not many conventional medicines that are helpful in those cases. I mean, there are some that reduce the risk of a second heart attack, but not all that great. So ask, “Who?” In most cases, it’s already people who have established disease or, the other side of it is people who are living, you know, the average American lifestyle which is pretty unhealthy. Okay. So, that’s number three.
Number four: ask the question, “What?” What are they studying? What is this nutrient? Okay, since I’ve been using grape seed extract, let’s just go with that one. What about this grape seed extract? Where did they get it from? Is it the right dose? Is it the right quality? Are they even asking those questions? What you’re going to find is the next question--the “How?”—the how are they studying this? Sam, it’s mostly surveys. So, when you ask questions like, “Well, how much grape seed extract were these people taking? What was the quality of it?” They don’t have any of those answers because the way—the process for gathering; the how did they gather that information—was simply through surveys where all they do is they simply ask the people in this study about something totally different, by the way. “Hey, who’s taking grape seed extract?” “Oh, I am.” And maybe that person took it once in their life. Maybe they took it three times. Maybe they do take it every day, but they lump all of those people as grape seed extract users. They lump them all together, so the “what”—the dose of the grape seed, the quality of the grape seed—it’s never even asked because they don’t ask that question. All they do in the “how”—how are they studying this—it’s with a survey. They’re just lumping you into two different groups. People who have used grape seed extract at some point in their life and people who have never used it. Or, at least that’s what they say, based on these surveys. So, if you just remember: headlines are sensationalized, okay? In most of these anti-supplement studies, they are using a subanalysis approach. So, they’re borrowing from an original study that was studying something totally different, right? Number three: in many cases, the “who” are already sick people or unhealthy people—unhealthy lifestyles. The “what”—the quality of the extract, the dose, is not even answered because they’re using “how”? Surveys.
So, look for those things. Look for was this a new study or an old study? Who are they studying? What are they studying? How are they studying it? In most cases, you can answer it with sick people, you know, surveys and an old study. So, again, it’s not that you can’t do that. It’s not that you can’t study things that way, you just can’t draw definitive conclusions.
Kevin, how am I doing on time? I know you said in my ear, but I…Okay. Thirty seconds.
I got so wrapped up in all of that because this is important. We’re going to see more and more anti-supplement studies come out there. So, be a smart consumer. Use PubMed, ask the “who” the “what” and the “how” before you believe those headlines.
This is Healthy Talk on RadioMD. I’m Dr. Mike. Stay well.