Articles

Evidence based

Evidence Based

This article is based on scientific evidence, written by experts and fact checked by experts.

Our team of licensed nutritionists and dietitians strive to be objective, unbiased, honest and to present both sides of the argument.

This article contains scientific references. The numbers in the parentheses (1, 2, 3) are clickable links to peer-reviewed scientific papers.

Should You Trust “Science-Based” Healthcare Claims?

Written by Dr. Michael Ruscio, DC on July 18, 2020

Written by Dr. Michael Ruscio, DC on July 18, 2020 — Reviewed by Hilary Watts

• How Research is Misrepresented|
• Levels of Research Quality|
• My Standards|
• Bottom Line|

A quick guide for healthcare consumers.

If you’ve spent any time on the internet researching healthcare topics, you’ve probably bumped into a number of outrageous product claims and lots of conflicting information. Everyone’s healthcare claim, it seems, is backed by a scientific study.

Sometimes it’s deliberate misinformation, but more often healthcare “experts” don’t take steps to test their personal biases and don’t invest the considerable time needed to fully understand a research topic.

How Marketers Misrepresent the Research

Here are some of different ways that internet marketers misrepresent scientific evidence:

• They claim a rat or petri dish study is proof that a product works for humans.
• They cherry pick one or two individual studies that support their claim, and ignore the rest of the research.
• They jump on bandwagons when an exciting new study is published, without waiting for follow-up research.

As a healthcare consumer, it’s important to educate yourself so that you can spot faulty scientific claims.

Levels of Research Quality

There’s a widely-accepted hierarchy of research quality, as shown in this pyramid diagram.

The difference between these levels of evidence is similar to the way restaurants are reviewed. You could ask one person’s opinion about a restaurant, but they may have been there on a particularly bad night. If you ask a group of friends, you might get a few different opinions. If you look online, you might find hundreds of reviews and now you can see overall trends and an average score. That’s super helpful.

So, with that in mind, let’s have a closer look at each of the four levels of evidence:

• The highest levels of evidence are summaries of multiple studies. These include:
  • Systematic Reviews – A review of all the available research data on a given topic.
  • Meta-Analyses – A review of all the available research using math to fully analyze the overall results.
• Human Clinical Trials (also known as randomized control trials) are human studies that test specific health interventions and control for the placebo effect. Human clinical trials are a very high level of research. Very large human clinical trials (with hundreds or thousands of subjects) are considered higher evidence than human clinical trials with fewer subjects.
• Observational Studies are studies where researchers observe humans but don’t make any health interventions. Observational studies can identify associations. For example, one study found that patients with SIBO had a higher frequency of coronary artery disease. [1] But this doesn’t establish cause and effect. In other words, there’s not enough information to claim that SIBO causes coronary artery disease. There are two types of observational studies:
  • Case Studies observe an individual patient.
  • Cohort Studies observe a group of people.
• Cell Studies and Animal Studies—Petri dish studies and rat studies are useful starting points in the research process, but they are not evidence for human health claims. Unfortunately, marketers and the media often cite animal and cell studies. This is the source of much confusion surrounding many health care topics.

Scientific research is a slow, step-by-step process of investigating and building a body of evidence. A single study may be interesting or intriguing, but it’s probably not definitive proof of anything. Over time and with additional studies, we can start to extract more meaningful conclusions from the research.

My Standards for Healthcare Claims

In a world where sloppy and misleading use of research is the norm, I’ve developed standards to ensure that we provide you with reliable and trustworthy healthcare information. For example:

• I rely on and cite the highest level evidence.
• I don’t just search for references to fit my beliefs. I start with a hypothesis and we search for all evidence that supports and contradicts the hypothesis.
• When the evidence is limited, I disclose that.
• I am open minded, yet cautious.
• I don’t like to jump on research bandwagons. While others hype an “exciting breakthrough product” based on just one study, I prefer to wait for supporting evidence.  When that becomes available, I try out products in my clinical practice for at least 6-8 months before I consider adding them to my product line.

How Poor-Quality Evidence Can Hurt You

I want to illustrate the kind of misinformation that can harm you as you try to make health care decisions.

Unresearched Claims

Digging through the research takes time and expertise. Many internet marketers simply can’t be bothered. So they parrot each other’s statements, until these claims become accepted as “the truth.” For example, many bloggers and marketers claim that probiotics are a bad idea if you have SIBO. Here are two examples:

• “With SIBO, when you take bifidobacterium probiotics, they jump off the digestive tract train in your small intestine. They can add to the overgrowth that’s already happening, and that means more discomfort for you.” [2]
• “There is no research-based answer as to whether probiotics for SIBO are helpful. But despite this lack of evidence, the internet is full of self-proclaimed experts advising all manner of probiotic supplement protocols claiming to treat or prevent SIBO. Consider this: If SIBO represent an overgrowth of normal (aka benign or “good”) bacteria in a neighborhood that’s too hospitable to bacteria, isn’t it possible–and even likely–that these supplements are actually “seeding” the recurrence by delivering concentrated doses of bacteria day after day?” [3]

In the first example, no evidence is presented to support the claim that probiotics add to SIBO overgrowth. The second example claims there is no research into probiotics for SIBO.

Both statements are flat-out wrong.

Probiotics (including bifidobacterium) have been CLEARLY shown to be effective in treating SIBO, both for improving symptoms and SIBO lab test values. Consider these studies:

• A meta-analysis (the highest quality science) summarizing 18 clinical trials concluded that probiotics are an effective treatment for SIBO. [4] The analysis concluded that probiotics reduce bacterial overgrowths and hydrogen concentrations, and improve symptoms, including abdominal pain.
• A small clinical trial with 25 patients found probiotics to be more effective than the antibiotic metronidazole (a standard treatment for SIBO). [5]
• Another small clinical trial found that probiotics work substantially better in patients with both IBS and SIBO, as compared to those who have IBS without SIBO. [6]

In this example, unresearched blog claims might have you avoiding a therapy that research shows is likely to help you.

Unresearched Products

Another potentially dangerous practice is marketing of products that simply have no research to support them. For example, most of the strains in this probiotic supplement have no research to support their safety and effectiveness.

With over 500 research studies that support the safety and effectiveness of a wide range of probiotic species, you have to wonder why this company chose to formulate their product this way. Ultimately it’s you, the healthcare consumer that is taking the risk.

Cherry Picking Results

Internet marketers like to cherry-pick data that supports their marketing goals. For example, here is a study that is often cited to sell probiotics for weight loss:

• Researchers found that subjects who ate yogurt with probiotic strains Lactobacillus fermentum or Lactobacillus amylovorus lost 3-4% body fat over a 6-week period, while a control group lost 1% in the same period. [7]

Sounds good, right?

Except, citing only one clinical trial with 28 participants, while ignoring a large body of higher-quality research is very misleading. In this case, the high-quality data suggests that probiotics are not very helpful for weight loss:

• A 2019 meta-analysis of 19 clinical trials suggests that probiotic supplements have a small effect on reducing waist circumference but no effect on body weight or BMI. The study also concluded that results are not definitive because the quality of data to date is low to moderate. [8]
• A 2017 meta-analysis of 15 clinical trials showed that probiotics have small effects on weight loss and no effect on fat mass. [9]
• A 2017 systematic review evaluated the effects of Lactobacillus on weight loss and/or fat mass in overweight adults. Of 14 studies included in the review, nine showed decreased body weight and/or body fat, three did not find effect and two showed weight gain. [10]
• A 2015 meta-analysis found no significant effect of probiotics on body weight and BMI. Researchers concluded that the quantity and quality of data available were insufficient to draw definitive conclusions. [11]

Reputable practitioners and marketers will provide you with a full summary of the evidence for their claims.

Bottom Line

It’s my personal goal to give you practical, science-based advice. I hope the information I’ve shared in this article will help you to become a more informed and savvy healthcare consumer.