Read Research with Discretion

Today's post is about reading research as it is presented to consumers. Our example article is meant to be read by the average person, not scholars. It is meant to inform the general public. It is meant to be trusted. 

This particular article is about Non-Celiac Gluten Sensitivity. 

So, should you trust what this article is telling you? No. The article is actively trying to get you to feel secure in the second study presented. After all, the researcher did the second study "like any meticulous scientist." The article is telling you that the scientist is meticulous. That phrasing should send warning signals to you right away. 

So let's take a closer look at a few things in this article...

First, it should be noted that there is not currently a universally accepted method of diagnosing someone with non-celiac gluten sensitivity (NCGS). Therefore, the participants that have this issue are most likely self-diagnosed or diagnosed in a "best guess" method.

Now let's consider that first study - the one the article wants us to think wasn't very good. It is listed as “double-blinded, randomized, and placebo-controlled” by the article. This would imply that the study had participants randomly placed into 2 groups. One group would receive food containing gluten, and one group would receive food without gluten. This provides you with a test group and a control group. The term “double-blind” implies that neither the participants nor the persons supplying the food to the participants knew which group was which. This is important because it keeps the researchers from projecting specific, desired outcomes onto the different groups. What about this sounds like a bad study to you?

According to the article, the second study did a better job at controlling variables. This was done by removing more dietary components, such as lactose. This is stated to be in an effort to more accurately pinpoint the cause of the gastrointestinal distress. However, by removing more items from the food, you have added more variables. A successful nutritional study should have only one variable at a time, in this case the presence of gluten. The FODMAPs should have been present in both the control and the test. By doing this, you would only be testing the difference between when gluten is present and when it is not. This study also failed to properly randomize the order in which a baseline is given, resulting in the opportunity to have a strong physiological response, in this case a nocebo effect. This is a bad experiment design.

Some things to think about regarding the second study:

·   The study is listed as using only 37 subjects. This is an extremely small sampling, since it is meant to represent the population of the world. Generally speaking, you shouldn't give credit to a study with less than 70 participants.

·   The “baseline” in the second study was a diet with the newly introduced variables removed. This is not an accurate example of what a normal diet would consist of, and therefore should not be used as a baseline. A baseline is generally defined as a set of critical observations used as a control. However, this study used at least two variables and a placebo, whey protein, and no real control.

·    It did not randomize the order in which the participants received the “baseline” diet. Therefore, the participants were expecting to not become ill from the first, the baseline, diet that they received. This allowed the participants to project expectations on to the diets that were not “baseline,” creating negative reactions to all but the baseline diets. This skewed the overall findings of the research to imply that the dietary issues were the result of the presence of FODMAPs in the participants’ foods. This therefore created the opportunity to have a nocebo effect.

·   The second study only tested persons who self-identified as having NCGS. Therefore, conclusions such as “[removal of FODMAPs] could explain why the millions of people worldwide who swear by gluten-free diets feel better after going gluten-free” should be avoided. 

In addition, you should take into account things like the presence of the NPD study which states that 30% of American’s would like to eat less gluten. The NPD Group is a private research corporation. In general, private research corporations make money by releasing information that can be used by other companies to promote money making business strategies, i.e. releasing a study saying that gluten free is important. Therefore, information received from such research corporations is often times skewed. This can be done through a variety of methods, such as being overly selective in who is allowed to answer survey questions. The survey questions could be linked to a health food website, for example. Providing information from a private research corporation such as The NPD Group along with research done by a gastroenterology professional as though both sets of research are of a scale would therefore skew an average reader’s perception of the outcome.

What you need to take away from all of this: Don't take every article as the gospel. Please read discerningly.