I don’t much care for Neal Barnard as a nutrition authority–I frequently complain about him to my vegan friends–but the scientific illiteracy demonstrated in this tweet is frustrating and sad. And tweets like this are representative of much of nutrition Twitter. “Study shows X. Yet I saw Y. Conspiracy!!!” So this post is really about the use of anecdote in science. The results in the Barnard study represent the intervention group average under certain trial conditions; the anecdote on the other hand represents a single person’s experience under what could be entirely different conditions.
Barnard’s study is not incompatible with many individuals in that study also experiencing a similar outcome as the anecdote. It is even possible that Bernard’s intervention is more effective on average than the intervention advocated by the person criticizing Bernard.
This is because the person criticizing Bernard might in fact be an outlier compared to many other people pursuing the same strategy as him. Without more information, we don’t know.
As an aside, I do think a low-carbohydrate diet can play an important role in treating type 2 diabetes. I also think that a high-carbohydrate diet can also effectively treat type 2 diabetes.
The current evidence suggest that weight loss is the main factor in type 2 diabetes treatment, and that carbohydrate restriction can produce immediate gains in glycemic control that are not obtainable on a high-carbohydrate diet.
This would make carbohydrate restriction a desirable treatment modality in patients who cannot permanently lose adequate weight to reverse the underlying metabolic dysfunction. However, because a low- and high-carbohydrate diet do not produce different levels of weight loss, inasmuch as the underlying metabolic dysfunction is being reversed (which is driven by organ fat), a low- and high-carbohydrate diet are in this respect basically equivalent.
Back to the scientific problem.
Such extreme anecdotes lend toward their being overreported. That’s why we hear about them instead of about failures, etc. This makes the testimonials of people who follow diet gurus look so much more impressive than most trial results. This is also why case studies are not regarded as acceptable scientific evidence for an intervention: because they could be highly selective and unusual.
In fact, this is why small trials are regarded with the same skepticism.
A well-known pattern in the Alzheimer’s literature is in fact a highly successful case study or small trial followed by a failed larger clinical trial.
What seemed extraordinarily promising turns out to be totally ineffective when tested rigorously and in large numbers. This is in part because, out of many small trials being conducted, a number of them will turn out to show great promise by sheer chance. Large trials showing such benefit by sheer chance are far, far less likely.
In any case, this person’s anecdote amounts to a case study with less documentation. It is not even as trustworthy as a small trial.
What is frustrating is how popular this sort of anecdote is on Twitter. These anecdotes are only compelling if one believes that their representativeness of the efficacy of the treatment itself has been shown. But this is not the case. To my awareness, high- vs. low-carbohydrate diabetes reversal trials (using highly intensive interventions in each group, such as with Virta) have never been conducted.
I don’t think that a vegan diet in itself is a good idea for treating diabetes, and I do think that Bernard’s results are rather unimpressive and show the inadequacy of his specific intervention for treating the disease. However, the use of an anecdote to criticize the study is just plain silly. An anecdote cannot be used to criticize a large trial. That people seem to think that it can shows that many people simply do not understand what or why scientists do what they do.
It is my hope that some people who are just now learning about science can benefit from this discussion. If you would like to learn more, please ask questions in replies or over DM.