Last week, Peter Attia wrote a criticism of Game Changers, where he presented the argument (long repeated him) that nutritional epidemiology is fundamentally flawed. By implication, it cannot be used to make nutrition recommendations. His post can be found here.
I am responding to this post because:
a) Peter’s post made it into my message inbox from friends;
b) it misunderstands the basic purpose of nutritional epidemiology;
c) it is a widespread (if misguided and fallacious) view.
I have many problems with Game Changers, which I have discussed elsewhere and which I hope to review sometime this month in a concise format. I have a video here, but I recommend you wait until I can produce the more concise version.
So this is not a defense of TGC. It is a criticism of a widely held view of nutritional epidemiology.
Now, Peter’s article could have been summed up in the singular phrase: “residual confounding”.
A complete summary might be:
Any diet is better than the standard American diet, plant-based or not. Moreover, plant-based diets are associated not just with lower mortality but also with other healthy behaviors. These other healthy behaviors might cause plant-based diets to appear more healthy than they really are, simply by association. Therefore, the conclusions of nutritional epidemiology are not formally scientifically valid.
And that would be that.
This is the version of the argument that I am responding to. I am summarizing it in this way to avoid having my readers get lost in the weeds, and to focus on the essentials. Now the arguments.
To start with, in nutrition science and much of the biomedical sciences, the hypotheses cannot be directly tested: we cannot lock people in cages for their entire lives and control all variables to directly and precisely test the hypotheses of nutrition science. Even if we could, this confinement would itself confound the results, i.e. we would ask: would free-living humans respond the same way as confined ones? Thus, even confinement might not be a solution, depending on the question, even if it were practical or ethical.
Nutrition science and much of the biomedical sciences must therefore adduce evidence for their hypotheses indirectly. Since many hypotheses are not directly testable, much of nutrition science and the biomedical sciences are not, strictly speaking, falsifiable, and are not, therefore, according to the Popperian definition, sciences.
What is nutrition science, then? To provide a full explanation, let’s progress further through the problem and revisit this question later.
Now, to be sure: Peter is right that restricting meat might not account for the health advantages that we see in the epidemiology of people who eat less meat. Again, his article is just one long way to say “residual confounding.”
But the evidence in favor of eating plants over meat is not just the epidemiology (which I review here; please read if you are not familiar with the epidemiology on meat and animal products versus plants). The evidence in favor of plants can also bee found in the feeding studies in animals, in short-term biomarker studies in humans, and in long-term randomized controlled trials. Peter is aware that protein drives mTOR, IGF-1, etc., and thus potentially cancer. Peter is aware that saturated fat raises LDL in many people and that LDL drives cardiovascular disease. He is aware of the classic long-term studies, which, while incredibly flawed in design, also collectively tend to show superior outcomes when saturated fat is replaced by unsaturated fat:
And he has done much to bring attention to these facts in the growing community of people interested in longevity.
But Peter is making the point that, formally, even though these lines of evidence converge–epidemiology, short-term biomarkers, randomized controlled trials, and animal studies–none of these alone demonstrate the long-term detrimental effect of meat intake in humans.
This is true. As I wrote above, nutrition science–given current methods–cannot formally test the causal linkages between many lifestyle practices and health. And the randomized controlled trials, too, cannot stand on their own, because of their gross design flaws. (Thus a proliferation of contradictory meta-analyses on saturated fat randomized controlled trials.)
As I pointed out, the lack of solid RCT evidence means that formal scientific tests of these linkages are impossible. Short-term biomarkers do not establish the causal linkages, and neither do animal studies, and neither does epidemiology. In his criticism, Peter merely highlights this simple fact, this veritable law of most of nutrition science and much of health science.
Yet in every institution in which the government must make decisions about which foods to serve–schools, prisons, public hospitals, etc.–it must also have some sorts of guidelines governing how those decisions are to be made to maximize the human benefits and minimize human costs. The government must have dietary guidelines, and these guidelines must be based on evidence. This, despite the fact that our evidence is incomplete and weak, and we do not have immediate practical means to decisively test our hypotheses.
This makes criticizing nutrition science markedly different from criticizing more traditional sciences. In medicine, we can conduct a clinical trial and, based on the results of that trial, decide whether or not to use the drug. If the drug does not work, we can in principle decide not to use anything. But this is not the same for nutrition science. For nutrition science, we must eat something; the government must formulate nutrition policy.
It is not just that nutrition science can say little conclusively–but that it must say something, because we must act. This makes nutrition science radically and essentially different from many other sciences.
Let us return to the question: What is nutrition science? The conclusion of the above discussion is that nutrition science is in many cases not a science. It is not a Popperian enterprise relying upon definitive falsification of directly empirically testable (and incorrect) hypotheses but a practice that can, in best case scenario, lean upon falsification but more frequently relies upon accretion of disparate pieces of evidence, many of which are not direct tests of our hypothesis.
In a word, there is no such thing in many areas as nutrition science–though there is sometimes, for some questions. Instead, what researchers are doing is not science but a practice that uses the systematic principles of science, combined with judgment, for practical ends. We are talking therefore not so much about nutrition science but of scientific nutrition.
In this case, a case where we must act with incomplete information, it is not appropriate to apply Popperian scientific criteria to the problem. Rather, as in clinical medicine, we must approach the question using our best judgment to manage the risk. We must do scientific nutrition.
The question then is: what are the likely tradeoffs of reducing meat intake?
The answer: in the average context, it is likely either beneficial or neutral.
It is unlikely, after all, that so much data could point in the direction of meat being harmful relative to plants that the opposite should be true.
To explain: if eating plants instead of meat is harmful, this effect is certainly negligible relative to the many health decisions that are consistently obscuring that fact in the epidemiological literature. After all, the very point to making the “healthy user effect” argument is that the healthy user effect is stronger than diet. In other words, if a meat-heavy diet does happen to be more healthful than a plant-heavy diet, this effect is likely to be very minor compared to other lifestyle practices.
In other words, in the most meat-favorable scenario, eating meat is not likely to be very beneficial to health relative to many other things that one could do.
On the other hand, if the nutritional epidemiology, echoing most other evidence we have from other areas of nutrition research, is onto something with respect to meat, then we only stand to gain by eating more plants and less meat.
In other words, we have very little to lose and something substantial, perhaps, to gain. This is the appropriate framework–a risk management framework–through which to understand the meat versus plants discussion. Not a Popperian one or one that takes as its criteria that from drug trials.
What we have is a situation where someone is saying “meat might or might not be unhealthful; we don’t know.” So why wouldn’t we hedge in the direction of harmful, given that’s all the data we have?
Peter still talks about the dietary guidelines being a disaster. He still believes in low-carbohydrate diets–going so far as to recently have Jason Fung on his podcast, which I believe as a podcaster purporting to promote science was a very serious mistake. But I think that such considerations are what’s driving his views on meat.
As far as no data weighing conclusively against meat, he is scientifically right. But that simply doesn’t matter much: when the data are insufficient, the point in scientific nutrition is not to be scientifically right in pointing out that no point of view is strictly speaking formally scientifically correct, but to help people.
So I think the reason Peter leans away from recommending lower meat intakes is that he still believes in the carbohydrate-insulin model. Otherwise, I don’t understand how someone interested in risk management can ignore the data–as imperfect as it is–on meat. So which way should the recommendations run?
It should run in the direction of best evidence: because that’s all that in many cases, scientific nutrition can achieve. If we are interested in the actual practice of nutrition and the actual helping of patients according to best evidence, “best evidence” and not “scientific definitiveness” is the approach we should all be taking. And even if carbohydrates are as bad as Peter seems to think, there is still the possibility of plant-based low-carbohydrate diets. The question is: in which direction does the net balance of risks weigh? And I think the answer, given current data, is crystal clear.