So … this was meant to be where I would post the description of the 6th (and final) statistical trend on my list to revolutionise astronomical data analysis, but I’ve been lost this week in a labyrinth of c++ coding; meaning that for now I have only a single random musings to put down on imaginary internet paper. Next week I’ll finish my list with a discussion of Bayesian design (hopefully).
Is the Plos One editorial/reviewing process as good as we need it to be?
A recent(ish) article by Burkert et al. (2014) came to my attention this week when it was twice posted to my Facebook newsfeed: once by a statistician, and once by a paleo dieter. The article describes a basic matched-sample analysis of self-reported health measures for four different dietary cohorts in the Austrian Health Interview Survey. In particular, the authors aim to examine the relationship between health and diet: with a focus on the question of how “restrictive and monotonous vegetarian diets” might convey health benefits, or alternatively, be a cause for poor health. The statistical analysis is of the straightforwards, frequentist-type typical of the public health literature, and seems (without checking their computations in detail) to have been done correctly. A surprising result, perhaps, is that the authors find a vegetarian diet to be associated with “higher incidences of cancer, allegories, and mental health disorders” leading them to conclude: “Therefore, public health programs are needed in order to reduce the health risk due to nutritional factors.”
At first I thought, ‘Oh dear, the classic correlation-causation blunder!’. Then in their discussion I noticed that they do acknowledge the limitations of their study: “no statements can be made whether the poorer health in vegetarians in our study is caused by their dietary habit or if they consume this form of diet due to their poorer health status. We cannot state whether a causal relationship exists, but describe ascertained associations.” That is, they admit that they have no justification for their conclusion, but have been allowed to state it anyway. What’s worse is that this has allowed a third party—“nutritionist”, Uwe Knop—to put out a press release only further ramping up the unjustified conclusion: [via google translate] “In view of the current findings, the unacceptable health promise of vegetarian lobbyists have far more questionable act”. And, hence, to the Daily Fail, Paleo forums, and various other grim recesses of the interwebs. [At least the NHS website is lending a sane voice to the discussion.]
Long story short, I was rather surprised that the Editor responsible for this article at Plos One allowed the conclusion to go through as it is written, given the clear lack of justification and great potential for mis-interpretation (deliberate or otherwise) … and in fact I am now less confident in Plos One overall as a forum for rigorous scientific publishing.