I noticed this paper by Pinho & Martins on astro ph today (accepted to Phys Lett B) concerning the alleged spatial variation of the fine structure constant; I say alleged but from reading this paper you’d think it was a settled debate with only the precise functional form of the final spatial model left to be decided. In this latest instalment the authors propose to consider what updates can be made to the parameters of the spatial dipole model given 10 new quasar absorption datapoints (along 7 unique sight lines) drawn from post-Webb et al. studies published in the recent literature, with “the aim of ascertaining whether the evidence for the dipolar variation is preserved”. Which, since they don’t consider the possibility of systematic errors* in the Webb et al. dataset, it is … since 10 data points with slightly lower standard measurement errors—and supposedly lower systematic errors—cannot trump the couple of hundred original measurements used in the Webb et al. analysis.
*Here I pause to note that Pinho & Martins propagate Webb/King et al.’s obtuse definition of “systematic uncertainties” as being strictly zero-mean random errors, whereas usually systematic errors are taken to include the possibility of an unknown bias inherent to the instrument or technique. That is, according to the canonical definition, if my bathroom scales have a systematic error of +/- 0.1 kg I cannot hope to learn my true weight to better than a +/- 0.1 kg accuracy no matter how many times I weigh myself and average the measurements. In order to improve the accuracy I would need to average measurements taken with many different bathroom scales. (Replace bathroom scales with telescope+instrument pairings.)
Problematically the authors don’t investigate the new data in terms of hypothesis testing, which would have been a worthwhile approach since the Webb/King et al. model makes quite specific predictions for along these sight lines. Since the dataset is so small I was easily able to code this test up (see plot below) and compute a Bayes factor comparing the marginal likelihood of the new data points under the dipole model (marginalising over the posterior parameter uncertainties from the King et al. paper) against that of the null ( everywhere). The result is a Bayes factor of 3.9 in favour of the null; obviously not strong evidence (which it no surprise given the sample size) but also obviously not in any way strengthening the dipole hypothesis. So, more people compliment the emperor on his wonderful new clothes and the charade continues …
(Blue is predicted by the dipole model; magenta is observed. These error bars are 95% credible intervals for the predictions and 2 the standard errors [i.e., also ~95%] for the observations.)