A new arXival (of a Science paper) by Do et al. offers a positive example of how the contemporary best practice approach to Bayesian model building can be brought to bear on an interesting astrophysical dataset. The authors aim to investigate the preference for a general relativistic model vs a Newtonian (+special relativistic) model in a dataset of spectroscopic and astrometric observations of the star S0-2 (which orbits the supermassive black hole in our galactic centre). In building their model the authors consider a variety of possible forms for the instrumental noise, including the possibility of systematic biases in the measurements from particular instruments(*), using Bayes factors and a posterior predictive check to iterative towards a sensible representation of the data generating process. In doing so, they use a reliable marginal likelihood estimator: nested sampling; they note that the draws from the nested sampler (which are inevitably following a distribution broader than the posterior itself) allow for an approximation (without actually leaving one out) of the leave-one-out expected log probability density that doesn’t suffer the infinite variance of the equivalent importance sampling estimate from a standard posterior sample. Finally, they consider their priors in a way that explicitly acknowledges that uniform priors don’t necessarily mean free of influence on the result.
An interesting aspect of the analysis is that, after computing Bayes factors for weighting between the canonical models, the authors also estimate a parameter representing the proportion of general relativity signal in their observations. This looks a lot like the ‘hypothesis testing as mixture model estimation’ approach of Kamary et al., and seems to me a worthwhile addition as it gives yet another way to understand the preference of the data towards each model.
(*) Something that the fine structure dipole advocates (and indeed numerous other astronomical sub-communities) steadfastly refuse to believe can be examined through statistical modelling!!