To kick off this blog on a positive note I thought I might look into my crystal ball and prognosticate on what I see as the (hopeful) future of astrostatistics.
Experimental Design Perhaps the most under-utilised area of modern statistical methodology relevant to astronomical research is that of experimental design; and in particular, Bayesian experimental design. While the Fisher matrix formalism of ‘classical’ statistics has been of indisputable utility in the past for teams of observational cosmologists aiming to optimize the design of weak lensing and CMB experiments for parameter constraint under a particular cosmological model, there remains much more that could be done to advance such analyses within the Bayesian design framework: most notably perhaps the introduction of adaptive designs and the introduction of sophisticated utility functions balancing goals of cost, parameter learning, and model choice. The power of this approach is perhaps most readily appreciable with reference to state-of-the-art clinical trials; where Bayesian methodologies allow for ‘more ethical’ designs in which treatments inferred to be harmful or ineffective may be dropped early from multi-arm studies, for instance. A hypothetical application in the astronomical domain would be an adaptive design for assigning the observational time per filter (supposing one is attached to the final instrument) for Euclid observations depending on how significant the impact of galaxy color gradient on shear measurement proves to be.
[Loredo & Chernoff have also speculated on this topic at some length in their contribution to “Statistical Challenges of Astronomy”.]
Approximate Bayesian Computation A pet project of mine has been to introduce Likelihood-Free Inference or Approximate Bayesian Computation (ABC) to the astronomical community (see Cameron & Pettitt 2012; and from another team, Weyant et al. 2012). ABC offers a rigorous framework for parameter inference (and even, now, model selection) in the case that one cannot readily compute the likelihood function of their proposed model, yet can easily simulate mock datasets from it. The core of the ABC methodology is the intuitive step of ‘optimization’ with respect to an observed-to-simulated data discrepancy distance; and indeed ABC-like algorithms are (re-)invented at regular intervals in the astronomical literature (e.g. Kashyap et al. 2002; Sana et al. 2012). The point of conducting such analyses within the ABC framework is that it offers a powerful suite of techniques for optimization of the inference process as well as important theoretical results concerning the reliability and asymptotic behaviour of ABC-based estimates. With many astronomical datasets subject to complex selection effects (which are difficult to derive likelihood for, but easy to simulate) the potential for ABC in astronomical research seems, to me, quite profound.
Principled Model Selection Bayesian model selection (BSM) is now a regular feature in cosmological and exoplanet research and the astronomical community has been remarkably active in embracing and developing the latest computational techniques in this field (e.g. Feroz & Hobson 2009; Weinberg 2012). [Though I have some reservations concerning the Weinberg approach, which I hope to discuss here in a later post!] However, the application of model selection techniques in astronomy at present often lacks the rigour suggested by such computational sophistication. In particular, although sometimes acknowledged with a brief mention the issue of prior sensitivity is rarely addressed in quantitative detail such as via recomputation of the Bayes factor under the doubling and halving of key hyperparameters (cf. Kass & Raftery 1995; and see Cameron & Pettitt 2013 for related examples). There is also often something of an mis-understanding regarding the interpretation of Bayes factors; their representation of the posterior odds ratio being neglected in a fruitless quest for calibration as frequentist type confidence intervals and/or by a universal master scale (typically Jeffreys). One can only hope & anticipate therefore that as the use of BSM methodology in astronomy becomes even better established its application will also become better principled; and hopefully therefore its impact on scientific decision making can be made all the more effective.