I noticed a paper by Johnson et al. presenting cosmological constraints from the 6dF galaxy velocity survey on astro ph today. What caught my eye here was their discussion of the “hyper-parameter method” for combined inference from a collection of individual datasets that “may contain *systematic errors*, requiring them to be re-weighted in the likelihood analysis”. The so-called “hyper-parameter method” introduced by Lahav et al. (2000) and Hobson et al. (2002) proposes to solve this problem by supposing the scale of the random errors in each experiment to be a free (hyper-)parameter, assigned a Jeffrey’s prior and marginalised out in the final analysis. My problem with this idea is that it involves *a highly non-standard definition of systematic error*: which would more conventionally be defined as a measurement bias that cannot be reduced to zero, like that of the random error, simply by averaging over more and more data. While we might (given due cause) still introduce a hyper-parameter to model uncertainty in the scale of our random error estimates, a far more principled approach to the problem of reconciling surveys under possible systematic errors is to follow the mixed effects model approach popular in clinical meta-analysis studies (for which a contemporaneous equivalent to the Lahav et al. reference would be that of Sutton & Abrams 2001). Oh, and I also have a discussion of these ideas in my fine structure II manuscript …

- Follow Another Astrostatistics Blog on WordPress.com
### View Posts by Category

ABC Astronomy Astrostatistics Bad Science Big Data Bayes Dirichlet Processes Fourier analysis Gaussian Processes Infinite-Dimensional Inference INLA Marginal Likelihood Estimation Measure Theory Non-Parametric Order Statistics Particle MCMC Quantile Regression Rants Semi-Parametric Statistics Uncategorized Zoology, Epidemiology, & Clinical Trials### Archive

- August 2017
- July 2017
- June 2017
- May 2017
- April 2017
- March 2017
- February 2017
- January 2017
- December 2016
- November 2016
- October 2016
- September 2016
- August 2016
- July 2016
- March 2016
- February 2016
- January 2016
- December 2015
- October 2015
- September 2015
- August 2015
- July 2015
- June 2015
- May 2015
- April 2015
- March 2015
- February 2015
- January 2015
- December 2014
- November 2014
- October 2014
- September 2014
- August 2014
- July 2014
- June 2014
- May 2014
- April 2014
- March 2014
- February 2014
- January 2014
- December 2013
- November 2013
- October 2013
- September 2013
- August 2013
- July 2013
- June 2013
- May 2013
- April 2013