Testing random effects, i.e. the null hyp that the variance is 0

What I believe today:

  • Supposed structure of the data can be used to decide whether you include a random effect. (By analogy, one would use a repeated measures ANOVA for repeated measures designs even if the between-subject variation was minimal.)
  • If you want to make a test, then use the log likelihood ratio test (LRT), not MCMC-based methods as none of the MCMC samples will have zero variance.
  • A significant p from LRT means you should definitely leave the random effect in.
  • A non statistically significant p might mean you can leave it out.
  • Though you can’t really trust the loglikelihood ratio to have a chi-square distribution; you’d be better off using numerical methods.