8
$\begingroup$

I'm estimating an ICC from 2 and 3-level hierarchical models using rstanarm. The simplest models are: y ~ (1|group) or y ~ time + (1|individual) + (1|group)

For the group variance parameters there is not much information in the dataset. I'm using the default rstanarm/stan_lmer priors.

enter image description here

My question is: given the posterior is so skewed, does it still make sense to report the mean of this parameter (or the ICC calculated from all the variance parameters, which is also skewed). The black lines in the figure show the mode (peak density), median and mean (from left to right).

I realise that no single statistic can adequately summarise the posterior, and will report the posterior density plot but I'm trying to get some intuition for how the different estimates would be interpreted. For example, would the mode here be overly-conservative because the prior location was zero and variance estimates are constrained to be >0?

EDIT: Many thanks for the comments, which have helped clarify somewhat. Thanks also for pointing out that I should calculate the ICC from the draws — I was doing this, but didn't make this clear in my question which only referred to the variance parameter. The problem remains though because the posterior for the ICC is similarly skewed.

Thanks to Ben for highlighting the different loss functions implied. This led me to these two pages on Bayes estimators and median unbiased estimators which expanded on that point. It also made it clear that, although appealing based on the posterior density plot, the mode is not as straightforward to interpret as it might seem.

It's very interesting to hear that the mode is most analogous to the MLE because in this case the mode is closest to previous estimates from the literature, which were mostly based on MLE estimators. My temptation to simplify to a single statistic is driven by the desire to simplify the story for a non technical audience, but it's obvious that the real story here is that that we know less than we previously thought.

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ There is nothing special about posterior distribution. Check stats.stackexchange.com/questions/200282/…, stats.stackexchange.com/questions/14089/…, stats.stackexchange.com/questions/14210/…, stats.stackexchange.com/questions/203233/…, stats.stackexchange.com/questions/96371/… etc. $\endgroup$
    – Tim ♦
    Aug 18, 2017 at 16:58
  • 3
    $\begingroup$ This is a good question if you are looking for a point estimate of the parameter based on the posterior distribution. In the spirit of maximum likelihood I would suggest using the mode. $\endgroup$
    – Michael R. Chernick
    Aug 18, 2017 at 21:36
  • 2
    $\begingroup$ If the ICC is what is of interest, then calculate the posterior distribution of the ICC by evaluating the ICC function at every draw from the posterior distribution of the parameters that you have. Then figure out how to adequately describe the posterior distribution of the ICC (I would probably just show the plot). The posterior distribution is special in the sense that the posterior mean minimizes expected loss when the loss function is squared error and the posterior median minimizes expected absolute error. The posterior mode is most analogous to a MLE but I don't consider that a virtue. $\endgroup$
    – Ben Goodrich
    Aug 18, 2017 at 22:25
  • 3
    $\begingroup$ Unless you're in a situation where you know which summary of the posterior is most relevant, is there any reason to report only one aspect of the posterior distribution? $\endgroup$
    – Glen_b
    Aug 19, 2017 at 0:33

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.