Other distribution for random effects than Normal

Question

In frequentist statistics, we always assume that the distribution of the random effects is Normal due to the fact that it makes computations more easy. In Bayesian statistics, we can easily change this assumption and use different distributions. In my Bayesian analysis, I want to test whether other distributions are more appropriate. I already fitted a t-distribution and a log-normal distribution. But I was wondering what other distributions can be considered? I was thinking about an exponential distribution, but then my random effects can only be positive. Which distributions can I use or are most often used next to the Normal distribution?

Answer 1

Suppose $n=100$ independent observations $X_1, X_2, \dots, X_{100}$ are taken from an exponential distribution and we wish to find a 95% confidence interval (CI) for the population mean $\mu,$ where $E(X_i) = \mu.$

Then one can show that $\frac{\bar X}{\mu} \sim \mathsf{Gamma}(\mathrm{shape}=n, \mathsf{rate}=n).$ Then if $L$ and $U$ cut 2.5%, respectively, from the lower and upper tails of $\mathsf{Gamma}(n,n),$ we have

$$P\left(L \le \frac{\bar X}{\mu}\le U\right) = P\left(\frac{\bar X}{U} \le \mu \le \frac{\bar X}{L}\right) = 0.95,$$ so that a 95% CI for $\mu$ is of the form $\left(\frac{\bar X}{U},\, \frac{\bar X}{L}\right).$

In R, the exponential distribution is parameterized by the rate $\lambda = 1/\mu.$ In particular, suppose we have a vector x of observations from $\mathsf{Exp}(\lambda = 0.02),$ which has $\mu = 50.$

set.seed(2020)
x = rexp(100, 0.02)
summary(x)
    Min.   1st Qu.    Median      Mean   3rd Qu.      Max. 
 0.04116  13.32663  37.43140  52.98382  68.58117 293.37596 

Then a 95% CI for $\mu$ based on these $n=100$ observations is $(43.96,65.12).$ Notice that $\bar X$ is not exactly at the center of this CI. (This is one of the 'lucky' 95% of situations in which the CI covers $\mu.$)

mean(x)/qgamma(c(.975,.025), 100, 100)
[1] 43.95941 65.11949

Falsely assuming normality. If we had made the incorrect assumption that the data in x were normal, then we might have tried to use a t CI as an interval estimate of $\mu,$ to get $(41.75, 64,21),$ as below: This interval also happens to include $\mu=50,$ but it is based on a false assumption, so we cannot expect that such an interval will cover $\mu$ in 95% of situations--not even when the number of observations is as large as $n=100$ so that $\bar X$ itself may be roughly normally distributed, in view of the Central Limit Theorem.

t.test(x)$conf.int
[1] 41.75369 64.21395
attr(,"conf.level")
[1] 0.95

Note: The following simulation in R illustrates that for $n=20$ the correct gamma CI covers $\mu=50$ in almost exactly 95% of the 100,000 iterations, whereas the incorrect t CI covers $\mu$ in only about 92% of them.

set.seed(2020)
m= 10^5;  n = 20
x = rexp(m*n, .02)
DTA = matrix(x, nrow=m) # m x n matrix each row a sample
a = rowMeans(DTA)       # 100,000 sample means
s = apply(DTA, 1, sd)   # 100,000 sample SDs

LCL.g = a/qgamma(.975,n,n)
UCL.g = a/qgamma(.025,n,n)
COV.g = (LCL.g < 50) & (UCL.g > 50)
mean(COV.g)
[1] 0.95003

LCL.t = a+qt(.025,n-1)*s/sqrt(n)
UCL.t = a+qt(.975,n-1)*s/sqrt(n)
COV.t = (LCL.t < 50) & (UCL.t > 50)
mean(COV.t)
[1]  0.91857