# Mixed model idea and Bayesian method

In mixed model, we assume the random effects (parameters) are random variables that follow normal distributions. It looks very similar to the Bayesian method, in which all the parameters are assumed to be random.

So is the random effect model kind of special case of Bayesian method?

This is a good question. Strictly speaking, using a mixed model does not make you Bayesian. Imagine estimating each random effect separately (treating it as a fixed effect) and then looking at the resulting distribution. This is "dirty," but conceptually you have a probability distribution over the random effects based on a relative frequency concept.

But if, as a frequentist, you fit you model using full maximum likelihood and then wish to "estimate" the random effects, you've got a little complication. These quantities aren't fixed like your typical regression parameters, so a better word than "estimation" would probably be "prediction." If you want to predict a random effect for a given subject, you're going to want to use that subject's data. You'll need to resort to Bayes' rule, or at least the notion that $$f(\beta_i | \mathbf{y}_i) \propto f(\mathbf{y}_i | \beta_i) g(\beta_i).$$ Here the random effects distribution $g()$ works essentially like a prior. And I think by this point, many people would call this "empirical Bayes."

To be a true Bayesian, you would not only need to specify a distribution for your random effects, but distributions (priors) for each parameter that defines that distribution, as well distributions for all fixed effects parameters and the model epsilon. It's pretty intense!

• Really clear, straightforward answer. – D L Dahly Dec 15 '13 at 10:46
• @baogorek - a fairly robust default is Cauchy priors for fixed effects and half cauchy for variance parameters - not that "intense" - it just looks like penalised likelihood – probabilityislogic Dec 15 '13 at 13:12

Random effects are a way to specify a distributionial assumption by using conditional distributions. For example, the random one-way ANOVA model is: $$(y_{ij} \mid \mu_i) \sim_{\text{iid}} {\cal N}(\mu_i, \sigma^2_w), \quad j=1,\ldots,J, \qquad \mu_i \sim_{\text{iid}} {\cal N}(\mu, \sigma^2_b), \quad i=1,\ldots,I.$$ And this distributional assumption is equivalent to $$\begin{pmatrix} y_{i1} \\ \vdots \\ y_{iJ} \end{pmatrix} \sim_{\text{iid}} {\cal N}\left(\begin{pmatrix} \mu \\ \vdots \\ \mu \end{pmatrix}, \Sigma\right), \quad i=1,\ldots,I$$ where $\Sigma$ has an exchangeable structure (with diagonal entry $\sigma^2_b+\sigma^2_w$ and covariance $\sigma^2_b$). To Bayesianify the model, you need to assign prior distributions on $\mu$ and $\Sigma$.

If you're talking in terms of reproducing the same answers, then the answer is yes. The INLA (google "inla bayesian") computational method for bayesian GLMMs combined with a uniform prior for the fixed effects and variance parameters, basically reproduces the EBLUP/EBLUE outputs under the "simple plug in" gaussian approximation, where the variance parameters are estimated via REML.

I don't think so, I consider it part of the likelihood function. It's similar to specifying the error term follows a Normal distribution in a regression model, or a certain binary process can be modeled using a logistic relationship in a GLM.

Since no prior information, or distributions, are used I do not consider it Bayesian.

• No prior information used hey? How did you specify the functional form for the likelihood function then? :-D – probabilityislogic Dec 15 '13 at 12:18
• Some people argue that the distinction between likelihood and prior is somewhat artificial. – Christoph Hanck Apr 7 '15 at 4:34