Consider the following model $$\left \{ \begin{array}{l} y_i = x_i\beta + z_ib + \varepsilon_i,\\\\ b_i \sim \mathcal N(0, \Sigma), \quad \varepsilon_i \sim \mathcal N(0, \sigma^2), \end{array} \right.$$


\begin{equation} \Sigma = \begin{bmatrix} \sigma_1^2 & \rho\sigma_1\sigma_2 \\ \rho\sigma_1\sigma_2 & \sigma_2^2 \end{bmatrix}. \end{equation}

The log-likelihood function for this model is given by $$\begin{eqnarray} \ell(\theta) & = & \sum_{i = 1}^n p(y_i; \theta)\\ & = & \sum_{i = 1}^n \int p(y_i, b_i; \theta), \\ & = & \sum_{i = 1}^n \int p(y_i \mid b_i; \theta) \, p(b_i; \theta) \; db_i, \end{eqnarray}$$

where $\theta = (\beta, \sigma, \sigma_1, \sigma_2, \rho)$ denotes the parameters of the model.

Suppose I fix $\theta = \hat{\theta}$, and want to calculate the empirical Bayes estimates $\hat{b}_i = \text{argmax}_b \{\log p(y_i, b_i; \hat{\theta})\}$.

Is $\log p(y_i, b_i; \hat{\theta})$ considered a log-likelihood? I understand that the $b_i$ are random effects and not parameters, but they are unknown quantities. Would it be wrong to term this quantity a log-likelihood?


It's a little bit semantics. Namely, to do empirical Bayes you need to write down the posterior distribution of the random effects $b_i$ given the data $y_i$ and (the maximum likelihood) estimates of the parameters $\hat \theta$, i.e.,

$$p(b_i \mid y_i, \hat \theta) \propto p(y_i \mid b_i, \hat \theta) p(b_i \mid \hat \theta).$$

Now in this expression, the likelihood is the first term $p(y_i \mid b_i, \hat \theta)$ and the prior the second term $p(b_i \mid \hat \theta)$. Hence, to find the modes $\hat b_i$ of this posterior, you need to find the mode of the combined likelihood and prior terms, which is equivalent to finding the mode of $\log p(y_i, b; \hat \theta)$ wrt $b$.

With that being said though, these two combined terms have been called a likelihood. An example of this is the H-likelihood approach for fitting mixed models.

  • $\begingroup$ This makes sense. Although, from a Bayesian perspective, I find it somewhat confusing that the terms "posterior", and "prior" are used. $\endgroup$
    – JLee
    Nov 29 '19 at 12:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.