Consider a hierarchical model. For each observatory unit $i$, we observe a vector of Data $D_i$, and we define some probability (density or mass) function $P(D_i|\theta_i)$, which is governed by a (possibly vector-valued) first-level parameter $\theta_i \in \mathbb{R}^k$. We assume that each $\theta_i$ is itself randomly distributed, with some density $P(\theta_i|\mu)$ governed by the (possibly vector-valued) parameter $\mu$. (If it simplifies the answer: I am personally only interested in cases where $\theta_i$, or a transformation of $\theta_i$, follows a normal distribution - in that case, the parameter $\mu$ may define mean and variance-covariance matrix in some suitable form). We now collect data from $n$ units, and wish to estimate $\mu$ via maximum-likelihood estimation. Most approaches I know use the estimator $\hat{\mu}=\underset{\mu}{\text{argmax }}L(\mu)$, where $$L(\mu) = \prod_{i = 1}^{n}P(D_i|\mu) = \prod_{i = 1}^{n}\int_{\mathbb{R}^k}P(D_i|\theta_i)P(\theta_i|\mu)d\theta_i .$$
So for each $i$, integration over the possible values of $\theta_i$ is necessary. I would like to know now, what are the exact disadvantages of instead jointly maximizing $$L(\mu,\theta_1,\dots,\theta_n)=\prod_{i = 1}^{n}P(D_i|\theta_i)P(\theta_i|\mu),$$ and taking as estimate for $\mu$ the respective component of the argument maximizing this function? Can we discern conditions under which this approach would give approximately the same estimator for mu than the first one? And conditions under which this approach is much worse? I am wondering because the second optimization problem looks rather easy to me to solve computationally, while the first demands sophisticated methods, such as expectation-maximization, Laplace-approximation of the integral, or similar.

This question is related, but it does not quite answer my question. I am only interested in finding an estimator for $\mu$, but I wonder what the exact cost is of "superfluously" also estimating each $\theta_i$.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.