I'm struggling with hierarchical bayesian modeling. I need to derive a full likelihood function for the given hierarchical structure of the model.

$a_{it}|\lambda_i\sim TN(\lambda_i,\beta)$

$x_{it}|\lambda_i,\xi_i,a_{it}\sim Pr\{x_{it}\}$

$\lambda_{i}|\theta,\tau\sim N(\theta,1/\tau)$

$\xi_{i}|\mu,\omega\sim N(\mu,1/\omega)$ with priors for $\theta, \tau,\mu,\omega$ given as well.

Here, the parameters to be estimated are $\theta, \tau,\mu,\omega,\{\lambda_{it}\},\{\xi_{it}\}$ and the data observation is $\{\{a_{it}\},\{x_{it}\}\}$.

The question is, what is the full likelihood function for this model. What I think is $P(\{a_{it}\},\{x_{it}\}|\theta, \tau,\mu,\omega,\{\lambda_{it}\},\{\xi_{it}\})=P(\{x_{it}\}|\lambda_i,\xi_i,a_{it})*P(\lambda_{i}|\theta,\tau)*P(\xi_{i}|\mu,\omega)$

Is this the right way? Thanks in advance!

  • $\begingroup$ Welcome to Cross Validated, Liz. Are you sure it is the likelihood you need to determine? Usually, you are looking for the full posterior density. But if it is indeed the likelihood you are looking for, then what you have written looks pretty close. I notice that $Pr\{x_{it}\}$ is not apparently conditional on $a_{it}$ or $\xi_i$. Do you mean that? It looks like $\xi_i$ falls out of the likelihood altogether! Also don't forget to take the product over the $i$s. Your final expression should be in terms of the RHS of your definitions, so more $N$'s please. $\endgroup$ – Peter Leopold May 27 '19 at 22:10
  • $\begingroup$ Of course, the product over the $i$'s assumes independence of acquired data points, which perhaps you don't have, and in any event, your notation is fine without the product, so I withdraw that comment. $\endgroup$ – Peter Leopold May 27 '19 at 22:24
  • $\begingroup$ @PeterLeopold Thank you so much! yes $Pr\{x_{it}\}$ is a long function of $a_{it}$ and $\xi_i$. I just skipped it for the space. Also data points are assumed to be independent. So I'd just need to add $\Pi_i$ in front of the whole equation? $\endgroup$ – liz May 27 '19 at 23:03

For independent observables $\{x_{it}\}$ and $\{a_{it}\}$, I would write the joint conditional likelihood as

$$ \begin{aligned} P(\{x_{it}\},\{a_{it}\}|\{\lambda_i\},\{\xi_i\},\beta) = \prod_i TN(a_{it}|\lambda_i,\beta) Pr(x_{it}|\lambda_i,\xi_i) \end{aligned} $$

Some comments:

  1. Notice that I've only included mention of the parameters whose values are explicitly needed to specify the likelihood: $\beta, \{\lambda_j\},$ and $\{\xi_k\}$. I've dropped any mention of the hyperparameters. This is because the hyperparameters specify the prior distributions for the parameters, and since we are only writing a likelihood, we don't need the priors.
  2. I'm following your indication in the comment that $Pr(x_i)$ should be made conditional the list of $\lambda$s and $\xi$s.
  3. I've assumed that the observations are statistically independent of each other, so $P(x_1,x_2,\dots,n_n)=\prod_i P(x_i)$.
  4. The use of $i$ to index the $x, \xi, $ and $\lambda$ variables implies that there is a 1:1 relationship between them. I have preserved that, but don't immediately know what to make of it. There seems to be two parameters for each data point. That's not good.
  5. I don't know what the $TN$ function is. Is it just the "normal" normal $N$?
  6. It is not clear what the subscript $t$ means. I've preserved it since it doesn't hurt, but it isn't being averaged over, so I'm not sure what it is doing in the model.
  7. It doesn't make sense for the independent observables $x_{it}$ and $a_{it}$ to be contingent on each other, but this is what you are hinting with your expression $x_{it}|a_{it}, \lambda_i, \xi_i$. I dropped the $a_{it}$ from the conditional.

On the other hand, if you wanted to specify the posterior distribution of the parameters, you would have to include the priors for the parameters, complete with their hyperparameters. But that seems not to be the task.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.