I was wondering if it is possible to use conjugacy "locally" in a Bayesian hierarchical model. Locally is most likely not the right word but I'll explain the problem.
For example, the likelihood of the data $X$ given variables $Y$ and $Z$ is $P(X \mid Y,Z)$. The prior is defined as $P(Y,Z)=P(Z \mid \theta)P(\theta)P(Y)$ and it is not conjugate to the likelihood. $P(\theta)$ is however conjugate to $P(Z\mid \theta)$ and in my case $\theta$ is of no interest. I need to estimate the posterior distribution $P(Y,Z,\theta \mid X) \propto P(X \mid Y,Z)P(Z \mid \theta)P(\theta)P(Y)$ using MCMC, keeping in mind that I am only interested in $Y$ and $Z$.
Is it possible to calculate $\tilde P(\theta \mid Z)$ at each MCMC iteration using the conjugacy relationship between $P(Z \mid \theta)$ and $P(\theta)$ and therefore sample from $P(Y,Z \mid X) \propto P(X \mid Y,Z)\tilde P(\theta\mid Z)P(Y)$ instead?