I have the following form for a joint distribution
$$ P(w, \lambda, \phi \vert y) = P(\phi) \times P(w \vert \lambda) \times P(\lambda) \times \prod_{i=1}^{N}P(y_i \vert w_i, \phi, \lambda) $$
I have some approximation for $w$ and what I am trying to do is update the parameters for the distributions for $\lambda$ and $\phi$. To take advantage of the conjugate prior properties, I have the following setup.
$\lambda$ is modelled as a Gamma distribution where $a$ and $b$ are the scale and rate parameters of the distribution respectively.
$$ P(\lambda; a, b) = \frac{b^a \lambda^{a-1} \exp{(-b\lambda)}}{\Gamma(a)} $$
The log-posterior on $\lambda$ can be then written as: $$ \log P(\lambda; a, b) = (a-1)\log(\lambda) -b\lambda + const(\lambda) $$
$\phi$ which is the noise precision is also modelled as a Gamma distribution and the log posterior can be written similarly as:
$$ \log P(\phi; s, t) = (s-1)\log(\phi) -t\phi + const(\phi) $$
The log transformed prior on $w$ (which depends on $\lambda$ is: $$ \log P(w \vert \lambda) = \frac{1}{2}(\log(\lambda \Lambda) - w^T(\lambda \Lambda)w) + const (\lambda, w) $$
This is just a zero mean MVN with the precision being defined as $\lambda \Lambda$.
Finally, the log-likelihood term can be written as:
$$ \log P(y \vert x, w, \phi) = \frac{N}{2}\log(\phi) -\frac{\phi}{2} \sum_{i=1}^{N}(y_i - t(x_i, w_i)) + const(w, \phi) $$
x and y are observed and t is a non-linear transformation parameterised by $w$, which is the parameter of interest.
So, what I wanted to do was for a given $w$ obtain the posterior marginal distribution for $\lambda$. Now my, log posterior is given as (ignoring constant terms for the moment):
$$ \mathcal{L} = \log P(\lambda; a, b) + \log P(\phi; s, t) + \log P(w \vert \lambda) + \log P(y \vert x, w, \phi) $$
So, I thought that we can condition on $w$ treating it as a constant and integrate over $\phi$ and we should get the parameters for $\lambda$ as the posterior should also be Gamma distributed due to conjugate properties. So, the posterior density on $\lambda$ is: $$ P(\lambda) = \mathcal{L}\:d\phi $$
Now, doing the integrating I get something complicated as:
$$ \phi((a-1)\log\lambda - b\lambda) + (s-1)\phi(log \phi - 1) -\frac{t\phi^2}{2} + \phi (\frac{1}{2}(\log(\lambda \Lambda) - w^T(\lambda \Lambda)w)) + \frac{N}{2}\phi(\log\phi -1) -\frac{\phi^2}{2}(y-t(x, w))^T(y-t(x, w)) $$
So basically it becomes something complicated and it is not clear to me at all how the posterior Gamma distribution can be inferred from it which makes me think I have made a mistake with my update strategy. Can someone comment whether updating like this should be feasible? i thought it should be as this is whole point of using conjugate priors but I am at a loss as to what I have done wrong. I am inclined to think that the problem lies in the step
$$ P(\lambda) = \mathcal{L}\:d\phi $$
but I am not sure what is the main issue.