0
$\begingroup$

Currently I am exploring topics for my undergrad thesis. Although I took a course in Bayesian statistics, I am not yet sure how to proceed in finding the posterior in the following case.

I have a d-dimensional prior distribution $\theta \sim \Pi =N(\mu, \Sigma)$. Define $\lambda = sum(\theta)=\theta'\iota$. I have the model distributions $Y|\theta\sim N(\lambda, \tau^2)$. This is clearly a one dimensional random variable, with $\tau$ fixed. How do I compute the posterior distribution $\theta|Y=y$? If I am correct, this should be a multivariate normal distribution again. I would already be really helped by a formula for d=2, or d=3, but for arbitrary d would be amazing.

$\endgroup$

1 Answer 1

4
$\begingroup$

Since we seek for a PDF in terms of $\theta$, terms other than $\theta$ can be fused into normalizing constant, which results in the following proportionality argument:

$$p(\theta|Y) = \frac{p(Y|\theta)p(\theta)}{p(Y)}\propto p(Y|\theta)p(\theta)$$

Substituting into RHS yields (let $m=[1,1,...,1]^T\rightarrow \lambda=m^T\theta$): $$\begin{align}p(\theta|Y=y) &\propto \exp\left(-\frac{(y-m^T\theta)^2}{2\tau^2}\right)\exp\left(-\frac{1}{2}(\theta-\mu)^T\Sigma^{-1}(\theta-\mu)\right)\\ &\propto \exp\left(y\frac{m^T\theta}{\tau^2}-\frac{\theta^Tmm^T\theta}{2\tau^2}-\frac{\theta^T\Sigma^{-1}\theta-2\mu^T\Sigma^{-1}\theta}{2}\right) \\ & \propto \exp\left(-\frac{1}{2}\theta^T\left(\Sigma^{-1}+\frac{mm^T}{\tau^2}\right)\theta +\left(\frac{ym^T}{\tau^2}+\mu^T\Sigma^{-1}\right)\theta\right)\end{align}$$

And, this expression is in MV normal form, i.e. it is proportional to $$\exp\left(-\frac{1}{2}(\theta-\mu_{\theta})^T\Sigma_{\theta}^{-1}(\theta-\mu_{\theta})\right)\propto\exp\left(-\frac{1}{2}\theta^T\Sigma_{\theta}^{-1}\theta+\mu_{\theta}\Sigma_{\theta}^{-1}\theta\right)$$ So, we're just going to match term by term and obtain the covariance matrix and mean vector of $\theta$ posterior: $$\Sigma_{\theta}=\left(\Sigma^{-1}+\frac{mm^T}{\tau^2}\right)^{-1}, \mu_{\theta}=\left(\frac{ym^T}{\tau^2}+\mu^T\Sigma^{-1}\right)\Sigma_{\theta}$$

Note: $mm^T$ is actually a matrix of only 1's.

$\endgroup$
5
  • $\begingroup$ Thanks a lot! That seems like what I was looking for! I got lost after the first proportionality step, thanks for helping me out! Now I will look at whether this is indeed implementable within the thesis setting that I was looking at! $\endgroup$
    – J. Dekker
    Apr 8, 2019 at 20:09
  • $\begingroup$ Do you still need help understanding? I'm asking because you accepted the answer. $\endgroup$
    – gunes
    Apr 9, 2019 at 7:56
  • $\begingroup$ No, I understand it now! Thanks for asking. Turns out my matrix algebra skills were just too rusty after not doing much of that for a year! $\endgroup$
    – J. Dekker
    Apr 9, 2019 at 7:59
  • $\begingroup$ I just start doubting about the last line.. are you sure that we are only dealing with matrix inverses? I feel that the last Sigma_theta inverse should just be a Sigma_theta? $\endgroup$
    – J. Dekker
    Apr 18, 2019 at 18:53
  • $\begingroup$ And, the inside should be $\mu^T$ (now changed it) $\endgroup$
    – gunes
    Apr 18, 2019 at 18:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.