1
$\begingroup$

Absolutely stuck on section A of this question I found. I've been trying over 3 hours and still failing, can anyone here guide me through?

Suppose a random sample $x = (x_1, x_2,\ldots, x_n)$ is taken from a normal N($\Theta$, 1) distribution. It is desired to estimate the mean $\Theta$. A normal distribution with zero mean and variance $\frac{1}{t^2}$ is used as a prior distribution for $\Theta$.

(a) Show that the posterior density $\pi(\Theta|x)$ satisfies $$ \pi (\Theta |x)\propto\exp\left \{ -\frac{1}{2}\left ( \Theta ^{2}\left ( n +t^{2} \right ) - 2n\mu \Theta \right ) \right \} $$

where

$$ \mu = \frac{1}{n}\sum_{i=1}^{n}x_{i} $$

By writing this posterior density in the form

$$ \pi (\Theta |x)\propto exp\left ( -\frac{\left ( \Theta -m \right )^{2}}{2\nu ^{2}} \right ) $$

deduce the posterior distribution of $\Theta$.

(b) Using your posterior distribution for $\Theta$, what is your estimate for $\Theta$? By recalling that $X_i - N(\Theta, 1)$, obtain the mean and variance of your estimate. (c) Discuss what happens to your estimate (i) if $n$ is large, (ii) if $t$ is large, (iii) if $t$ is small. (d) Discuss why someone might choose the case (i) large $t$, (ii) small $t$.

$T_{post} = T_{prior} + T_{data}$ but I'm not sure how to utilize this.

For Ci. is it correct that the estimate will become closer to the sample mean when $N$ is large? Cii. make $t$ converge towards the mean? Ciii. the larger the ratio it is towards the estimate?

$\endgroup$
2
  • $\begingroup$ Are you using the fact that the posterior density is proportional to that quantity? i.e. $$\pi (\Theta |x) \propto \exp\left \{ -\frac{1}{2}\left ( \Theta ^{2}\left ( n +t^{2} \right ) - 2n\mu \Theta \right ) \right \}$$ $\endgroup$
    – user25658
    Sep 3 '13 at 20:00
  • $\begingroup$ Check if this helps: stats.stackexchange.com/questions/39261/… $\endgroup$
    – Zen
    Sep 4 '13 at 2:02
1
$\begingroup$

Solution for part A:

\begin{align*} \pi(\theta|x)&\propto\pi(x|\theta)\pi(\theta)\\ &\propto\exp\left\{-\frac{1}{2}\sum^n_{i=1}(x_i-\theta)^2\right\}\exp\left\{-\frac{t^2}{2}\theta^2 \right\}\\ &\propto\exp\left\{-\frac{1}{2}\sum^n_{i=1}(x_i-\theta)^2-\frac{t^2}{2}\theta^2 \right\}\\ &\propto\exp\left\{-\frac{1}{2}\left[\sum^n_{i=1}(x_i-\theta)^2+t^2\theta^2\right] \right\}\\ &\propto\exp\left\{-\frac{1}{2}\left[\sum^n_{i=1}(x_i^2-2x_i\theta+\theta^2)+t^2\theta^2\right] \right\}\\ &\propto\exp\left\{-\frac{1}{2}\left[-2\theta\sum^n_{i=1}x_i+n\theta^2+t^2\theta^2\right] \right\}\\ &\propto\exp\left\{-\frac{1}{2}\left[-2\theta n\mu+n\theta^2+t^2\theta^2\right] \right\}\\ &\propto\exp\left\{-\frac{1}{2}\left[-2\theta n\mu+\theta^2(n+t^2)\right] \right\}\\ &\propto\exp\left\{-\frac{1}{2}\left[\theta^2(n+t^2)-2\theta n\mu\right] \right\}\\ \end{align*}

So all you have to do is complete the square and you will obtain the kernel for another normal distribution (to complete the problem)

$\endgroup$
6
  • $\begingroup$ er, it's actually suppose to be X with a line on top. I must've typed the wrong thing accidentally, but it wouldn't change anything right? But any tips for the rest of the question? Is Ci etc correct? $\endgroup$
    – Kei Cheung
    Sep 3 '13 at 20:23
  • 2
    $\begingroup$ self-study questions look for guidance instead of detailed answers. You could, e.g., skip from the second to the last line (omitting two mistakes) by noting the argument of $\exp$ is a quadratic function of $\theta$. Its first two coefficients can be read right off: the coefficient of $\theta^2$ is evidently $-n/2$ from the sum plus $-t^2/2$ and the coefficient of $\theta$ can be found by letting $\theta=0$ in the derivative $\sum_i(x_i-\theta)-t^2\theta$, giving $n\mu$ (not $\mu$). The constant coefficient (not involving $\theta$) will be absorbed in the constant of proportionality. $\endgroup$
    – whuber
    Sep 3 '13 at 20:48
  • $\begingroup$ whuber, is there anything you can guide me to read up. I have troubles following your explanation such as "like he coefficient of θ2 is evidently −n/2 from the sum plus −t2/2". $\endgroup$
    – Kei Cheung
    Sep 3 '13 at 21:00
  • 1
    $\begingroup$ I hate to say it, but this is elementary algebra: you just look at where $\theta$ can be multiplied by $\theta$ and add up all the coefficients. By examining $(x_i-\theta)^2$ you know the only way to obtain $\theta^2$ comes from multiplying $-\theta$ by $-\theta$, giving a coefficient of $1$; there are $n$ of these in the sum; and that sum is multiplied by $-1/2$: altogether that gives a coefficient of $-n/2$. To this is added the coefficient $-t^2/2$ that explicitly appears. (The derivative trick is elementary too (it doesn't really use Calculus) but is not usually taught in high schools.) $\endgroup$
    – whuber
    Sep 3 '13 at 22:41
  • $\begingroup$ I edited my arithmetic mistake. $\endgroup$
    – user25658
    Sep 3 '13 at 22:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.