I'm working on a course problem,

Suppose that $\textbf{x}=\{x_1,\dots,x_n\}$ and $\textbf{y}=\{y_1\dots,y_m\}$ are independent random samples from $\text{N}(x|\mu_x,1/\lambda)$ and $\text{N}(y|\mu_y,1/\lambda)$, respectively. Using the (improper) prior $$\pi(\mu_x,\mu_y,\lambda)=\lambda^{-1},$$ prove that $\mu_x$ and $\mu_y$ are not independent a posteriori and find $\text{Cov}[\mu_x,\mu_y|\textbf{x},\textbf{y}]$.

The given solution begins,

The likelihood can be written as $$\text{L}(\mu_x,\mu_y,\lambda|\textbf{x},\textbf{y})\propto\lambda^{\frac{n+m}{2}}\exp\left(-\lambda\left(\frac{n}{2}(s_x^2+(\bar{x}-\mu_x)^2)+\frac{m}{2}(s_y^2+(\bar{y}-\mu_y)^2)\right)\right)$$ thus the joint posterior is $$\pi(\mu_x,\mu_y,\lambda|\textbf{x},\textbf{y})\propto\lambda^{\frac{n+m}{2}-1}\exp\left(-\lambda\left(\frac{n}{2}(s_x^2+(\bar{x}-\mu_x)^2)+\frac{m}{2}(s_y^2+(\bar{y}-\mu_y)^2)\right)\right)$$ and the marginal for the means is $$\pi(\mu_x,\mu_y|\textbf{x},\textbf{y})\propto\left(1+\frac{1}{ns_x^2+ms_y^2}(n(\bar{x}-\mu_x)^2+m(\bar{y}-\mu_y)^2)\right)^{-\frac{n+m}{2}}.$$

I see we have \begin{align}\pi(\mu_x,\mu_y,\lambda|\textbf{x},\textbf{y})=\frac{\Gamma(a)}{b^a}\text{Ga}(\lambda|a,b),\text{ with } & a = \frac{n+m}{2}, \\ & b = \frac{n}{2}(s_x^2+(\bar{x}-\mu_x)^2)+\frac{m}{2}(s_y^2+(\bar{y}-\mu_y)^2),\end{align} but I can't follow the manipulation of fractions in \begin{align}b&=\frac{n}{2}(s_x^2+(\bar{x}-\mu_x)^2)+\frac{m}{2}(s_y^2+(\bar{y}-\mu_y)^2) \\ &=1+\frac{1}{ns_x^2+ms_y^2}(n(\bar{x}-\mu_x)^2)+(m(\bar{y}-\mu_y)^2)\end{align} between the last two steps. Would someone mind clarifying?

  • $\begingroup$ @Zhanxiong Thanks for fixing the $\mu_x$ for $\mu_y$ typo. The $x_i$ and $y_i$ were accurately quoted, but I don't feel confident in agreeing with the original or with your edit. $\endgroup$
    – mjc
    Jan 17 at 22:02
  • $\begingroup$ My edit is based on the convention of "$X_1, \ldots, X_n \sim F(x)$" (CDF) or "$f(x)$" (pdf), but seldom saw the notation $X_1, \ldots, X_n \sim F(x_i)$, though it is not particularly incorrect. It's OK for you to revert my edit to follow your original source. $\endgroup$
    – Zhanxiong
    Jan 17 at 22:09
  • 1
    $\begingroup$ The support for the prior should be given. Presumably it's on the positive half line for $\lambda$ and on the real line for the other two components, but that's not the only possibility. The two expressions you give for $b$ are not equivalent, but that doesn't of itelf imply an error was made in the original, just in your premises that there's an equality between those expressions. $\endgroup$
    – Glen_b
    Jan 17 at 22:22
  • $\begingroup$ @Glen_b "The two expressions you give for $b$ are not equivalent" is useful. Is the difference somehow made up by the final proportionality? I thought the interior of the brackets being 'bound up' by the index $-\frac{n+m}{2}$ would prevent that. $\endgroup$
    – mjc
    Jan 17 at 22:30
  • 2
    $\begingroup$ Consider where the "$1\ +\ $" must come from: a _constant_ $(ns_x^2+ms_y^2)/2$ has been factored out of the argument of $\exp.$ $\endgroup$
    – whuber
    Jan 17 at 23:09

1 Answer 1


To find the marginal, you need to integrate out the $\lambda$ term in


To do this, let's simplify it by replacing that constant positive multiple of $\lambda$ by a single name, such as $z,$ and compute (since evidently $\lambda\gt 0$)

$$\int_0^\infty \lambda^{\frac{n+m}{2}-1}\exp\left(-\lambda z\right)\,\mathrm dz = z^{-\frac{n+m}{2}}\int_0^\infty (\lambda z)^{\frac{n+m}{2}}\exp\left(-\lambda z\right)\frac{\mathrm d(\lambda z)}{\lambda z} = \frac{z^{-\frac{n+m}{2}}}{\Gamma\left(\frac{n+m}{2}\right)}.$$

(This calculation shows up so often in statistics and mathematics that it's worth remembering.)

We need to identify the portion of this result that depends on $(\mu_x,\mu_y),$ but any multiplicative factors not involving these variables can be ignored.

Begin by ignoring the $1/\Gamma((n+m)/2)$ factor and focus on $z$ itself before raising it to the $-(n+m)/2$ power:

$$\begin{aligned} z &= \frac{n}{2}(s_x^2+(\bar{x}-\mu_x)^2)+\frac{m}{2}(s_y^2+(\bar{y}-\mu_y)^2) \\ &= \frac{n}{2}s_x^2 + \frac{m}{n}s_y^2 + \frac{n}{2}(\bar{x}-\mu_x)^2 + \frac{m}{2}(\bar{y}-\mu_y)^2 \\ &= \left(\frac{n}{2}s_x^2 + \frac{m}{n}s_y^2\right)\left(1+\frac{1}{ns_x^2+ms_y^2}(n(\bar{x}-\mu_x)^2+m(\bar{y}-\mu_y)^2)\right). \end{aligned}$$

This expresses $z$ in the form $z = (C)\,(f(\mu_x,\mu_y)).$ Its $-(n+m)/2$ power therefore has a constant factor of $C^{-(m+n)/2},$ which can be ignored. Thus

$$z^{-\frac{n+m}{2}}\ \propto\ f(\mu_x,\mu_y)^{-(n+m)/2} = \left(1+\frac{1}{ns_x^2+ms_y^2}(n(\bar{x}-\mu_x)^2+m(\bar{y}-\mu_y)^2)\right)^{-(n+m)/2},$$


The motivation for this last move is, of course, to steer the expression towards something that looks like a Student-t density.

  • 3
    $\begingroup$ @mjc It's worth working through this approach on your own; many similar manipulations are going to crop up again and again in a Bayesian context (and indeed, to an extent outside of Bayesian contexts). $\endgroup$
    – Glen_b
    Jan 18 at 1:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.