# Marginal distribution of normal random variable with a normal mean

I have a question about calculation of conditional density of two normal distributions. I have random variables $$X|M \sim \text{N}(M,\sigma^2)$$ and $$M \sim \text{N}(\theta, s^2)$$, with conditional and marginal densities given by:

\begin{equation} \begin{aligned} f(x|m) &= \frac{1}{\sigma \sqrt{2\pi}} \cdot \exp \Big( -\frac{1}{2} \Big( \frac{x-m}{\sigma} \Big)^2 \Big), \\[10pt] f(m) &= \frac{1}{s \sqrt{2\pi}} \cdot \exp \Big( - \frac{1}{2} \Big( \frac{m-\theta}{s} \Big)^2 \Big). \end{aligned} \end{equation}

I would like to know the marginal distribution of $$X$$. I have multiplied the above densities to form the joint density, but I cannot successfully integrate the result to get the marginal density of interest. My intuition tells me that this is a normal distribution with different parameters, but I can't prove it.

• In second line, $M\sim$ should be $m\sim$. Cannot edit for 1 character. Oct 16, 2018 at 3:35
• Tangentially on-topic, but you might want to read about conjugate priors - the normal distribution is the conjugate prior for the mean of a normal distribution. Oct 16, 2018 at 8:10

Your intuition is correct - the marginal distribution of a normal random variable with a normal mean is indeed normal. To see this, we first re-frame the joint distribution as a product of normal densities by completing the square:

\begin{equation} \begin{aligned} f(x,m) &= f(x|m) f(m) \\[10pt] &= \frac{1}{2\pi \sigma s} \cdot \exp \Big( -\frac{1}{2} \Big[ \Big( \frac{x-m}{\sigma} \Big)^2 + \Big( \frac{m-\theta}{s} \Big)^2 \Big] \Big) \\[10pt] &= \frac{1}{2\pi \sigma s} \cdot \exp \Big( -\frac{1}{2} \Big[ \Big( \frac{1}{\sigma^2}+\frac{1}{s^2} \Big) m^2 -2 \Big( \frac{x}{\sigma^2} + \frac{\theta}{s^2} \Big) m + \Big( \frac{x^2}{\sigma^2} + \frac{\theta^2}{s^2} \Big) \Big] \Big) \\[10pt] &= \frac{1}{2\pi \sigma s} \cdot \exp \Big( -\frac{1}{2 \sigma^2 s^2} \Big[ (s^2+\sigma^2) m^2 -2 (x s^2+ \theta \sigma^2) m + (x^2 s^2+ \theta^2 \sigma^2) \Big] \Big) \\[10pt] &= \frac{1}{2\pi \sigma s} \cdot \exp \Big( - \frac{s^2+\sigma^2}{2 \sigma^2 s^2} \Big[ m^2 -2 \cdot \frac{x s^2 + \theta \sigma^2}{s^2+\sigma^2} \cdot m + \frac{x^2 s^2 + \theta^2 \sigma^2}{s^2+\sigma^2} \Big] \Big) \\[10pt] &= \frac{1}{2\pi \sigma s} \cdot \exp \Big( - \frac{s^2+\sigma^2}{2 \sigma^2 s^2} \Big( m - \frac{x s^2 + \theta \sigma^2}{s^2+\sigma^2} \Big)^2 \Big) \\[6pt] &\quad \quad \quad \text{ } \times \exp \Big( \frac{(x s^2 + \theta \sigma^2)^2}{2 \sigma^2 s^2 (s^2+\sigma^2)} - \frac{x^2 s^2 + \theta^2 \sigma^2}{2 \sigma^2 s^2} \Big) \\[10pt] &= \frac{1}{2\pi \sigma s} \cdot \exp \Big( - \frac{s^2+\sigma^2}{2 \sigma^2 s^2} \Big( m - \frac{x s^2 + \theta \sigma^2}{s^2+\sigma^2} \Big)^2 \Big) \cdot \exp \Big( -\frac{1}{2} \frac{(x-\theta)^2}{s^2+\sigma^2} \Big) \\[10pt] &= \sqrt{\frac{s^2+\sigma^2}{2\pi \sigma^2 s^2}} \cdot \exp \Big( - \frac{s^2+\sigma^2}{2 \sigma^2 s^2} \Big( m - \frac{x s^2 + \theta \sigma^2}{s^2+\sigma^2} \Big)^2 \Big) \\[6pt] &\quad \times \sqrt{\frac{1}{2\pi (s^2+\sigma^2)}} \cdot \exp \Big( -\frac{1}{2} \frac{(x-\theta)^2}{s^2+\sigma^2} \Big) \\[10pt] &= \text{N} \Big( m \Big| \frac{xs^2+\theta\sigma^2}{s^2+\sigma^2}, \frac{s^2 \sigma^2}{s^2+\sigma^2} \Big) \cdot \text{N}(x|\theta, s^2+\sigma^2). \end{aligned} \end{equation}

We then integrate out $$m$$ to obtain the marginal density $$f(x) = \text{N}(x|\theta, s^2+\sigma^2)$$. From this exercise we see that $$X \sim \text{N}(\theta, s^2+\sigma^2)$$.

• (+1), would this have been simpler using the moment generating function (MGF)? Oct 16, 2018 at 5:01
• @SecretAgentMan: Thanks for the up-votes. The OP said he was having trouble integrating the joint density, so that is the method I have used.
– Ben
Oct 16, 2018 at 5:03
• @Ben , poor word choice on my part. I didn't mean to criticize you. My comment was for anyone that looks at this in the future. Agree the OP asked re: density. Future people needing help should know alternative approach with MGF might be simpler. Again, I wan't trying to criticize you, your approach, or the OP. Oct 16, 2018 at 16:15
• @SecretAgentMan: That is totally fine - I didn't take it as a criticism. Since you have another method in mind, perhaps the best thing would be to give it as an alternative answer? (That way people can see exactly what you have in mind.)
– Ben
Oct 16, 2018 at 22:28
• @Ben-ReinstateMonica no problem, you saved me 200 hours if I would to try this alone. So you're still in a positive balance ;-) Jul 27, 2020 at 10:12

Let $$X = m +\epsilon$$ where $$m \sim N(\theta,s^2)$$ and $$\epsilon \sim N(0,\sigma^2)$$ and they are independent.

Then $$X|m$$ and $$m$$ follows the distributions specified in the question.

$$E(X)=E(m) = \theta$$

$$Var(X) = Var(m) +Var(\epsilon) = s^2+\sigma^2$$

According to "The sum of random variables following Normal distribution follows Normal distribution", and the normal distribution is determined by mean and variance, we have

$$X \sim N(\theta, s^2+\sigma^2)$$

• Ok thanks, I understand your answer but I would like to have a mathematical proof fo your first line X=m+ϵ. Is it possible ? Oct 16, 2018 at 2:29
• In math prove, some time to need to create some thing. For example, + x - x, divides by a then times a.... So here we need to construct a X, such that 1) the original properties are kept and 2) give us the convenience to proving. Do you mean to prove "Then $X|m$ and $m$ follows the distributions specified in the question."? Oct 16, 2018 at 2:38
• Thank you I understand your POV. It's a smart way to answer to my question. Oct 16, 2018 at 11:48
• sorry ... point of view Oct 17, 2018 at 19:24

Here's a solution using moment generating functions, as suggested by @SecretAgentMan, that also ties in with the very slick answer provided by @user158565. If you like, you can view this as an (overly) rigorous justification of the decomposition provided by @user158565.

Let $$M \sim N(\theta, s^2)$$ and $$X|M \sim N(M,\sigma^2)$$. We are asked to find the unconditional distribution of $$X$$. To this end, define $$\varepsilon \equiv X - M$$. We show that $$\varepsilon \sim N(0, \sigma^2)$$ by calculating its moment generating function (mgf). We have \begin{align*} \mathbb{E}\left[e^{t\varepsilon}\right] &= \mathbb{E}\left[\exp\left\{t(X-M)\right\}\right] = \mathbb{E}\left[\mathbb{E}\left(\left. e^{tX}e^{-tM}\right|M\right) \right]\\ &= \mathbb{E}\left[e^{-tM}\mathbb{E}\left(\left. e^{tX}\right|M\right) \right] = \mathbb{E}\left[ e^{-tM} \exp\left\{tM + \frac{1}{2}\sigma^2 t^2 \right\}\right]\\ &= \mathbb{E}\left[ \exp\left\{\frac{1}{2}\sigma^2 t^2 \right\}\right] \end{align*} using iterated expectations and the fact that $$X|M$$ is a normal random variable with mean $$M$$ and variance $$\sigma^2$$, so that its moment generating function is $$\exp\left\{ tM + \frac{1}{2}\sigma^2 t^2 \right\}$$. We recognize $$\mathbb{E}[e^{t\varepsilon}]$$ as the mgf of a normal random variable, hence $$\varepsilon \sim N(0,\sigma^2)$$. Next, we show that $$M$$ and $$\varepsilon$$ are independent by showing that their joint mgf equals the product of the respective marginal mgfs. Again using iterated expectations and the mgf of $$X|M$$, we have \begin{align*} \mathbb{E}\left[ \exp\left\{ t_1 M + t_2 \varepsilon \right\} \right] &= \mathbb{E}\left[ \exp\left\{ t_1 M + t_2(X -M) \right\} \right] = \mathbb{E}\left[ \exp\left\{(t_1 - t_2)M + t_2 X\right\} \right]\\ &= \mathbb{E}\left[ \exp\left\{(t_1 - t_2)M\right\} \mathbb{E}\left(\left. e^{t_2 X}\right|M \right) \right]\\ &= \mathbb{E}\left[ \exp\left\{(t_1 - t_2)M\right\} \exp\left\{ t_2 M + \frac{1}{2}\sigma^2 t_2^2 \right\} \right]\\ &= \mathbb{E}\left[ \exp\left\{ t_1 M + \frac{1}{2}\sigma^2 t_2^2 \right\} \right]\\ &= \mathbb{E}\left[e^{t_1 M}\right] \exp\left\{\frac{1}{2}\sigma^2 t_2^2 \right\} \\ &= \mathbb{E}\left[e^{t_1 M}\right] \mathbb{E}\left[ e^{t_2 \varepsilon} \right] \end{align*} as claimed. We have shown that $$X = M + \varepsilon$$ where $$\varepsilon \sim N(0, \sigma^2)$$ independently of $$M \sim N(\theta, s^2)$$. It follows that $$X \sim N(\theta, s^2 + \sigma^2)$$.

Finally I found a easier solution of my problem by the MGF, without to much calculations.

Let assume $$X_1, X_2, ... X_n$$ which follow a specific law

The MGF of this variables is $$M(t_1,t_2,...,t_n) = E[\exp(t_1X_1 + t_2X_2 + ... + t_nX_n)]$$

In a similar way we can define moments in a conditional approach like this:

$$E[X|M]=∫xf(x|M) dx.$$

Using this, we can prove that: $$X \sim N(\theta, s^2+ \sigma^2)$$