Distribution of the mean of distributions for samples with a given standard deviation

Question

I have a normal distribution $N(0, \sigma^2)$. If I start selecting all distributions $N(\mu, \sigma^2/2)$ from this distribution, what are the parameters of the distribution for the mean $\mu$: $N(0, ?)$? What will the standard deviation be?

In other words, if there are a set of $N(\mu, \sigma^2/2)$ distributions that together form $N(0, \sigma^2)$ distribution, what the distribution for $\mu$ should be? There are two extreme points. This is if I chose distributions with the same standard deviation $\sigma^2$. In this case, the standard deviation for $\mu$ would tend to zero with n->inf. And if I were to select one point from $N(0, \sigma^2)$, then the standard deviation for $\mu$ would be equal to $\sigma^2$.

Answer 1

Let $X$ represent the random variable for $\mu$ and $Y\sim\mathcal{N}(X,\sigma^2/2)$ the random variable for the final distribution. We will aim to demonstrate that $Y\sim\mathcal{N}(0,\sigma^2)$ when $X\sim \mathcal{N}(0,\sigma^2/2)$.

We begin by marginalizing the joint distribution.

\begin{align} P(Y=y)&=\frac{1}{\sqrt{2\pi \sigma^2}}e^{-y^2/2\sigma^2}\\ &=\int_{-\infty}^\infty P(X=x,Y=y)dx\\ &=\int_{-\infty}^\infty P(Y=y\mid X=x)P(X=x)dx\\ &=\int_{-\infty}^\infty \frac{1}{\sqrt{\pi \sigma^2}}e^{-(y-x)^2/\sigma^2}P(X=x)dx \end{align}

We will presume going forward $X$ will take on a normal distribution such that $X\sim\mathcal{N}(0,a\sigma^2)$, where $a$ is an unknown parameter. We will complete the square on $X$ to obtain the kernel for the normal distribution.

\begin{align} P(Y=y)&=\frac{1}{\sqrt{2\pi \sigma^2}}e^{-y^2/2\sigma^2}\\ &=\int_{-\infty}^\infty \frac{1}{\sqrt{\pi \sigma^2}}e^{-(y-x)^2/\sigma^2}\frac{1}{\sqrt{2a\pi \sigma^2}}e^{-x^2/2a\sigma^2}dx\\ &=\int_{-\infty}^\infty \frac{1}{\sqrt{2a\pi^2 \sigma^4}}e^{(-y^2+2yx-x^2)/\sigma^2-x^2/2a\sigma^2}dx\\ &=\int_{-\infty}^\infty \frac{1}{\sqrt{2a\pi^2 \sigma^4}}e^{-x^2/(2a/2a+1)\sigma^2+2xy/\sigma^2-y^2/\sigma^2}dx\\ &=\int_{-\infty}^\infty \frac{1}{\sqrt{2a\pi^2 \sigma^4}}e^{-(x-2ay/(2a+1))^2/(2a/2a+1)\sigma^2-y^2/(2a+1)\sigma^2}dx\\ &=\frac{1}{\sqrt{(2a+1)\pi \sigma^2}}\int_{-\infty}^\infty \frac{\sqrt{2a+1}}{\sqrt{2a\pi\sigma^2}}e^{-(x-(a+1)y/a)^2/(2a/2a+1)\sigma^2-y^2/(2a+1)\sigma^2}dx\\ &=\frac{1}{\sqrt{(2a+1)\pi \sigma^2}}e^{-y^2/(2a+1)\sigma^2} \end{align}

In the second to last line, the first part of the exponential was kernel to the normal distribution in $X$, so that what remains after integration is a normal distribution in $Y$. It is evident that we achieve the target distribution when $a=1/2$, thereby showing what we set out to do.