Sampling distribution of variance of $N$ distinct normal random variables

Question

Say I have a set of $N$ distinct and independent sources of noise. Each source, $X_{n}$, follows a normal distribution with individual means and variances.

\begin{equation} X_{1}\sim\mathcal{N}\left(\mu_{1},\sigma_{1}^{2}\right), X_{2}\sim\mathcal{N}\left(\mu_{2},\sigma_{2}^{2}\right), \dots, X_{N}\sim\mathcal{N}\left(\mu_{N},\sigma_{N}^{2}\right) \end{equation}

Now consider a new r.v, $P_{n}$, which is defined as the difference of two i.i.d. $X_{n}$ such that

\begin{equation} P_{n}=X_{n,1}-X_{n,2} \sim\mathcal{N}\left(\mu_{n}-\mu_{n},\sigma_{n}^{2}+\sigma_{n}^{2}\right) \to\mathcal{N}\left(0,2\sigma_{n}^{2}\right) \end{equation}

I want to find the pdf of the r.v. $Z$ which is defined to be the sample variance of the vector of differences, divided by $2$ i.e.

\begin{equation} Z=%\frac{1}{2(N-1)}\sum_{n=1}^{N}\left(P_{n}-\overline{P_{n}}\right)^{2} \frac{\mathrm{Var}\bigl[P_{1},\dots,P_{N}\bigr]}{2} \end{equation}

An attempt:

\begin{equation} Z=\frac{1}{2(N-1)}\sum_{n=1}^{N}\left(P_{n}-\mu_{P_{n}}\right)^{2} \end{equation}

We know $\mu_{P_{n}}=0\ \forall \,n$ therefore,

\begin{equation} Z=\frac{1}{2(N-1)}\sum_{n=1}^{N}P_{n}^{2} \end{equation}

Introducing $W_{n} = P_{n}/\sqrt{2}\sim \mathcal{N}(0,\sigma_{n}^{2})$ gives

\begin{equation} Z=\frac{1}{(N-1)}\sum_{n=1}^{N}W_{n}^{2} \end{equation}

Since $W_{n}/\sigma_{n}\sim\mathcal{N}(0,1)$, it follows that $W_{n}^{2}\sim \sigma_{n}^{2}\,\Gamma(1/2,2)=\Gamma(1/2,2\sigma_{n}^{2})$.

Introducing $Y_{n}=W_{n}/(N-1)\sim \Gamma(1/2,2\sigma_{n}^{2}/(N-1))$. Therefore

\begin{equation} Z=\sum_{n=1}^{N}Y_{n} \end{equation}

If each $\sigma_{n}$ is equal then the sum of these gamma random variables is another gamma random variable such that

\begin{equation} Z\sim \Gamma\left(\frac{N}{2},\frac{2\sigma^{2}}{N-1}\right) \quad \text{for}\ \sigma_{1}^{2}=\sigma_{2}^{2}=\dots=\sigma_{N}^{2} \end{equation}

I am not sure if this is right or not but I am looking for the solution for when the $\sigma_{n}^{2}$'s are not equal. Any help is appreciated.

Answer 1

Since you have different variances in your sources of noise, any sample variance taken across these different groups will not serve as a sensible estimator of any individual variance. (It will implicitly serve as an estimator of a linear combination of the variances.) That being said, it is generally inappropriate to use Bessel's correction in a sample variance estimator where you are using the (known) true mean, so it would be better to divide by $N$ in your 'sample variance' formula than by $N-1$. With that said, here are some results that might help.

---

For each $n = 1, ..., N$ you have $P_n - \mu_{P_n} \sim \text{N}(0, 2 \sigma_n^2)$, which gives you:

$$\begin{matrix} (P_n - \mu_{P_n})^2 \sim 2 \sigma_n^2 \cdot C_n & & C_n \sim \chi_1^2. \end{matrix}$$

You can then use a 'sample variance' given by:

$$\begin{equation} \begin{aligned} S_{\mu}^2 \equiv \frac{1}{N} \sum_{i=1}^N (P_n - \mu_{P_n})^2 &= \frac{1}{N} \sum_{i=1}^N 2 \sigma_n^2 \cdot C_n. \end{aligned} \end{equation}$$

This random variable is a weighted sum of chi-squared random variables (each with one degree-of-freedom). In the special cases where $\sigma \equiv \sigma_1 = ... = \sigma_N$ you get $S_{\mu}^2 \sim \frac{2 \sigma}{N} \cdot \chi_n^2$. The more general case is a well-known distributional problem (see e.g., here). The distribution of this type of random variable is complicated and cannot be written in closed form. The quantity of interest to you is just a multiple of this.