2
$\begingroup$

Say I have a set of $N$ distinct and independent sources of noise. Each source, $X_{n}$, follows a normal distribution with individual means and variances.

\begin{equation} X_{1}\sim\mathcal{N}\left(\mu_{1},\sigma_{1}^{2}\right), X_{2}\sim\mathcal{N}\left(\mu_{2},\sigma_{2}^{2}\right), \dots, X_{N}\sim\mathcal{N}\left(\mu_{N},\sigma_{N}^{2}\right) \end{equation}

Now consider a new r.v, $P_{n}$, which is defined as the difference of two i.i.d. $X_{n}$ such that

\begin{equation} P_{n}=X_{n,1}-X_{n,2} \sim\mathcal{N}\left(\mu_{n}-\mu_{n},\sigma_{n}^{2}+\sigma_{n}^{2}\right) \to\mathcal{N}\left(0,2\sigma_{n}^{2}\right) \end{equation}

I want to find the pdf of the r.v. $Z$ which is defined to be the sample variance of the vector of differences, divided by $2$ i.e.

\begin{equation} Z=%\frac{1}{2(N-1)}\sum_{n=1}^{N}\left(P_{n}-\overline{P_{n}}\right)^{2} \frac{\mathrm{Var}\bigl[P_{1},\dots,P_{N}\bigr]}{2} \end{equation}

An attempt:

\begin{equation} Z=\frac{1}{2(N-1)}\sum_{n=1}^{N}\left(P_{n}-\mu_{P_{n}}\right)^{2} \end{equation}

We know $\mu_{P_{n}}=0\ \forall \,n$ therefore,

\begin{equation} Z=\frac{1}{2(N-1)}\sum_{n=1}^{N}P_{n}^{2} \end{equation}

Introducing $W_{n} = P_{n}/\sqrt{2}\sim \mathcal{N}(0,\sigma_{n}^{2})$ gives

\begin{equation} Z=\frac{1}{(N-1)}\sum_{n=1}^{N}W_{n}^{2} \end{equation}

Since $W_{n}/\sigma_{n}\sim\mathcal{N}(0,1)$, it follows that $W_{n}^{2}\sim \sigma_{n}^{2}\,\Gamma(1/2,2)=\Gamma(1/2,2\sigma_{n}^{2})$.

Introducing $Y_{n}=W_{n}/(N-1)\sim \Gamma(1/2,2\sigma_{n}^{2}/(N-1))$. Therefore

\begin{equation} Z=\sum_{n=1}^{N}Y_{n} \end{equation}

If each $\sigma_{n}$ is equal then the sum of these gamma random variables is another gamma random variable such that

\begin{equation} Z\sim \Gamma\left(\frac{N}{2},\frac{2\sigma^{2}}{N-1}\right) \quad \text{for}\ \sigma_{1}^{2}=\sigma_{2}^{2}=\dots=\sigma_{N}^{2} \end{equation}

I am not sure if this is right or not but I am looking for the solution for when the $\sigma_{n}^{2}$'s are not equal. Any help is appreciated.

$\endgroup$
  • $\begingroup$ Are these random variables independent? $\endgroup$ – Michael Chernick Feb 15 '17 at 20:52
  • $\begingroup$ @MichaelChernick Yes. Each $X_{n,j}$ is independent as well as each $P_{n}$. $\endgroup$ – Aaron Hendrickson Feb 16 '17 at 16:44
  • 1
    $\begingroup$ Since you know that $\mu_{P_n} \equiv \mathbb{E}(P_n) = 0$, and you are using this fact in your variance expression (as opposed to using the sample mean), you should not be applying Bessel's correction to the sample variance (i.e., the denominator should be $N$, not $N-1$). $\endgroup$ – Ben Jan 23 '18 at 0:28
1
$\begingroup$

Since you have different variances in your sources of noise, any sample variance taken across these different groups will not serve as a sensible estimator of any individual variance. (It will implicitly serve as an estimator of a linear combination of the variances.) That being said, it is generally inappropriate to use Bessel's correction in a sample variance estimator where you are using the (known) true mean, so it would be better to divide by $N$ in your 'sample variance' formula than by $N-1$. With that said, here are some results that might help.


For each $n = 1, ..., N$ you have $P_n - \mu_{P_n} \sim \text{N}(0, 2 \sigma_n^2)$, which gives you:

$$\begin{matrix} (P_n - \mu_{P_n})^2 \sim 2 \sigma_n^2 \cdot C_n & & C_n \sim \chi_1^2. \end{matrix}$$

You can then use a 'sample variance' given by:

$$\begin{equation} \begin{aligned} S_{\mu}^2 \equiv \frac{1}{N} \sum_{i=1}^N (P_n - \mu_{P_n})^2 &= \frac{1}{N} \sum_{i=1}^N 2 \sigma_n^2 \cdot C_n. \end{aligned} \end{equation}$$

This random variable is a weighted sum of chi-squared random variables (each with one degree-of-freedom). In the special cases where $\sigma \equiv \sigma_1 = ... = \sigma_N$ you get $S_{\mu}^2 \sim \frac{2 \sigma}{N} \cdot \chi_n^2$. The more general case is a well-known distributional problem (see e.g., here). The distribution of this type of random variable is complicated and cannot be written in closed form. The quantity of interest to you is just a multiple of this.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.