How to get Variance from Gaussian distribution and Random Initialization

Question

I am following deeplearning.ai's videos on Coursera. Prof Ng mentions that specific random initialisations for the weights(for example, by Xavier or He initialisations) can help optimise learning.

Particularly he says that multiplying a standard normal distribution by $\sqrt{\frac{2}{n^{[l-1]}}}$ produces a normal distribution with variance $ \frac{2}{n}$. Is there a proof for this statement? More particularly how can I derive the variance for a normal distribution multiplied by some factor?

Some background:

I attempted to find the variance for c*f(x) assuming f(x) is Gaussian and c is a constant by following the steps mentioned in the following question(particularly the accepted answer):

https://math.stackexchange.com/questions/518281/how-to-derive-the-mean-and-variance-of-a-gaussian-random-variable

However, I found some discrepancies in the results. I came to understand that this is probably caused by the fact that c*f(x) is no longer a PDF and hence we can no longer use the traditional definition of E(x), V(x) to estimate the mean and variance. So how are these particular random initialisations which lead to particular variances for the normal distribution arrived at?

Edit:

$ f(x) = \frac{1}{\sqrt{2\pi\sigma^2}}e^{\frac{-(x-\mu)^2}{2\sigma^2}} $

So, c*f(x) =

$\frac{\sqrt{c^2}}{\sqrt{2\pi\sigma^2}}e^{\frac{-(x-\mu)^2}{2\sigma^2}} $

I guess this is the part I don't understand. From what I can see, the constant only modifies the other term of the above expression. So, how is the new expression a valid PDF? The value of $2\sigma^2$ in the denominator of the exponent remains unchanged. So, how does this affect the variance?

Edit:

The following is an answer I got from another question.

The expressions for the mean and standard deviation are true for any random variable: $$ E(X+k) = EX+Ek = EX + k, \quad V(X+k)=VX+Vk=VX, $$ $$ E(kX) = kEX, \quad V(kX) = k^2VX $$ I am sure you can find proofs for these easily. But we still need to show that the transformed variables have normal distributions. We can do that as follows. Assume $k>0$. Then: $$ P(kX<x) = P(X<\frac xk) = \int_{-\infty}^\frac xk \frac{1}{\sqrt{2\pi}\sigma}\exp \left(-\frac{(t-\mu)^2}{2\sigma^2}\right)dt $$ $$ = \int_{-\infty}^x \frac{1}{\sqrt{2\pi}k\sigma}\exp \left(-\frac{(u -k\mu)^2}{2(k\sigma)^2}\right)du $$ where we substituted $u=kt$. This is the CDF of $\mathcal N(k\mu, k^2\sigma^2)$, so $kX$ has that distribution. (Note that we rediscovered the mean and variance). The cases $kX$ for $k<0$, and $X+k$ can be shown similarly.

Answer 1

There's a difference between the parameters of a distribution $F$ and a multiple of a random deviate drawn from $F$.

In your example, suppose we have $x\sim \mathcal{N}(0, 1^2)$. Then $y=cx$ for some $c>0$ has distribution $y\sim \mathcal{N}(0, c^2)$. For details, see Going from a normal distribution to a standard normal distribution with a change of variable

This is obviously not the same as $cf(x;0, \sigma^2)$; by inspection, we can see that $cf$ is not a valid distribution because it will only integrate to 1 in the special case of $c=1$.

The key detail here is that you're multiplying the standard normal deviate by $c$, not the standard normal distribution.