How initializing weights to a certain variance reduce vanishing / exploding gradients? Let's assume we have a network with 4 features and one neuron:

Accroding to the course "weight-initialization-for-deep-networks" with Andrew Ng, we want to set the variance of the weights to $Var(w)=1/n$ to avoid vanishing/exploding gradients.
My questions are:


*

*Why we want $Var(w)=1/n$ and not some other variance. Is this because the variance of the weighted sum will be 1? If $z = w_1x_1 + w_2x_2 + ... + w_nx_n$ then $Var(z)=Var(w_1x_1+...w_nx_n)=n \cdot Var(w)=n \cdot (1/n)=1$? 

*Why we want to keep variance the same in all layers? I thought the whole idea of normaliing the weights is to make the training faster. Because if we don't normalize the weights we might get vanishing/exploding gradients. So, how keeping the variance the same will achieve this goal? I think I'm really confused

 A: First of all, I'm pretty sure the $Var(w_1x_1+...w_nx_n)=n \cdot Var(w)$ part is incorrect, but yes in theory you are correct, as they want to keep the layer's total variance "manageable". 
The goal of this initialization is to keep $Var(y_L) \approx Var(y_1)$, where $y_1$ is the output of the first layer and $y_L$ is the output of the last. In order to do so the following condition must be met, for any given layer $l$: $\frac{1}{2} n_l Var(w_l) = 1$. 
This initialization strategy, was first proposed by He et al. A more detailed explanation is provided in their research paper. This strategy is most commonly referred to as He normal initialization. This technique is draws the weights from a truncated normal distribution, with $μ = 0$ and $σ = \sqrt{1 \over n_l}$, where $n_l$ is the number of neurons in the previous layer. 
Another very common initialization strategy, which was the inspiration for the previous work is the Xavier (or glorot) initialization (Glorot et al.). This technique, again, draws the weights from a truncated normal distribution but this time with $μ = 0$ and $σ = \sqrt{2 \over (n_l + n_{l+1})}$, where $n_l$ is the number of neurons in the previous layer and $n_{l+1}$ is the number of neurons the current layer.

Edit
For your second question He et al, in their publication they state (pg 4, after eq 9):

A proper initialization method should avoid reducing or magnifying the magnitudes of input signals exponentially.

While they do try to keep $Var(y_L) \approx Var(y_1)$, their actual goal is to keep $Var(y_L) = c \cdot Var(y_1)$, where $c$ is a constant. At the very least they try to make the variance not grow exponentially from layer to layer.
The same motivation appears to be guiding Glorot et at, in their choice of an initializer (see Section 4.2.1)
