normal distribution in machine learning

Question

In many machine learning papers, there are notations like N(0,I). It seems that notation describes a normal distribution. If so, why did not use N(0,1)?

Could someone describe the difference between normal distribution N(0,I) and N(0,1)? Many thanks

Answer 1

It is a bit hard to understand what the $I$ is really, but in general this notation is used to designate a squared matrix with $1$ on the diagonal and $0$ elsewhere. $I$ is called the identity matrix.

$N(0,I)$ is not the normal distribution and $0$ is a vector of zeros, let us note it in bold $\mathbf{0}$ to distinguish it from a scalar. $N(\mathbf{0},I)$ belongs to a more general distribution, the multivariate normal distribution. The multivariate normal distribution in $\mathbb{R}^n$ can be characterized by two parameters, the mean $\mathbf{\mu} \in \mathbb{R}^n$ and covariance matrix $\Sigma \in \mathbb{R}^{n \times n}$, with $\Sigma$ a positive definite matrix. The probability density function of $N(\mathbf{\mu},\Sigma)$ is given by:

$$ f(\mathbf{x}) = \frac{1}{(2\pi)^{n/2}\det(\Sigma)^{1/2}}e^{ -\frac{1}{2}(\mathbf{x} - \mathbf{\mu})^t \Sigma^{-1}(\mathbf{x} - \mathbf{\mu}) }. $$

Then, if you replace $\mathbf{\mu}$ by $\mathbf{0}$ and $\Sigma$ by $I$, you have your distribution with density function:

$$ f(\mathbf{x}) = \frac{1}{(2\pi)^{n/2}}e^{ -\frac{1}{2}\mathbf{x}^t \mathbf{x} }. $$

For $n= 1$, the previous expression corresponds to the density function of a standard normal distribution.

I hope this answer your question.