3
$\begingroup$

In many machine learning papers, there are notations like N(0,I). It seems that notation describes a normal distribution. If so, why did not use N(0,1)?

Could someone describe the difference between normal distribution N(0,I) and N(0,1)? Many thanks

$\endgroup$
2
  • 1
    $\begingroup$ $I$ is generally used in the context of multivariate normals - in this case, you not only need to describe the variance of one variable, but also the variance of all the other variables in your multivariable distribution, as well as how each variable is correlated with the others. In this case, we use a matrix to summarzie the variance and cross-variable correlations/covariances between variables. The $I$ refers to the simplest case when all variables have variance 1 are uncorrelated with each other. In $\endgroup$ Commented Mar 30, 2023 at 17:24
  • $\begingroup$ Thanks for your answer. That is really helpful. $\endgroup$ Commented Mar 31, 2023 at 2:00

1 Answer 1

7
$\begingroup$

It is a bit hard to understand what the $I$ is really, but in general this notation is used to designate a squared matrix with $1$ on the diagonal and $0$ elsewhere. $I$ is called the identity matrix.

$N(0,I)$ is not the normal distribution and $0$ is a vector of zeros, let us note it in bold $\mathbf{0}$ to distinguish it from a scalar. $N(\mathbf{0},I)$ belongs to a more general distribution, the multivariate normal distribution. The multivariate normal distribution in $\mathbb{R}^n$ can be characterized by two parameters, the mean $\mathbf{\mu} \in \mathbb{R}^n$ and covariance matrix $\Sigma \in \mathbb{R}^{n \times n}$, with $\Sigma$ a positive definite matrix. The probability density function of $N(\mathbf{\mu},\Sigma)$ is given by:

$$ f(\mathbf{x}) = \frac{1}{(2\pi)^{n/2}\det(\Sigma)^{1/2}}e^{ -\frac{1}{2}(\mathbf{x} - \mathbf{\mu})^t \Sigma^{-1}(\mathbf{x} - \mathbf{\mu}) }. $$

Then, if you replace $\mathbf{\mu}$ by $\mathbf{0}$ and $\Sigma$ by $I$, you have your distribution with density function:

$$ f(\mathbf{x}) = \frac{1}{(2\pi)^{n/2}}e^{ -\frac{1}{2}\mathbf{x}^t \mathbf{x} }. $$

For $n= 1$, the previous expression corresponds to the density function of a standard normal distribution.

I hope this answer your question.

$\endgroup$
1
  • $\begingroup$ Thank you for such a detailed answer. It makes much sense and makes me understand deeper. $\endgroup$ Commented Mar 31, 2023 at 2:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.