0
$\begingroup$

What is the intuition behind the change in formula from the univariate gaussian to the multivariate gaussian? Why are the determinant and $(2\pi)^{n/2}$ added into the equation?

Thanks

enter image description here

enter image description here

$\endgroup$
  • 1
    $\begingroup$ Linear algebra says there is an orthogonal transformation of $x$ that diagonalizes $\Sigma$. Among other things, such transformations do not change volumes. Thus, you can (and always should) think of the multivariate formula as really being the product of $n$ versions of the first formula, each with its own variance $\sigma_i^2$. Your question then comes down to asking why the determinant is the product of the $\sigma_i^2$. $\endgroup$ – whuber Dec 31 '17 at 16:14
1
$\begingroup$

You are attempting to generalize; instead, particularize from the multivariate case to the univariate case.

The $n$-variate normal density is, as you say, $$f(\mathbf x; \mathbf \mu, \Sigma) = \frac{1}{(2\pi)^{n/2}|\det \Sigma|^{1/2}}\exp\big(-\frac 12 (\mathbf x-\mathbf \mu)^T\Sigma^{-1}(\mathbf x-\mathbf \mu)\big) \tag{1}$$ where $\mathbf x$ is a column vector of length $n$, and $\Sigma$ is the $n\times n$ covariance matrix with variances down the main diagonal. Note that $\frac{1}{(2\pi)^{n/2}|\det \Sigma|^{1/2}}$ is a constant that makes the $n$-dimensional integral of the density $(1)$ have value $1$. Now, for the case $n=1$, the $1\times 1$ covariance matrix is just $[\sigma^2]$ with determinant $\sigma^2$ and inverse matrix $[\sigma^{-2}]$. So, getting rid of the distinction between $1\times 1$ matrices and ordinary scalars, the univariate normal distribution is obtained as \begin{align} f(x; \mu, \sigma^2) &= \frac{1}{(2\pi)^{1/2}(\sigma^2)^{1/2}}\exp\big(-\frac 12 (x-\mu)\sigma^{-2}(x-\mu)\big)\\ &= \frac{1}{\sigma(2\pi)^{1/2}}\exp\Bigr(-\frac 12 \left(\frac{x-\mu}{\sigma}\right)^2\Bigr)\tag{2} \end{align} exactly as you have it.

| cite | improve this answer | |
$\endgroup$
1
$\begingroup$

In both cases, everything before the exponential is simply a normalizing constant that makes the expression integrate to one. This is needed to have a proper probability distribution.

For example, in the univariate case:

$$\int_{-\infty}^\infty \exp \left (-\frac{(x-\mu)^2}{2 \sigma^2} \right ) = \sigma \sqrt{2 \pi}$$

So, multiplying by $\frac{1}{\sigma \sqrt{2 \pi}}$ makes $p(x; \mu, \sigma^2)$ integrate to one.

| cite | improve this answer | |
$\endgroup$
0
$\begingroup$

I like to think about how the gaussian distribution is constructed from the "inside". The middle of the equation, or the $(x-\mu)^2$, is a unit deviance, a function that satisfies:

$$ d(y; y) = 0$$

and

$$d(y; \mu) > 0$$ There is actually a big group of distributions constructed from unit deviances (see The Theory of Dispersion Models). Anyway, if we plot $(x-\mu)^2$, with $\mu = 0$ for example, varying $x$, we have this: enter image description here

But this doesn't look like a distribution, right? What is the first step to change this plot? Put a $-$ before the equation, or $-(x-\mu)^2$, which turns the plot upside down: enter image description here

Ok, but we want the distribution to give us plausible values. We can smooth the tails of the equation with $exp(-(1/2\sigma^2)(x-\mu)^2)$. The plot for this, considering $\sigma^2 = 1$ is: enter image description here

And now we "just" need to scale the equation in a way that it produces values between 0 and 1 and integrates 1 (I used quotes there because finding a good normalizing constant is not always an easy task). Then, we end up with: $$ f(x) = \frac{exp(-(1/2\sigma^2)(x-\mu)^2)}{\sqrt{(2 \pi \sigma^2}}$$

Plotting this, we have: enter image description here

This is not a very formal explanation (the way I did it in here), but a good one for understanding the Gaussian distribution. The multivariate case is then a generalization of what I explained above.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.