2
$\begingroup$

I was going through Chuong B Do' notes on The Multivariate Gaussian Distribution. The multivariate Gaussian distribution according to it, is given as follows:

$p(x; \mu, \Sigma) = \frac{1}{(2\pi)^{n/2}|\Sigma|^{1/2}} \exp(-\frac{1}{2}(x − \mu)^{T}\Sigma^{−1}(x − \mu))$

While attempting to come up to this conclusion myself I got stuck at,

$P(x_1,x_2) = P(x_1)P(x_2) = \frac{1}{2\pi\sigma^{2}}\exp(-\frac{1}{2\sigma^{2}}((x_1 - \mu_{X_1})^{2} + (x_2 - \mu_{X_2})^{2}))$

How can I relate the two equation or rather generalize the second one to reach to first?

$\endgroup$
1
  • 2
    $\begingroup$ What principles are you using to "come up to" this result, given this is a definition and not a conclusion? There are many principles that could be used and it's interesting to read about them: see stats.stackexchange.com/questions/4364 and focus on the answers that refer to multivariate distributions. $\endgroup$
    – whuber
    Jan 17, 2020 at 14:28

1 Answer 1

1
$\begingroup$

There are errors in your second expression in particular terms involving $\sigma$.

Assuming $X_1, X_2$ are independent normal with variance $\sigma^2$.

Here $\Sigma = \begin{bmatrix} \sigma^2 & 0 \\ 0 & \sigma^2\ \end{bmatrix}=\sigma^2I_2$, then $|\Sigma|=\sigma^4$ and $|\Sigma|^\frac12=\sigma^2$

\begin{align} P(x_1, x_2) &= \prod_{i=1}^2 P(x_i) \\ &=\prod_{i=1}^2 \frac1{\sqrt{2\pi}\sigma} \exp\left(-\frac1{2\sigma^2} (x_i-\mu_i)^2\right) \\ &= \frac1{2\pi\sigma^2} \exp\left(-\frac12\sum_{i=1}^2\frac1{\sigma^2} (x_i-\mu_i)^2\right) \\ &= \frac1{2\pi\sigma^2} \exp\left(-\frac12\left(\frac1{\sigma^2} (x_1-\mu_1)^2+\frac1{\sigma^2} (x_2-\mu_2)^2\right)\right) \\ &= \frac1{2\pi\sigma^2} \exp\left(-\frac12\begin{bmatrix} x_1-\mu_1, & x_2-\mu_2\end{bmatrix}\begin{bmatrix} \frac1{\sigma^2} & 0 \\ 0& \frac1{\sigma^2}\end{bmatrix}\begin{bmatrix} x_1-\mu_1 \\ x_2-\mu_2\end{bmatrix}\right) \\ &= \frac1{(2\pi)^{2/2}\sigma^2}\exp \left(-\frac12(x-\mu)^T\Sigma^{-1} (x-\mu) \right) \end{align}

$\endgroup$
2
  • $\begingroup$ Thanks a lot for the response. My question is how were you able to map from univariate to multivariate? As in, how was the $ \Sigma^{-1} $ get introduced in that particular location between $(x - \mu)^{T}$ and $(x - \mu)$ ? $\endgroup$
    – user256228
    Jan 17, 2020 at 14:19
  • 1
    $\begingroup$ I added a few more lines of details. Typically we have to be familiar with the expansion of $x^TAx$. Notice that there is no mixed terms in the form of $(x_1-\mu_1) (x_2-\mu_2)$, hence the off diagonal parts are $0$ and the diagonal entries are given by the coefficeint of the square terms. $\endgroup$ Jan 17, 2020 at 14:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy