Deriving the normalizing constant for the multivariate Gaussian I am trying to derive the normalizing constant for the multivariate Gaussian. The book I'm following suggests diagonalizing the covariance matrix and then using a change of variables.
So, we consider the following density for a random $d$-dimensional vector $\mathbf{x}$ and a positive definite symmetric matrix $\Sigma$.
$$
p(\mathbf{x}) \propto e^{-\frac{1}{2}(\mathbf{x}-\mathbf{\mu})^T\Sigma^{-1}(\mathbf{x}-\mathbf{\mu})}
$$
We can diagonalize $\Sigma=Q\Lambda Q^T$ and let $\mathbf{y}=Q(\mathbf{x}-\mathbf{\mu})$ and $z=\Lambda^{-1/2}\mathbf{y}$. Then
$$
p(\mathbf{x}) \propto e^{-\frac{1}{2}\mathbf{z}^T\mathbf{z}} = e^{-\frac{1}{2}\sum_i \lambda_i y_i^2}
$$
which is just a representation of the joint density of the independent $y_i$'s.
If everything's right, this should integrate to 
$$
\int_{-\infty}^{\infty}e^{-\frac{1}{2}\sum_i \lambda_i y^2_i}d\mathbf{y}=\sqrt{(2\pi)^d|\Sigma|}
$$
which seems likely considering it the value of the Gaussian integral, though I'll admit that my calculus is still slightly beneath this one.
My questions are:


*

*Does that last expression indeed integrate to that? Is it easier to integrate than the original density for $\mathbf{x}$, or was it all for nothing?

*How can the change of variables formula be helpful here? This formula is
$$
p_y(\mathbf{y}) = p_x(\mathbf{x}) \hspace{.2em} |\mbox{det } J_{x \rightarrow y}|
$$
where $J_{x \rightarrow y}$ is the Jacobian matrix of $\mathbf{x}$ with respect to $\mathbf{y}$. 

 A: Trying to put it all together for convenience.
The probability density of multivariate normal distribution is
$$p(\mathbf{x})  = \frac{1}{Z} e^{- \frac{1}{2} (\mathbf{x} - \boldsymbol{\mu})^T \Sigma^{-1} (\mathbf{x} - \boldsymbol{\mu})}$$
where $\Sigma$ is the covariance matrix, and can be eigen-decomposed into
$$\Sigma = Q \Lambda Q^T$$
with $Q$ being an orthonormal matrix, and $\Lambda$ being a diagonal matrix filled with eigenvalues.
so
$$\Sigma^{-1} = Q \Lambda^{-1} Q^T$$
Then, $p(\mathbf{x})$ can be rewritten as
\begin{align*}
p(x)
&= \frac{1}{Z} e^{- \frac{1}{2} (\mathbf{x} - \mathbf{\mu})^T Q \Lambda^{-1} Q^T (\mathbf{x} - \mathbf{\mu})} \\
&= \frac{1}{Z} e^{- \frac{1}{2} \mathbf{y}^T \Lambda^{-1} \mathbf{y}} \\
&= \frac{1}{Z} e^{- \frac{1}{2} \sum_i^d \frac{y_i^2}{\lambda_i}} \\
&= \frac{1}{Z} \prod_i^d e^{- \frac{y_i^2}{2\lambda_i}}
\end{align*}
where 


*

*$\mathbf{y} = Q^T (\mathbf{x} - \mathbf{\mu})$,

*$y_i$ is the ith element of $\mathbf{y}$, and

*$\lambda_i$ is the ith element along the diagonal of $\Lambda$
\begin{align*}
Z
&= \int_{\mathcal{R}^d} \prod_i^d e^{- \frac{y_i^2}{2\lambda_i}} d\mathbf{y} \\
&= \int_{y_d} e^{- \frac{y_d^2}{2\lambda_d}} \cdots \int_{y_1}  e^{- \frac{y_1^2}{2\lambda_1}} dy_1 \cdots dy_d \\
&= \int_{y_d} e^{- \frac{y_d^2}{2\lambda_d}} \cdots \int_{y_2}  e^{- \frac{y_2^2}{2\lambda_2}} \sqrt{2 \pi \lambda_1} dy_2 \cdots dy_d \\
&= \prod_{i=1}^d \sqrt{2 \pi \lambda_i} \\
&= (2 \pi)^{\frac{d}{2}} \prod_{i=1}^d (\lambda_i)^{\frac{1}{2}} \\
&= (2 \pi)^{\frac{d}{2}} \left|\Sigma\right|^{\frac{1}{2}} \\
\end{align*}


*

*The 3rd equality used the calculation of normalization constant in the case of centered univariate normal distribution ($\int_{-\infty}^{\infty} e^{- \frac{x^2}{2 \sigma^2}} dx = \sigma \sqrt{2\pi}$).

*The 6th equality uses the fact that the product of eigenvalues of $\Sigma$ is equal to its determinant ($\prod_i^d \lambda_i = \left| \Sigma \right|$)
A: *

*The reason that $\int_{\mathcal{R}^d} e^{-\frac{1}{2} \sum_i \lambda_i y_i^2} dy$ is easier to integrate is that it can be expressed as a product of univariate integrals:


\begin{align}
\int_{\mathcal{R}^d} e^{-\frac{1}{2} \sum_i \lambda_i y_i^2} dy &= \int_{\mathcal{R}^d}\Pi_{i=1}^d e^{-\frac{1}{2} \lambda_i y_i^2} dy \\
&= \Pi_{i=1}^d \int_{-\infty}^{\infty} e^{\frac{-1}{2} \lambda_i y_i^2} dy_i
\end{align}
Now we can apply the formula for integrating under a univariate normal distribution:
\begin{align}
\Pi_{i=1}^d \int_{-\infty}^{\infty} e^{\frac{-1}{2} \lambda_i y_i^2} dy_i &= \Pi_{i=1}^d \left(2 \pi \lambda_i \right) ^{-\frac{1}{2}} \\
&= \sqrt{ (2 \pi)^d \Pi_{i=1}^d \lambda_i }
\end{align}
To finish this integral, note that the when you take the eigendecomposition $\Sigma = Q^T \Lambda Q$, the diagonal values of $\Lambda$ ($\lambda_i$) are the eigenvalues of $\Sigma$, and the product of the eigenvalues of $\Sigma$ is the determinant of $\Sigma$. That is, $\Pi_{i=1}^d \lambda_i = \mathrm{det}(\Sigma)$. This finally gives us $\int_{\mathcal{R}^d} e^{-\frac{1}{2} \sum_i \lambda_i y_i^2} dy = \sqrt{ (2 \pi)^d \left| \Sigma \right| }$.


*

*But wait! How did we get the right answer already? Shouldn't there have been a Jacobian involved when we transformed from $x$ to $y$? We needed to prove that $\int_{\mathcal{R^d}} e^{-\frac{1}{2} (x - \mu)^T Q^T \Lambda^{-1} Q (x-\mu)} dx = \int_{\mathcal{R^d}} e^{-\frac{1}{2} y^T \Lambda^{-1} y} \left| \frac{\partial x}{\partial y} \right| dy = \sqrt{ (2 \pi)^d \left| \Sigma \right| }$, but we've only shown that $\int_{\mathcal{R^d}} e^{-\frac{1}{2} y^T \Lambda^{-1} y} dy = \sqrt{ (2 \pi)^d \left| \Sigma \right| }$. 


To finish the proof, we need to show that $\left| J \right| = \left| \frac{\partial x}{\partial y} \right| = 1$. Let's look more carefully at that diagonalization.
Since $\Sigma$ is a covariance matrix, it should be symmetric positive definite. Therefore, there is an eigendecomposition where $Q$ is orthonormal, so $\Sigma = Q^{-1} \Lambda Q$, where $\Lambda$ is diagonal and $Q^{-1} = Q^T$.
So we have $y = Q(x - \mu)$, or $x = Q^T y + \mu$. Therefore, $\left| \frac{\partial x}{\partial y} \right| = |Q^T| = 1$, since $Q$ is orthonormal.
