# Deriving the normalizing constant for the multivariate Gaussian

I am trying to derive the normalizing constant for the multivariate Gaussian. The book I'm following suggests diagonalizing the covariance matrix and then using a change of variables.

So, we consider the following density for a random $d$-dimensional vector $\mathbf{x}$ and a positive definite symmetric matrix $\Sigma$. $$p(\mathbf{x}) \propto e^{-\frac{1}{2}(\mathbf{x}-\mathbf{\mu})^T\Sigma^{-1}(\mathbf{x}-\mathbf{\mu})}$$

We can diagonalize $\Sigma=Q\Lambda Q^T$ and let $\mathbf{y}=Q(\mathbf{x}-\mathbf{\mu})$ and $z=\Lambda^{-1/2}\mathbf{y}$. Then $$p(\mathbf{x}) \propto e^{-\frac{1}{2}\mathbf{z}^T\mathbf{z}} = e^{-\frac{1}{2}\sum_i \lambda_i y_i^2}$$ which is just a representation of the joint density of the independent $y_i$'s.

If everything's right, this should integrate to $$\int_{-\infty}^{\infty}e^{-\frac{1}{2}\sum_i \lambda_i y^2_i}d\mathbf{y}=\sqrt{(2\pi)^d|\Sigma|}$$ which seems likely considering it the value of the Gaussian integral, though I'll admit that my calculus is still slightly beneath this one.

My questions are:

• Does that last expression indeed integrate to that? Is it easier to integrate than the original density for $\mathbf{x}$, or was it all for nothing?
• How can the change of variables formula be helpful here? This formula is $$p_y(\mathbf{y}) = p_x(\mathbf{x}) \hspace{.2em} |\mbox{det } J_{x \rightarrow y}|$$ where $J_{x \rightarrow y}$ is the Jacobian matrix of $\mathbf{x}$ with respect to $\mathbf{y}$.

• The reason that $\int_{\mathcal{R}^d} e^{-\frac{1}{2} \sum_i \lambda_i y_i^2} dy$ is easier to integrate is that it can be expressed as a product of univariate integrals:

\begin{align} \int_{\mathcal{R}^d} e^{-\frac{1}{2} \sum_i \lambda_i y_i^2} dy &= \int_{\mathcal{R}^d}\Pi_{i=1}^d e^{-\frac{1}{2} \lambda_i y_i^2} dy \\ &= \Pi_{i=1}^d \int_{-\infty}^{\infty} e^{\frac{-1}{2} \lambda_i y_i^2} dy_i \end{align}

Now we can apply the formula for integrating under a univariate normal distribution: \begin{align} \Pi_{i=1}^d \int_{-\infty}^{\infty} e^{\frac{-1}{2} \lambda_i y_i^2} dy_i &= \Pi_{i=1}^d \left(2 \pi \lambda_i \right) ^{-\frac{1}{2}} \\ &= \sqrt{ (2 \pi)^d \Pi_{i=1}^d \lambda_i } \end{align}

To finish this integral, note that the when you take the eigendecomposition $\Sigma = Q^T \Lambda Q$, the diagonal values of $\Lambda$ ($\lambda_i$) are the eigenvalues of $\Sigma$, and the product of the eigenvalues of $\Sigma$ is the determinant of $\Sigma$. That is, $\Pi_{i=1}^d \lambda_i = \mathrm{det}(\Sigma)$. This finally gives us $\int_{\mathcal{R}^d} e^{-\frac{1}{2} \sum_i \lambda_i y_i^2} dy = \sqrt{ (2 \pi)^d \left| \Sigma \right| }$.

• But wait! How did we get the right answer already? Shouldn't there have been a Jacobian involved when we transformed from $x$ to $y$? We needed to prove that $\int_{\mathcal{R^d}} e^{-\frac{1}{2} (x - \mu)^T Q^T \Lambda^{-1} Q (x-\mu)} dx = \int_{\mathcal{R^d}} e^{-\frac{1}{2} y^T \Lambda^{-1} y} \left| \frac{\partial x}{\partial y} \right| dy = \sqrt{ (2 \pi)^d \left| \Sigma \right| }$, but we've only shown that $\int_{\mathcal{R^d}} e^{-\frac{1}{2} y^T \Lambda^{-1} y} dy = \sqrt{ (2 \pi)^d \left| \Sigma \right| }$.

To finish the proof, we need to show that $\left| J \right| = \left| \frac{\partial x}{\partial y} \right| = 1$. Let's look more carefully at that diagonalization.

Since $\Sigma$ is a covariance matrix, it should be symmetric positive definite. Therefore, there is an eigendecomposition where $Q$ is orthonormal, so $\Sigma = Q^{-1} \Lambda Q$, where $\Lambda$ is diagonal and $Q^{-1} = Q^T$.

So we have $y = Q(x - \mu)$, or $x = Q^T y + \mu$. Therefore, $\left| \frac{\partial x}{\partial y} \right| = |Q^T| = 1$, since $Q$ is orthonormal.