Multivariate Gaussian, rearranging means Looking through the the matrix cookbook, a collection of matrix identities, I came across this one called "rearranging means" in the multivariate Normal distribution (Sec. 8.1.5 or Eq. #356, also 357):
$$N_{\mathbf{Ax}}[\mathbf{m}, \mathbf{\Sigma}] = \frac{\sqrt{det(2 \pi(\mathbf{A^{T}\Sigma^{-1}\mathbf{A}} )^{-1})}}{\sqrt{det(2 \pi \mathbf{\Sigma})}} N_{\mathbf{x}}(\mathbf{A}^{-1}\mathbf{m}, (\mathbf{A}^{T}\mathbf{\Sigma}^{-1}\mathbf{A})^{-1} )$$
I haven't been able to find this identity in any other source.  From looking at it, it looks like it equates a distribution to itself post a linear transformation defined by $\mathbf{A}$.  Can anyone verify this thought?  It would seem to imply that in some situations it may perhaps be easier to compute a mvnpdf after some arbitrary transformation into a lower dimensional space. Could someone point me to a reference with more information on this identity?  Sorry I know this question is a bit vague.
 A: I couldn't find any references for the general identity with arbitrary $\mathbf{A}\in\mathbb{R}^{Q\times D}$, specially when $\text{rank}(\mathbf{A})<\min(Q,D)$. However, if we assume that $\text{rank}(\mathbf{A})=D$, then we can derive a similar identity.
Since $\mathbf{\Sigma}$ is full rank, then $\text{rank}(\mathbf{A}^T\mathbf{\Sigma}^{-1}\mathbf{A}) = D$, so $(\mathbf{A}^T\mathbf{\Sigma}^{-1} \mathbf{A})^{-1}$ is well defined. Then, the quadratic term of $N_{\mathbf{x}}$ would be something like:
$$\exp\left[-\frac{1}{2}(\mathbf{x}-\mathbf{\mu})^T(\mathbf{A}^T\mathbf{\Sigma}^{-1} \mathbf{A})(\mathbf{x}-\mathbf{\mu})\right]$$
By moving the $\mathbf{A}$ term to inside the brackets, we arrive at:
$$\exp\left[-\frac{1}{2}(\mathbf{A}\mathbf{x}-\mathbf{A}\mathbf{\mu})^T\mathbf{\Sigma}^{-1} (\mathbf{A}\mathbf{x}-\mathbf{A}\mathbf{\mu})\right]$$
Which is almost $N_{\mathbf{A}\mathbf{x}}$, meaning that $\mathbf{A}\mathbf{\mu} = \mathbf{m}$. We can use this equation figure out what $\mathbf{\mu}$ is:
\begin{align}
\mathbf{A}\mathbf{\mu} &= \mathbf{m}\\
\mathbf{A}^T\mathbf{A}\mathbf{\mu} &= \mathbf{A}^T\mathbf{m}\\
\left[\mathbf{A}^T\mathbf{A}\right]^{-1}\mathbf{A}^T\mathbf{A}\mathbf{\mu} &= \left[\mathbf{A}^T\mathbf{A}\right]^{-1}\mathbf{A}^T\mathbf{m}\\
\mathbf{\mu} &= \left[\mathbf{A}^T\mathbf{A}\right]^{-1}\mathbf{A}^T\mathbf{m}\\
\end{align}
Therefore, the correct formula for non-invertible but full column rank $\mathbf{A}$ should be:
$$N_{\mathbf{Ax}}[\mathbf{m}, \mathbf{\Sigma}] = \frac{\sqrt{\det(2 \pi(\mathbf{A^{T}\Sigma^{-1}\mathbf{A}} )^{-1})}}{\sqrt{\det(2 \pi \mathbf{\Sigma})}} N_{\mathbf{x}}(\left[\mathbf{A}^T\mathbf{A}\right]^{-1}\mathbf{A}^T\mathbf{m}, (\mathbf{A}^{T}\mathbf{\Sigma}^{-1}\mathbf{A})^{-1} )$$
