Let $\Sigma$ be a covariance matrix and let

$$x^T \Sigma^{-1} x = \|Ax\|_2^2.$$ What is the interpretation of matrix $A$?

I tried solving for $A$ with an eigenvalue decomposition of $\Sigma$ as follows,

$$x^T \Sigma^{-1} x = x^T (Q \Lambda Q^T)^{-1}x = x^T Q\Lambda^{-1}Q^{-1} x = x^TA^TAx,$$ and this is where I get stuck. I am unable to figure out a closed form solution to $A$.

  • $\begingroup$ Can you figure out the square root of $\Lambda^{-1}$? Do you know what $Q^{-1}$ inverse is? $\endgroup$ Mar 3, 2015 at 15:16
  • $\begingroup$ Yep, just figured it out to be $\frac{1}{\sqrt{\lambda}}$ where $\lambda$ is the eigenvalues of the covariance matrix $\endgroup$ Mar 3, 2015 at 19:03
  • 1
    $\begingroup$ This is a good forum for this question. $\endgroup$
    – user603
    Mar 3, 2015 at 22:55
  • 1
    $\begingroup$ I would still like to get an intuitive understanding of what matrix A means $\endgroup$ Mar 3, 2015 at 23:02
  • 1
    $\begingroup$ OK, if it's about interpretation of A (rather than getting a formula for A), then I agree with @user603 that it belongs here. Your original formulation was about "significance of A" and that was not very clear. $\endgroup$
    – amoeba
    Mar 3, 2015 at 23:45

1 Answer 1


$x^T \Sigma^{-1} x$ is a quadratic form, closely related to the notion of Mahalanobis distance. It's kind of a generalization of standard Euclidean distance in case when not all directions are equal. This is useful when the variation along one direction is much bigger that along the other one, so small distance along the later one is quite significant.

One can decompose $\Sigma^{-1}$ into $Q^T \Lambda Q = Q^T \Lambda^{1/2} \Lambda^{1/2} Q = (\Lambda^{1/2} Q)^T (\Lambda^{1/2} Q)$ (where $\Lambda$ is diagonal matrix of inverted eigenvalues of $\Sigma$, and $Q$ is orthogonal), so when applied to vectors,

$$ x^T \Sigma^{-1} x = x^T (\Lambda^{1/2} Q)^T (\Lambda^{1/2} Q) x = (\Lambda^{1/2} Q x)^T (\Lambda^{1/2} Q x) = \|\Lambda^{1/2} Q x\|^2 $$

What this formula says is that this quadratic form is equivalent to the standard Euclidean distance in new, morphed space. Namely, we apply the transformation $\Lambda^{1/2} Q$ to transform vector $x$, and then take the norm.

What does this transformation do to our $x$? Well, it consists of 2 "sub-transformation": the first one is a multiplication by $Q$, which just rotates the vector (this is what orthogonal matrices do), preserving its norm; and the second one is a multiplication by $\Lambda^{1/2}$ which stretches vectors along each axes in proportion to the square roots of diagonal elements of $\Lambda$.

enter image description here

The picture above shows what will happen with your 2D datapoints if you apply this transformation (for some $\Sigma$). You can think of Mahalanobis distance as of distance, which implicitly maps your vectors into a new space, measures its norm there and returns back to you.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.