Does covariance of a data rotated by some angle to the orginal data,have the information of rotation. i.e I read in this article(http://www.visiondummy.com/2014/04/geometric-interpretation-covariance-matrix/) (assuming S as an Identity matrix in the link) that covariance of the rotated data is nothing but \begin{equation*} \Sigma = R \, R^{-1} \end{equation*}

  • 1
    $\begingroup$ What do you mean by, "have the information of rotation?" $\endgroup$ – Matthew Gunn Sep 26 '16 at 23:22

Let $\Sigma$ be your covariance matrix for a random vector $\mathbf{x}$.

Now let's consider an eigenvalue decomposition of $\Sigma$ such $Q\Lambda Q' = \Sigma$ where $\Lambda$ is a diagonal matrix and $Q$ is an orthogonal matrix (i.e. $QQ'=I$). The intuitive interpretation of $Q$ is that it is a rotation matrix (or an improper rotation matrix) and that $\Lambda$ acts a scaling matrix.

Let $\mathbf{z} = \sqrt{\Lambda^{-1}} Q' \mathbf{x}$. That is, we rotate $\mathbf{x}$ using $Q'$ and scale it by the square root of the inverse of $\Lambda$. Then: $$\mathrm{Cov}\left(\mathbf{z} \right) = I$$ The different elements of the random vector $\mathbf{z}$ are orthogonal and have variance = $1$.

Alternatively, we could go the other direction. We could start with an orthogonal random vector $\mathbf{z}$ and obtain a random vector with covariance matrix $\Sigma$ by $\mathbf{x} = Q\sqrt{\Lambda} \mathbf{z}$. Observe that:

$$\begin{align*} \mathrm{Cov}\left( \mathbf{x} \right) &= \mathrm{Cov}\left( Q\sqrt{\Lambda} \mathbf{z} \right) \\ &= Q \sqrt{\Lambda} \mathrm{Cov}\left( \mathbf{z} \right) \sqrt{\Lambda}Q' \\ &= Q \sqrt{\Lambda} I \sqrt{\Lambda} Q' \\ &= Q \Lambda Q' \\ &= \Sigma \end{align*}$$

You can do this as long as $\Sigma$ is positive definite, that is, the elements of your random vector $\mathbf{x}$ aren't linearly dependent.


You can go back and forth between a random vector $\mathbf{z}$ where $\mathrm{Cov}(\mathbf{z}) = I$ and a random vector $\mathbf{x}$ where $\mathrm{Cov}(\mathbf{x}) = \Sigma$ using $Q$ and $\Lambda$ obtained from an eigenvalue decomposition of $\Sigma$. $Q$ and $\Lambda$ are uniquely defined for a positive definite covariance matrix $\Sigma$. If you rotate and scale $x$ to obtain some $z$ then toss $\Sigma$ and $Q$, there's no magic way to go back.

  • $\begingroup$ Q is an orthogonal matrix and QQ′=I but why is Q is a rotation matrix because if its a rotational matrix then it will be something of the form \begin{bmatrix}\cos \theta &-\sin \theta \\\sin \theta &\cos \theta \\\end{bmatrix} \begin{bmatrix}-0.8656 &0.5007 \\0.5007 &0.8656 \\\end{bmatrix} and the matrices like above even though they are orthogonal they can not be rotational matrices because cos theta is posititve and negative at the same theta $\endgroup$ – Niranjan Kotha Sep 27 '16 at 17:59
  • $\begingroup$ @NiranjanKotha That is an improper rotation. It's an orthogonal matrix with a determinant of -1. But you're right in that an orthogonal matrix isn't the exact same thing as a rotation matrix. $\endgroup$ – Matthew Gunn Sep 27 '16 at 19:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.