# If $\mathbf{x} \sim N(\mathbf{0,I})$ and $\mathbf{y} = \mathbf{Ax}$, what does $\mathbf{A}^T \mathbf{A}$ represent?

If $\mathbf{x} \sim N(\mathbf{0,I})$ then $\mathbf{AA}^T$ is the covariance matrix of $\mathbf{y} = \mathbf{Ax}$, but what does $\mathbf{A}^T \mathbf{A}$ represent?

In some places I have seen statements like "if $\mathbf{X}$ is the data, then $\mathbf{X}^T \mathbf{X}$ is (proportional/related to) the covariance matrix of the data." But here $\mathbf{A}$ is not data, it is a matrix of coefficients.

I was dealing with a problem where I had to reduce the size of $\mathbf{y}$ from $n$ to $1$ and it was solved via a PCA for $\mathbf{A}^T \mathbf{A}$ such that the reduce one-dimensional $y^*$ is given by $y^* = \mathbf{b}^T\mathbf{x}$ where $\mathbf{b}$ is the (normalised) eigenvector corresponding to the largest eigenvalue of $\mathbf{A}^T \mathbf{A}$.

• Is there a reason to expect it has any meaningful interpretation? May 15, 2018 at 9:17
• @shimao Thank you for your comment. I was dealing with a problem where I had to reduce the size of $\mathbf{y}$ from $n$ to $1$ and it was solved via a PCA for $\mathbf{A}^T \mathbf{A}$ such that the reduce one-dimensional $y^*$ is given by $y^* = \mathbf{b}^T\mathbf{x}$ where $\mathbf{b}$ is the (normalised) eigenvector of corresponding to the largest eigenvalue of $\mathbf{A}^T \mathbf{A}$ . (I will add this to the post). May 15, 2018 at 9:23
• This is just a way to compute the 1st Principal Component of $AA^T$. May 15, 2018 at 9:32
• @Elvis Thank you for your comment. $A$ is an $n \times m$ matrix so that the dimensions of $A^TA$ ($m \times m$) and $AA^T$ ($n \times n$) are not the same, so I don't think that the PCAs of the two are the same. May 15, 2018 at 9:36
• Yes, my bad, that's the PCs of A, which are proportional to the eigenvalues of $AA^T$. May 15, 2018 at 11:37

## Expansion of the term

Since, based on your comments, $\mathbf{A}$ has dimensions $n*m$ $\mathbf{A}^T \mathbf{A}$ expands to: $$A_{1,1}*A_{1,1} + …A_{n,1}*A_{n,1} ,... A_{1,1}*A_{1,m} + …A_{n,1}*A_{n,m}$$ $$A_{1,m}*A_{1,1} + …A_{n,m}*A_{n,1} ,... A_{1,m}*A_{1,m} + …A_{n,m}*A_{n,m}$$

## Interpretation of the expansion

The diagonal is the sum of squares for the coefficients of your regression, so the larger the magnitude of the contribution a coefficient is to your relationship, the larger the value will be.

## Limitations on the interpretation

Initially this sounds like a useful way to gauge the importance of each variable to the regression. What this actually means will depend heavily on your data types and pretreatment of variable prior to the regression and so is not usually used to assess variable importance. There is a lot of literature out there on variable importance and contribution.

https://www.sciencedirect.com/science/article/pii/S0951832015001672

https://aichamp.wordpress.com/2017/03/26/variable-importance-and-how-it-is-calculated/

• Thank you for you reply. In my situation, actually, all rows of $A$ are normalised to be unit vectors, so the diagonal of $A^TA$ is 1. So, $A^TA$ looks like a correlation matrix but it is not clear of what. May 16, 2018 at 9:58