Return to Answer

Although it has been already discussed that $\textbf{A}^T\textbf{A}$ has the meaning of taking dot products, I would only add a graphical representation of this multiplication.

Indeed, while rows of the matrix $\textbf{A}^T$ (and columns of the matrix $\textbf{A}$) represent variables, we treat each variable measurements as a multidimensional vector. Multiplying the row $p$ of $\textbf{A}^T$ with the column $p$ of $\textbf{A}$ is equivalent to taking the dot product of two vectors: $dot(A_{T, row_p}, A_{col_p})$ - the result being the entry at position $(p,p)$ inside the matrix $\textbf{A}^T \textbf{A}$.

Similarly, multiplying the row $p$ of $\textbf{A}^T$ with the column $k$ of $\textbf{A}$ is equivalent to the dot product: $dot(A_{T, row_p}, A_{col_k})$, with the result at position $(p,k)$.

The entry $(p, k)$ of the resulting matrix $\textbf{A}^T\textbf{A}$ has the meaning of how much the vector $A_{T, row_p}$ is in the direction of the vector $A_{col_k}$. If the dot product of two vectors $A_{T, row_i}$ and $A_{col_j}$ is other than zero, some information about a vector $A_{T, row_i}$ is carried by a vector $A_{col_j}$, and vice versa.

This idea plays an important role in Principal Component Analysis, where we want to find a new representation of our initial data matrix $\textbf{A}$ such that, there is no more information carried about any column $i$ in any other column $j \neq i$. Studying PCA deeper, you will see that a "new version" of the covariance matrix is computed and it becomes a diagonal matrix which I leave to you to realize that... indeed it means what I expressed in the previous sentence.

Although it has been already discussed that $\textbf{A}^T\textbf{A}$ has the meaning of taking dot products, I would only add a graphical representation of this multiplication.

Indeed, while rows of the matrix $\textbf{A}^T$ (and columns of the matrix $\textbf{A}$) represent variables, we treat each variable measurements as a multidimensional vector. Multiplying the row $row_p$ of $\textbf{A}^T$ with the column $col_p$ of $\textbf{A}$ is equivalent to taking the dot product of two vectors: $dot(row_p, col_p)$ - the result being the entry at position $(p,p)$ inside the matrix $\textbf{A}^T \textbf{A}$.

Similarly, multiplying the row $p$ of $\textbf{A}^T$ with the column $k$ of $\textbf{A}$ is equivalent to the dot product: $dot(row_p, col_k)$, with the result at position $(p,k)$.

The entry $(p, k)$ of the resulting matrix $\textbf{A}^T\textbf{A}$ has the meaning of how much the vector $row_p$ is in the direction of the vector $col_k$. If the dot product of two vectors $row_i$ and $col_j$ is other than zero, some information about a vector $row_i$ is carried by a vector $col_j$, and vice versa.

This idea plays an important role in Principal Component Analysis, where we want to find a new representation of our initial data matrix $\textbf{A}$ such that, there is no more information carried about any column $i$ in any other column $j \neq i$. Studying PCA deeper, you will see that a "new version" of the covariance matrix is computed and it becomes a diagonal matrix which I leave to you to realize that... indeed it means what I expressed in the previous sentence.

Although it has been already discussed that $\textbf{A}^T\textbf{A}$ has the meaning of taking dot products, I would only add a graphical representation of this multiplication.

Indeed, while rows of the matrix $\textbf{A}^T$ (and columns of the matrix $\textbf{A}$) represent variables, we treat each variable measurements as a multidimensional vector. Multiplying the row $p$ of $\textbf{A}^T$ with the column $p$ of $\textbf{A}$ is equivalent to taking the dot product of two vectors: $dot(A_{T, row_p}, A_{col_p})$.

Similarly, multiplying the row $p$ of $\textbf{A}^T$ with the column $k$ of $\textbf{A}$ is equivalent to the dot product: $dot(A_{T, row_p}, A_{col_k})$.

The element $(p, k)$ of the resulting matrix $\textbf{A}^T\textbf{A}$ has the meaning of how much the vector $A_{T, row_p}$ is in the direction of the vector $A_{col_k}$. If the dot product of two vectors $A_{T, row_i}$ and $A_{col_j}$ is other than zero, some information about a vector $A_{T, row_i}$ is carried by a vector $A_{col_j}$, and vice versa.

This idea plays an important role in Principal Component Analysis, where we want to find a new representation of our initial data matrix $\textbf{A}$ such that, there is no more information carried about any column $i$ in any other column $j \neq i$.

Although it has been already discussed that $\textbf{A}^T\textbf{A}$ has the meaning of taking dot products, I would only add a graphical representation of this multiplication.

Indeed, while rows of the matrix $\textbf{A}^T$ (and columns of the matrix $\textbf{A}$) represent variables, we treat each variable measurements as a multidimensional vector. Multiplying the row $p$ of $\textbf{A}^T$ with the column $p$ of $\textbf{A}$ is equivalent to taking the dot product of two vectors: $dot(A_{T, row_p}, A_{col_p})$ - the result being the entry at position $(p,p)$ inside the matrix $\textbf{A}^T \textbf{A}$.

Similarly, multiplying the row $p$ of $\textbf{A}^T$ with the column $k$ of $\textbf{A}$ is equivalent to the dot product: $dot(A_{T, row_p}, A_{col_k})$, with the result at position $(p,k)$.

The entry $(p, k)$ of the resulting matrix $\textbf{A}^T\textbf{A}$ has the meaning of how much the vector $A_{T, row_p}$ is in the direction of the vector $A_{col_k}$. If the dot product of two vectors $A_{T, row_i}$ and $A_{col_j}$ is other than zero, some information about a vector $A_{T, row_i}$ is carried by a vector $A_{col_j}$, and vice versa.

This idea plays an important role in Principal Component Analysis, where we want to find a new representation of our initial data matrix $\textbf{A}$ such that, there is no more information carried about any column $i$ in any other column $j \neq i$. Studying PCA deeper, you will see that a "new version" of the covariance matrix is computed and it becomes a diagonal matrix which I leave to you to realize that... indeed it means what I expressed in the previous sentence.

Although it has been already discussed that $\textbf{A}^T\textbf{A}$ has the meaning of taking dot products, I would only add a graphical representation of this multiplication.

Indeed, while the rows of the matrix $\textbf{A}^T$ (and columns of the matrix $\textbf{A}$) represent variables, we treat each variable measurements as a multidimensional vector. Multiplying the row $p$ of $\textbf{A}^T$ with column $p$ of $\textbf{A}$ is equivalent to taking a dot product of two vectors: $dot(A_{T, row_p}, A_{col_p})$.

Similarly, multiplying the row $p$ of $\textbf{A}^T$ with column $k$ of $\textbf{A}$ is equivalent to the dot product: $dot(A_{T, row_p}, A_{col_k})$.

The element $(p, k)$ of the resulting matrix $\textbf{A}^T\textbf{A}$ has the meaning of how much the vector $A_{T, row_p}$ is in the direction of the vector $A_{col_k}$. If the dot product of two vectors $A_{T, row_i}$ and $A_{col_j}$ is other than zero, some information about a vector $A_{T, row_i}$ is carried by a vector $A_{col_j}$, and vice versa.

This idea plays an important role in Principal Component Analysis, where we want to find a new representation of our initial data matrix $\textbf{A}$ such that, there is no more information carried about any column $i$ in any other column $j \neq i$.

Although it has been already discussed that $\textbf{A}^T\textbf{A}$ has the meaning of taking dot products, I would only add a graphical representation of this multiplication.

Indeed, while rows of the matrix $\textbf{A}^T$ (and columns of the matrix $\textbf{A}$) represent variables, we treat each variable measurements as a multidimensional vector. Multiplying the row $p$ of $\textbf{A}^T$ with the column $p$ of $\textbf{A}$ is equivalent to taking the dot product of two vectors: $dot(A_{T, row_p}, A_{col_p})$.

Similarly, multiplying the row $p$ of $\textbf{A}^T$ with the column $k$ of $\textbf{A}$ is equivalent to the dot product: $dot(A_{T, row_p}, A_{col_k})$.

The element $(p, k)$ of the resulting matrix $\textbf{A}^T\textbf{A}$ has the meaning of how much the vector $A_{T, row_p}$ is in the direction of the vector $A_{col_k}$. If the dot product of two vectors $A_{T, row_i}$ and $A_{col_j}$ is other than zero, some information about a vector $A_{T, row_i}$ is carried by a vector $A_{col_j}$, and vice versa.

This idea plays an important role in Principal Component Analysis, where we want to find a new representation of our initial data matrix $\textbf{A}$ such that, there is no more information carried about any column $i$ in any other column $j \neq i$.