Although it has been already discussed that $\textbf{A}^T\textbf{A}$ has the meaning of taking dot products, I would only add a graphical representation of this multiplication.

Indeed, while rows of the matrix $\textbf{A}^T$ (and columns of the matrix $\textbf{A}$) represent variables, we treat each variable measurements as a multidimensional vector. Multiplying the row $row_p$ of $\textbf{A}^T$ with the column $col_p$ of $\textbf{A}$ is equivalent to taking the dot product of two vectors: $dot(row_p, col_p)$ - the result being the entry at position $(p,p)$ inside the matrix $\textbf{A}^T \textbf{A}$.

Similarly, multiplying the row $p$ of $\textbf{A}^T$ with the column $k$ of $\textbf{A}$ is equivalent to the dot product: $dot(row_p, col_k)$, with the result at position $(p,k)$.

The entry $(p, k)$ of the resulting matrix $\textbf{A}^T\textbf{A}$ has the meaning of how much the vector $row_p$ is in the direction of the vector $col_k$. If the dot product of two vectors $row_i$ and $col_j$ is other than zero, some *information* about a vector $row_i$ is *carried* by a vector $col_j$, and vice versa.

This idea plays an important role in Principal Component Analysis, where we want to find a new representation of our initial data matrix $\textbf{A}$ such that, there is no more information carried about any column $i$ in any other column $j \neq i$. Studying PCA deeper, you will see that a "new version" of the covariance matrix is computed and it becomes a diagonal matrix which I leave to you to realize that... indeed it means what I expressed in the previous sentence.

[![enter image description here][1]][1]

  [1]: https://i.sstatic.net/m2xCt.png