Although it has been already discussed that $\textbf{A}^T\textbf{A}$ has the meaning of taking dot products, I would only add a graphical representation of this multiplication.
Indeed, while the rows of the matrix $\textbf{A}^T$ (and columns of the matrix $\textbf{A}$) represent variables, we treat each variable measurements as a multidimensional vector. Multiplying the row $p$ of $\textbf{A}^T$ with column $p$ of $\textbf{A}$ is equivalent to taking a dot product of two vectors: $dot(A_{T, row_p}, A_{col_p})$.
Similarly, multiplying the row $p$ of $\textbf{A}^T$ with column $k$ of $\textbf{A}$ is equivalent to the dot product: $dot(A_{T, row_p}, A_{col_k})$.
The element $(p, k)$ of the resulting matrix $\textbf{A}^T\textbf{A}$ has the meaning of how much the vector $A_{T, row_p}$ is in the direction of the vector $A_{col_k}$. If the dot product of two vectors $A_{T, row_i}$ and $A_{col_j}$ is other than zero, some information about a vector $A_{T, row_i}$ is carried by a vector $A_{col_j}$, and vice versa.
This idea plays an important role in Principal Component Analysis, where we want to find a new representation of our initial data matrix $\textbf{A}$ such that, there is no more information carried about any column $i$ in any other column $j \neq i$.
