Off diagonal elements of correlation and data rank. I have a data matrix M, and it's correlation matrix $M^TM$.  If the correlation matrix ($M^TM$) has higher magnitude elements off the diagonal, does this imply anything about the rank of the data matrix $M$?
 A: As the comments have pointed out, I'm going to assume that your data matrix $M \in \mathbb R^{n \times p}$ is normalized so that $M^T M$ actually is the correlation matrix. whuber pointed out that the diagonal of $M^T M$ is necessarily 1 and all other elements are at most 1 in absolute value, so it cannot be that off-diagonal elements are larger than the diagonal ones. I'm instead going to answer what happens when there are off-diagonal elements that are large in absolute value. 
Note that $M^T M$ is always positive semi-definite, and is positive definite if and only if $M$ is full rank. This means that we can answer your question about the rank of $M$ by looking at how "far" from being positive definite $M^T M$ is. This exactly answers the question of how far from full-rank $M$ is. See here for a proof that $\text{rank} \ M^T M = \text{rank} \ M$. 
If two columns of $M$ are perfectly correlated then we'll have an off-diagonal 1 in $M^T M$ (well, 2 1's by symmetry) and $M$ has rank of at most $p-1$ now (assuming $p < n$), because two columns are collinear. So we can relate the number of exact 1's and -1's in $M^T M$ to the number of collinear pairs of columns, but this doesn't tell us about larger sets of columns of $M$ that are collinear. We could easily have three columns of $M$ being perfectly collinear without any pair being collinear. To assess that we should look at the eigenvalues of $M^T M$.
By the spectral theorem we have $M^T M = Q \Lambda Q^T$ where $Q$ is an orthonormal matrix of eigenvectors and $\Lambda$ is a diagonal matrix containing the eigenvalues $\lambda_i$, ordered from largest to smallest. $M^T M$ is PD iff all $\lambda_i > 0$; by $M^T M$ being PSD we know all $\lambda_i \geq 0$. This means that the rank of $M$ is equal to the number of non-zero eigenvalues $M^T M$ has. What's nice about this is that looking for 1's and -1's in $M^T M$ will tell us if pairs of columns are collinear, but looking at the eigenvalues will tell us if any arbitrary combinations of columns are collinear and that includes the pairs that the correlations would detect. 
So, in a nutshell, you can look at the off-diagonal elements of $M^TM$ to learn about the rank of $M$, but its eigenvalues will be far more informative.
As a final comment, if you actually compute the eigenvalues of some observed matrix $M$, you likely won't find any eigenvalues exactly equal to 0 due to numerical rounding. Generally though it's pretty obvious which ones should be exactly 0. If you use R there's a function zapsmall which is often used for this.
