What is the R squared of a regression where none of the variables are collinear? If a regression has 100 observations and 100 variables and none of them are collinear, what is the r squared? I know that it means that the full rank assumption is satisfied, so does this mean the r squared equals 1? Also, what would the t-statistic for the 5th coefficient be?
 A: (Most of this a linear algebra question is disguise!)
If the $100\times100$ matrix $X$ is full-rank, that means the columns form a basis for $\mathbb R^{100}$. Since $y\in\mathbb R^{100}$, $y$ can be written as some linear combination of any basis for $\mathbb R^{100}$, such as the set of columns of $X$.
That is, the columns of $X$ perfectly predict $y$, and there is no prediction error (at least not in-sample). Consequently, $y=\hat y$, and $R^2=1$.
$$
R^2=1-\dfrac{
\overset{n}{\underset{i=1}{\sum}}\left(y_i-\hat y_i\right)^2
}{
\overset{n}{\underset{i=1}{\sum}}\left(y_i-\bar y\right)^2
}\\
=1-\dfrac{
\left. \overset{n}{\underset{i=1}{\sum}}\left(y_i-\hat y_i\right)^2 \middle/ n \right.
}{
\left. \overset{n}{\underset{i=1}{\sum}}\left(y_i-\bar y\right)^2 \middle/ n \right.
}
\\
1-\dfrac{
\text{var}(y-\hat y)
}{
\text{var}(y)
}\\
=1-\dfrac{0}{\text{var}(y)}=1
$$
(This assumes not all values of $y$ are equal, but if they are, that is not an interesting regression problem.)
With zero residual variance, it does not make much sense to talk about the t-stats for any of the coefficients, since t-stats divide by residual variance and dividing by zero is frowned upon.
A: If none of the explanatory variables is colinear (i.e., each pair has zero correlation) then the coefficient-of-determination for the regresion is equal to the sum of the coefficients-of-determination for individual simple linear regessions of each explanatory variable against the response variable.  If you would like to learn more about the relationships between collinearity and the coefficient-of-determination in linear regression, as wel as a broader geometric view of regression, you can find some discussion of this topic in my paper O'Neill (2019).
