# Understanding the significance of a few results derived in the course of linear regression

I am reading up on linear regression from 18.650 MIT. In the way of explaining it, the professor derives a few results and I have attached them in the image

1. The first result gets used along with the theorem's mentioned towards the end in deriving the distribution of $$\hat{\beta}$$ which consequently gets used in testing the significance of the parameters
2. The unbiased estimator of $$\sigma^2$$, the $$\hat{\sigma}^2$$ gets used in creating the prediction intervals (I have doubts about this. The professor did not mention this explicitly but, I think that is its purpose)

Questions:

1. What is the significance of the remaining three results? As one can see they depend upon $$\sigma^2$$ which is not known hence I cannot use them in their current form. These results were not used subsequently in deriving any other results as well
2. Is the claim I have made above about the significance of the unbiased estimator of the variance true? (The unbiased estimator of $$\sigma$$, the $$\hat{\sigma}$$ gets used in creating the prediction intervals)

1. $$\sigma^2 tr((X^TX)^{-1}$$ is telling us something about how difficult the regression problem is. First, $$tr((X^TX)^{-1})$$ is equal to the sum of the reciprocals of the eigenvalues of $$X^TX$$. So, if $$X^TX$$ is close to singular ($$1/\lambda_i\rightarrow \infty$$), we might have problems. This can occur with collinear columns in $$X$$. With this fixed design problem, we should choose a $$X$$ to avoid this. So, the difficulty of this problem is a function of the design of $$X$$ and the observation variance.
2. $$\sigma^2(n-p)$$ is telling us how the problem is affected by the dimensionality of the task. Consider $$n=p$$ though, for each point, you may simply pass your regression through each point. This may minimize prediction error on your observations at the cost of overfitting. In a sense, we may always minimize the observed prediction error by increasing the number of predictors. Again, if $$\sigma^2$$ is large, then we will on average make more errors in our prediction.
3. The theorem you have not discussed is used to derive the t-distribution for inference on $$\hat{\beta}$$, when $$\sigma^2$$ is unknown.