Linear Regression - Error Term Variance - Number of predictors I am trying to prep myself for data science interview & I saw the following question on a forum 
In the case of least square regression, how does the variance of the error term change with the number of predictors?
I am not sure how to answer this? Any ideas?
Any help will be appreciated 
 A: The variance of the error term decreases (or, at worst, does not increase) when you add more regressors.
The reason is that a new variable can explain some more variability in the data that wasn't explained by previous regressors. This will reduce the unexplained variations in the data, whch will cause the variance of the error term to decrease.
A: It doesn't increase. See these examples. Also, look at adjusted $R^2$, which doesn't have this defficiency. 
Here's the MATLAB code example to show how adding a completely irrelevant and inexplanatory variable increases ordinary $R^2$, but decreases Adjsuted $R^2$.
The true process is $y=1+t+\varepsilon$, where $\varepsilon\sim\mathcal{N}(0,1)$. I estimate the true model, then estimate the model with an extra variable $y=1+t+z+\varepsilon$, where $z\sim\mathcal{N}(0,1)$, basically a noise. Yet, this garbage variable inflates ordinary $R^2$ goodness-of-fit metric! 
rng(1)
X=(1:100)'/50;
beta = [1 1]';
y=X+ones(100,1)+randn(100,1);
mdl = fitlm(X,y);
disp 'Correct model'
mdl.Rsquared

X = [X rand(100,1)];
mdl = fitlm(X,y);
disp 'Extra variable model'
mdl.Rsquared

OUTPUT
Correct model

ans = 

    Ordinary: 0.2926
    Adjusted: 0.2854

Extra variable model

ans = 

    Ordinary: 0.2936
    Adjusted: 0.2790

