DF for regression SS in multiple linear regression when sigma is known

Question

In the typical environment for multiple linear regression, we have that $Y = \mathbf{X}\beta + \epsilon$ where $\epsilon$ is iid $\mathcal{N}(0, \sigma^2I) $ where $\sigma^2$ is unknown. In this case, regression sum of squares (SSR) has $\text{df}= p - 1$ ($\text{df}$ = degrees of freedom) where $p$ is the number of parameters in the model. I have two questions based on this.

1. Why is the $\text{df}= p - 1$, it seems to me that all $p$ of the parameters are free to vary.
2. If $\sigma^2$ is known, is the $\text{df}= p - 2$?

Edit: I am talking about sum of squares due to regression which is defined as $\sum (\hat{y}_i-\bar{y})^2$ this has degrees of freedom p-1 where p is the number of parameters in the model. SST which is defined as $\sum (y_i-\bar{y})^2$ has degrees of freedom n-1 and SSE (sum of squares due to error/residuals) is defined as $\sum (\hat{y}_i-y_i)^2$ and has degrees of freedom n-p.

SST = SSE + SSR and $df_{SST} = df_{SSR}+df_{SSE}$.

Answer 1

There is a common confusion in your understanding of the degrees of freedom in the typical situation. The degrees of freedom for a multiple regression is often written $\text{df}=N-p-1$. I think this is where you are getting the idea that the df for the regression is $p-1$. I typically prefer to write this as $\text{df}=N-(p+1)$ for this reason. Note that $-1*(p+1)=(-p-1)$, hence: $N-(p+1)=N-p-1$. This is a small piece of algebra that I think is slightly less intuitive for people.

Let's work though the degrees of freedom and see if we can make more sense of it:

• If you simply have $N$ data, and are doing nothing with them, then I suppose you could say you have $N~\text{df}$.
• If you wanted to calculate the mean of your data, then you have used $1~\text{df}$, so now you have only $N-1~\text{df}$ left. The standard way of explaining this is that all but one of your data could be any old real number, but in order to end up with the mean you've found, the last datum will be forced to be some particular value--it wouldn't be 'free to vary'.
• So, what if you wanted to fit a simple (i.e., 1 $X$ variable) regression model? Then you are estimating the mean of $Y$, i.e. $\bar y$, and the slope of the relationship between $X$ and $Y$, $\hat\beta_1$. (Most people typically think of this as estimating the slope, $\hat\beta_1$, and the intercept, $\hat\beta_0$, but note that once you have $(\bar x,~\bar y)$ and $\hat\beta_1$, the intercept falls out of this: $\hat\beta_0=\hat\beta_1\bar x-\bar y$; and remember that $\bar x$ is free / doesn't cost any degrees of freedom, because your $X$ variable is assumed to be a set of known constants.)
• And what if you wanted to fit a multiple regression model with $p$ $X$ variables? Then you would be estimating the mean of $Y$ and $p$ slopes, i.e., $\bar y$ and $\hat\beta_j$ (or $p$ slopes, $\hat\beta_j$, and the intercept, $\hat\beta_0$, if that's more intuitive for you). Thus, you would have $N-(p+1)~\text{df}$ remaining.
• Importantly, recognize that nowhere in this discussion have we discussed estimating $\hat\sigma^2_\varepsilon$ yet, nor is that part of calculating the $\text{df}$ for our model.

The residual variance has to do with how we form intervals. For example, if you wanted to form a confidence interval around a slope, $\hat\beta_j$, to test whether it was equal to some null value (typically $0$). If you know the residual variance a-priori, and your residuals are normal (or, your sample is sufficiently large for the Central Limit Theorem to cover for you), then you can use the normal distribution to form this CI / to test your slope. On the other hand, if you estimated your residual variance from your data, then you would use the $t$ distribution for this, with the model's $\text{df}$.

---

(Response to edit:)

I don't quite follow the thinking behind your question; I continue to suspect there is a misunderstanding at play. The ANOVA table for a regression model, e.g., is constructed like this:

\begin{array}{lllll} &\text{Source} &\text{SS} &\text{df} &\text{MS} &\text{F} \\ \hline &\text{Regression} &\sum(\hat y-\bar y)^2 &p &\frac{\text{SS}_{reg}}{\text{df}_{reg}} &\frac{\text{MS}_{reg}}{\text{MS}_{res}} \\ &\text{Residual} &\sum(y_i-\hat y)^2 &N-(p+1) &\frac{\text{SS}_{res}}{\text{df}_{res}} \\ &\text{Total} &\sum(y_i-\bar y)^2 &N-1 \end{array}

To be explicit:

• The sum of squares due to regression has $p\text{ df}$, not $p-1$.
• Knowledge of $\sigma^2_\varepsilon$ (or the lack thereof) is unrelated to the $\text{df}$.
• The parameters aren't free to vary, the data are.

Answer 2

I presume you mean by your $p$ the number of data points, which is usually denoted by $n$ (parameters refers to something different in statistics and are here conventionally denoted by $p$). The data fix the relationship between the $n$ points and hence they fix the SSR as a single value (boundary condition). Therefore, you have $n-1$ degrees of freedom. If you then need to estimate $\sigma^2$ from these as well, you lose another DoF, i.e., $n-1-1$.