I'm studying regression analysis but I'm struggling with really understanding how degrees of freedom are calculated. For example, if we have the simple scenario where $Y_i=\beta_0+\beta_1 X_i + \epsilon_i$ (and all the standard assumptions hold) then I read

$\frac{1}{\sigma^2} \sum_{i=1}^n (\hat{Y}_i - \bar{Y})^2 \sim \chi^2_{1}$

This seems reasonable when you make an argument like "$\hat{Y}_i$ has two parameters and so two degrees of freedom but $\bar{Y}$ takes one degree of freedom and so you're left with 1", but I guess I'm looking for an argument that's more theoretically grounded. Why does that summation have the same distribution as a standard normal squared?

I was able to understand why $\frac{1}{\sigma^2} \sum_{i=1}^n (Y_i-\bar{Y})^2 \sim \chi^2_{n-1}$ by considering the sum of squares as a projection of the $\epsilon_i$ onto a space of dimension $n-1$. A proof for the above case that follows that kind of argument would be fantastic!

  • $\begingroup$ But that's just the rest of the Pythagorean Theorem! Your first expression is the projection in the orthogonal direction, a space of dimension $n-(n-1) = 1$. $\endgroup$ – whuber Feb 2 '14 at 19:40
  • $\begingroup$ That sounds like a really slick proof! But, I'm having trouble seeing how the first sum is orthogonal to the second... Can you explain a bit more? And thanks! $\endgroup$ – random_forest_fanatic Feb 5 '14 at 11:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.