0
$\begingroup$

I have been reading Introduction to statistical learning and I was going through Multiple Linear Regression. This is the topic that Im reading enter image description here As i was further reading I encountered an equation that Im not able to understand. Below is that equation enter image description here

It further say that, enter image description here

Please try to explain explain the above mentioned problem as simple as possible.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

This method is called comparing the full and reduced model. Let's say for example, we have four variables and we want to check out if the fourth variable is significant or not. The NULL hypothesis is $\beta_4=0$. What you're doing is comparing the FULL model $\beta_1+\beta_2+\beta_3+\beta_4$ to a REDUCED model it.i.e.$\beta_1+\beta_2+\beta_3$.

If there is a significant difference between the two then clearly $\beta_4$ added something over and above the REDUCED model. In which case we reject the null hypothesis and conclude $\beta_4\neq0$. Also, instead of just testing $\beta_4$, i can test $\beta_3$ AND $\beta_4$ at the same time by comparing the FULL model of $\beta_1+\beta_2+\beta_3+\beta_4$ to a REDUCED model of $\beta_1+\beta_2$, and so on. That's why the authors are indexing it as $q$ and $p$ because either $q$ or $p$ can go up to any length.

The F test basically takes the residual sums of squares of the reduced model and subtracts the the residual sums of squares of the full model, and then compares it (through division) to the residual sums of squares of the full model. (after appropriately dividing each of them by the appropriate degrees of freedom). Remember, residual sums of squares is a measure of how well/unwell the model fit, because the residual is what's left over after you subtract your predicted value from your observed value.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ There is nothing clear about $\beta_4$ from a non statistically significant result. And $p>\alpha$ is not the same as $\beta_4=0$. In fact, based on what we know of the world, $\beta_4\neq0$, since most variables are always related, even if trivially. $\endgroup$
    – Heteroskedastic Jim
    Commented Nov 27, 2018 at 14:08
  • $\begingroup$ fair enough, good point, I have concluded the null hypothesis which i shouldn't have. What i should have said was that if the two models are significantly different from each other then $\beta_4$ IS adding something significant from the reduced model. Thanks for the tip I'll edit the answer. $\endgroup$
    – Huy Pham
    Commented Nov 27, 2018 at 14:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.