0
$\begingroup$

I have one response variable $Y$ and one predictor $X$. I am trying to fit a polynomial regression model and try to compare different model with different highest power term, the output of ANOVA in R is the following

Analysis of Variance Table
Model 1: Y ~ X
Model 2: Y ~ X + I(X^2)
Model 3: Y ~ X + I(X^2) + I(X^3)
Model 4: Y ~ poly(X, 5)

  Res.Df    RSS   Df  Sum of Sq       F    Pr(>F)    
    504    19472                                   
    503    15347  1    4125.1     151.693 < 2.2e-16 ***
    502    14616  1     731.8     26.909 3.104e-07 ***
    500    13597  2    1018.4     18.726 1.438e-08 ***

Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

I understand how to derive all these numbers in the table, but there are some contradiction. Here is my question: in this table, the last model is the biggest model in the sense that it contains all the predictors in all previous models, the "Sum of Sq" for Model 4 is 1018.4 = 14616-13597, i.e., the difference of the sum of residuals between model 3 and model 4 and the F statistic for Model 4, which is 18.726 is obtained by $\frac{RSS_3-RSS_4}{502-500}\div\frac{13597}{500}$, i.e., the difference in RSS between model 3 and model 4 divide by the difference of degree of freedom and then divide by the MSE of model 4. This makes a lot of sense. However, when I compute the F statistic for model 3, I am so confused. The "Sum of Sq" for model 3 is obtained via $731.8=15347-14616$, i.e,. the difference in RSS of model 2 and model 3. But the F statistic for model 3 is obtained via $\frac{RSS_2-RSS_3}{503-502}\div\frac{13597}{500}$, i.e., the difference in RSS of model 2 and model 3 divide by their difference in degree of freedom, BUTTTT then divide by the MSE of model 4. In my mind, it should finally divide the MSE of model 3 rather than model 4, since we are comparing the difference between model 2 and model 3.

$\endgroup$
  • $\begingroup$ Just some guess: doesn't the "poly"-function include a constant term (at $x^0$) ? Is so, then we have even - so-to-say - qualitatively different models. $\endgroup$ – Gottfried Helms Feb 3 '16 at 12:26
  • $\begingroup$ Yes, 'poly' function does have $x^0$ term, so the last model is the "full" model in this case among all these models $\endgroup$ – KevinKim Feb 3 '16 at 15:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.