Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I was trying to fit one time-series data (without replicates) using regression model. The data looks like follows:

> xx.2
          value time treat
    1  8.788269    1     0
    2  7.964719    6     0
    3  8.204051   12     0
    4  9.041368   24     0
    5  8.181555   48     0
    6  8.041419   96     0
    7  7.992336  144     0
    8  7.948658    1     1
    9  8.090211    6     1
    10 8.031459   12     1
    11 8.118308   24     1
    12 7.699051   48     1
    13 7.537120   96     1
    14 7.268570  144     1

Because of lack of replicates, I treat the time as continuous variable. Column "treat" shows the case and control data, respectively.

First, I fit the the model "value = time*treat" with "lm" in R:

summary(lm(value~time*treat,data=xx.2))

Call:
lm(formula = value ~ time * treat, data = xx.2)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.50627 -0.12345  0.00296  0.04124  0.63785 

Coefficients:
             Estimate Std. Error t value Pr(>|t|)    
(Intercept)  8.493476   0.156345  54.325 1.08e-13 ***
time        -0.003748   0.002277  -1.646   0.1307    
treat       -0.411271   0.221106  -1.860   0.0925 .  
time:treat  -0.001938   0.003220  -0.602   0.5606    

The pvalue of time and treat is not significant.

While with anova, I got different results:

 summary(aov(value~time*treat,data=xx.2))
            Df Sum Sq Mean Sq F value Pr(>F)  
time         1 0.7726  0.7726   8.586 0.0150 *
treat        1 0.8852  0.8852   9.837 0.0106 *
time:treat   1 0.0326  0.0326   0.362 0.5606  
Residuals   10 0.8998  0.0900                 

The pvalue for time and treat changed.

With linear regression, if I am right, it means the time and treat has no significant influence on value, but with ANOVA, it means time and treat has significant influence on value.

Could someone explain to me why there is difference in these two methods, and which one to use?

share|improve this question
2  
You may want to look up the different kinds of sums of squares. Specifically, I believe linear regression returns type III sum of squares, while anova returns a different kind. –  Max May 22 '12 at 15:50
1  
If you save the results of lm and aov you can check they produce identical fits; e.g., compare their residuals with the residuals function or examine their coefficients (the $coefficients slot in both cases). –  whuber May 22 '12 at 16:13

2 Answers 2

up vote 5 down vote accepted

The fit for lm() and aov() are identical but the reporting is different. The t tests are the marginal impact of the variables in question, given the presence of all the other variables. The F tests are sequential - so they test for the importance of time in the presence of nothing but the intercept, of treat in the presence of nothing but the intercept and time, and of the interaction in the presence of all the above.

Assuming you are interested in the significance of treat, I suggest you fit two models, one with, and one without, compare the two by putting both models in anova(), and use that F test. This will test treat and the interaction simultaneously.

Consider the following:

> xx.2 <- as.data.frame(matrix(c(8.788269, 1, 0,
+ 7.964719, 6, 0,
+ 8.204051, 12, 0,
+ 9.041368, 24, 0,
+ 8.181555, 48, 0,
+ 8.041419, 96, 0,
+ 7.992336, 144, 0,
+ 7.948658, 1, 1,
+ 8.090211, 6, 1,
+ 8.031459, 12, 1,
+ 8.118308, 24, 1,
+ 7.699051, 48, 1,
+ 7.537120, 96, 1,
+ 7.268570, 144, 1), byrow=T, ncol=3))
> names(xx.2) <- c("value", "time", "treat")
> 
> mod1 <- lm(value~time*treat, data=xx.2)
> anova(mod1)
Analysis of Variance Table

Response: value
           Df  Sum Sq Mean Sq F value  Pr(>F)  
time        1 0.77259 0.77259  8.5858 0.01504 *
treat       1 0.88520 0.88520  9.8372 0.01057 *
time:treat  1 0.03260 0.03260  0.3623 0.56064  
Residuals  10 0.89985 0.08998                  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 
> mod2 <- aov(value~time*treat, data=xx.2)
> anova(mod2)
Analysis of Variance Table

Response: value
           Df  Sum Sq Mean Sq F value  Pr(>F)  
time        1 0.77259 0.77259  8.5858 0.01504 *
treat       1 0.88520 0.88520  9.8372 0.01057 *
time:treat  1 0.03260 0.03260  0.3623 0.56064  
Residuals  10 0.89985 0.08998                  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 
> summary(mod2)
            Df Sum Sq Mean Sq F value Pr(>F)  
time         1 0.7726  0.7726   8.586 0.0150 *
treat        1 0.8852  0.8852   9.837 0.0106 *
time:treat   1 0.0326  0.0326   0.362 0.5606  
Residuals   10 0.8998  0.0900                 
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 
> summary(mod1)

Call:
lm(formula = value ~ time * treat, data = xx.2)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.50627 -0.12345  0.00296  0.04124  0.63785 

Coefficients:
             Estimate Std. Error t value Pr(>|t|)    
(Intercept)  8.493476   0.156345  54.325 1.08e-13 ***
time        -0.003748   0.002277  -1.646   0.1307    
treat       -0.411271   0.221106  -1.860   0.0925 .  
time:treat  -0.001938   0.003220  -0.602   0.5606    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 

Residual standard error: 0.3 on 10 degrees of freedom
Multiple R-squared: 0.6526,     Adjusted R-squared: 0.5484 
F-statistic: 6.262 on 3 and 10 DF,  p-value: 0.01154 
share|improve this answer
    
Thanks for thorough explanation, it reminds me of the ANCOVA (analysis of covariance). The first step of ANCOVA is to test the interaction between categorical factor and covariate to see if they have identical slope for both condition. It is quite similar to what I did here. In ANCOVA, it gives same pvalue for interaction in t-test and F-test since interaction is the last term in aov. –  hiberbear May 23 '12 at 7:26

Peter Ellis' answer is excellent, but there is another point to be made. The $t$-test statistic (and its $p$-value) is a test of whether $\beta = 0$. The $F$-test on the anova() printout is whether the added variable significantly reduces the residual sum of squares.

The $t$-test is order-independent, while the $F$-test is not. Hence Peter's suggestion that you try the variables in different orders. It is also possible that variables significant in one test may not be significant in the other (and vice-versa).

My sense (and other contributors are welcome to correct me) is that when you're trying to predict phenomena (as in a systems application), you are most interested in reducing variance with the fewest predictors, and therefore want the anova() results. If you are trying to establish the marginal effect of $X$ on $y$, however, you will be most concerned with significance of your particular $\beta$ of interest, and all other variables will just control for alternate explanations your peer reviewers will try to find.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.