4
$\begingroup$

I am doing a multiple linear regression where the predictors are two categorical variables: Time (3 levels) and Treatment (3 levels). The lm() results are:

Coefficients:
                Estimate Std. Error t value Pr(>|t|)    
(Intercept)      -1.6227     0.2627  -6.176 1.14e-06 ***
TimeT2           -0.3453     0.3941  -0.876 0.388383    
TimeT3           -0.9509     0.3941  -2.413 0.022628 *  
TreatmentTR2      1.9081     0.4736   4.029 0.000389 ***
TreatmentTR3      1.9485     0.3941   4.944 3.23e-05 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.7882 on 28 degrees of freedom
Multiple R-squared:  0.5163,    Adjusted R-squared:  0.4472 
F-statistic: 7.471 on 4 and 28 DF,  p-value: 0.0003159

In the table the t-test p-values are calculated using the estimated t-value with t-distribution of df=n-k-1. In my data, the total number of observation n=33, number of parameters k=4. So all the t-tests above use the identical df=28.

However, if I just think about the t-test for the purpose of comparing the mean of these two groups after adjusting other covariates, for instance TimeT2 vs TimeT1(ref). I would imagine that the observations from T3 are not relevant for this t-test, thus, the df would only be calculated by the number of observations from T1 and T2 and k=1.

Think it in another way, if my purpose is to compare pairwise means of multiple groups and I have three groups with large difference in n (n_T1=10, n_T2=10, n_T3=200), conducting a pairwise t-test between observations from T1 and T2 only would end up with much lower df, as compared to looking at the t-test result in the lm output. In the separated t-test, we end up with df=18, while lm gives df=217. Isn't the lm t-test deflates the p-value?

Thanks

$\endgroup$

1 Answer 1

6
$\begingroup$

I would imagine that the observations from T3 are not relevant for this t-test

While you're right in thinking that observations from T3 don't participate in the comparison of means in the numerator of the t-statistic, the t-statistic has both a numerator and a denominator.

Because of the assumption of constant variance in the regression model, all the residuals have information about the variance estimate used in the denominator, so the observations all participate in its estimate.

The degrees of freedom for the t-test come from the degrees of freedom in the estimate of the error variance. As suggested above, for regression this is based on the model using all observations, and does not relate to the subset of observations that participate in the comparison in the numerator of a t-statistic.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.