20
$\begingroup$

In answer to a question on model selection in the presence of multicollinearity, Frank Harrell suggested:

Put all variables in the model but do not test for the effect of one variable adjusted for the effects of competing variables... Chunk tests of competing variables are powerful because collinear variables join forces in the overall multiple degree of freedom association test, instead of competing against each other as when you test variables individually.

What are chunk tests? Can you give an example of their application in R?

Improve this question
$\endgroup$
3
  • 7
    $\begingroup$ I'd imagine this refers to testing blocks of variables at a time, where all potential collinearity occurs within a block, which does not affect omnibus tests like the $F$-test or the likelihood ratio test, but I've never heard the term "chunk test". $\endgroup$
    – Macro
    Commented May 1, 2012 at 2:40
  • 4
    $\begingroup$ One situation that I have seen them suggested (and have done them myself) is when you have a series of many mutually exclusive dummy variables (e.g. a nominal independent variable with many potential categories). A test of any individual coefficient associated with any single dummy variable is not very interesting, as it tests a particular contrast with whatever you choose as the baseline. Hence a more informative test is the likelihood ratio test of the restricted model omitting all of the dummy variables. $\endgroup$
    – Andy W
    Commented May 1, 2012 at 2:52
  • $\begingroup$ @AndyW Why would a test on an individual coefficient not be interesting because it tests the contrast with the chosen baseline? Isn't that an interesting comparison to make in many situations? Additionally, isn't the fact that it is more precise than testing whether they are all 0 or not useful? $\endgroup$
    – Geoff
    Commented Aug 31, 2023 at 12:37

2 Answers 2

Reset to default
18
$\begingroup$

@mark999 provided an excellent answer. In addition to jointly testing polynomial terms, you can jointly test ("chunk test") any set of variables. Suppose you had a model with competing collinear variables tricep circumference, waist, hip circumference, all measurements of body size. To get an overall body size chunk test, you could do

    require(rms)
    f <- ols(y ~ age + tricep + waist + pol(hip, 2))
    anova(f, tricep, waist, hip)  # 4 d.f. test

You can get the same test by fitting a model containing only age (if there are no NAs in tricep, waist, hip) and doing the "difference in $R^2$ test". These equivalent tests do not suffer from even extreme collinearity among the three variables.

Improve this answer
$\endgroup$
0
16
$\begingroup$

Macro's comment is correct, as is Andy's. Here's an example.

> library(rms)
> 
> set.seed(1)
> d <- data.frame(x1 = rnorm(50), x2 = rnorm(50))
> d <- within(d, y <- 1 + 2*x1 + 0.3*x2 + 0.2*x2^2 + rnorm(50))
> 
> ols1 <- ols(y ~ x1 + pol(x2, 2), data=d) # pol(x2, 2) means include x2 and x2^2 terms
> ols1

Linear Regression Model

ols(formula = y ~ x1 + pol(x2, 2), data = d)

                Model Likelihood     Discrimination    
                   Ratio Test           Indexes        
Obs       50    LR chi2     79.86    R2       0.798    
sigma 0.9278    d.f.            3    R2 adj   0.784    
d.f.      46    Pr(> chi2) 0.0000    g        1.962    

Residuals

    Min      1Q  Median      3Q     Max 
-1.7463 -0.4789 -0.1221  0.4465  2.2054 

          Coef   S.E.   t     Pr(>|t|)
Intercept 0.8238 0.1654  4.98 <0.0001 
x1        2.0214 0.1633 12.38 <0.0001 
x2        0.2915 0.1500  1.94 0.0581  
x2^2      0.2242 0.1163  1.93 0.0602  


> anova(ols1)
                Analysis of Variance          Response: y 

 Factor     d.f. Partial SS MS          F      P     
 x1          1   131.894215 131.8942148 153.20 <.0001
 x2          2    10.900163   5.4500816   6.33 0.0037
  Nonlinear  1     3.196552   3.1965524   3.71 0.0602
 REGRESSION  3   156.011447  52.0038157  60.41 <.0001
 ERROR      46    39.601647   0.8609054

Instead of considering the x2 and x2^2 terms separately, the "chunk test" is the 2-df test which tests the null hypothesis that the coefficients of those terms are both zero (I believe it's more commonly called something like a "general linear F-test"). The p-value for that test is the 0.0037 given by anova(ols1).

Note that in the rms package, you have to specify the x2 terms as pol(x2, 2) for anova.rms() to know that they are to be tested together.

anova.rms() will do similar tests for predictor variables which are represented as restricted cubic splines using, for example, rcs(x2, 3), and for categorical predictor variables. It will also include interaction terms in the "chunks".

If you wanted to do a chunk test for general "competing" predictor variables, as mentioned in the quote, I believe you would have to do it manually by fitting the two models separately and then using anova(model1, model2). [Edit: this is incorrect - see Frank Harrell's answer.]

Improve this answer
$\endgroup$
3
  • 3
    $\begingroup$ (+1) It is also worth noting that testing individual polynomial terms can be problematic because, except in special cases (like when the predictor,$X$, is symmetric around zero, ${\rm cor}(X,X^2)=0$), they are (often very highly) collinear with the other polynomial terms. $\endgroup$
    – Macro
    Commented May 1, 2012 at 12:12
  • 2
    $\begingroup$ For the sake of clarity, I think I was taught this as a "Partial F-test" where you test 2 or more variables for joint significance. Or whether a subset of variables in your model improves over the more restricted model (just like a likelihood ratio test). Am I correct? $\endgroup$
    – Raynor
    Commented May 1, 2012 at 12:18
  • $\begingroup$ @C.Pieters I don't know if you're correct, but it sounds reasonable. $\endgroup$
    – mark999
    Commented May 2, 2012 at 1:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.