Suppose I have a regression model with several predictors ind interactions thereof. For concreteness, suppose we are studying a company's data on salaries and we have predictors dept, title, sex, age, and exper (so there are 3 factors and two covariates). We settle on a model expressed SAS-style as

h(salary) = dept title sex age exper
            dept*sex dept*age title*exper title*age 
            sex*age sex*exper age*exper
            dept*sex*age sex*age*exper

This model includes many, but not all, interactions up through third-order, and suppose we find that for a suitable transformation h, the fit statistics and residual plots all look good.

The question to be answered is "does sex affect salary?" I think many people would answer based on, say, the "Type II" $F$ test for sex. But suppose I make the question more specific by asking "if sex were not taken into account at all, would it make any difference?" This question suggests comparing the above model with the reduced model where all terms involving sex are excluded:

h(salary) = dept title age exper
            dept*age title*exper title*age 

A test comparing these models would have quite a few numerator degrees of freedom, whereas the Type II test of sex has only one. Of course, I do think a lot of people would also look at the individual tests of each of the terms that involve sex. But it is possible that an overall test of all of these terms together could have a smaller $P$ value than any of the individual tests.

My question is: Is there a name for this kind of reduction-of-model test, where the full model is compared with a reduced model where a predictor is completely excluded? And do lots of people do this kind of thing routinely, and I should cringe in embarrassment for not having seen it much? I'd think people might be wise to consider such a test more often. Any comments?

  • $\begingroup$ Dare I say : Partial F-test type III? (aka Yates' weighted squares of means) From Analysis of Messy Data, Vol. I, by Milliken and Johnson. (circa 1984...) $\endgroup$ – usεr11852 Feb 23 '16 at 23:30
  • $\begingroup$ Please dare -- but I question if this is the same thing. The numerator d.f. for the test I'm talking about is a lot more than 1 -- it's the sum of the df for sex, dept*sex, ..., sex*age*exper. By comparison, am I not correct that the type III test for sex is a test of a single contrast of the cell means that gives equal weights to the levels of interactions that contain sex and zero weights to effects that don't involve sex. But I'll look that up in M & J when I get a chance. $\endgroup$ – rvl Feb 24 '16 at 0:23
  • $\begingroup$ Yes, you are correct but I think your test falls under the same category. In any case what you propose does not seem wrong as the smaller model is clearly nested within the other. $\endgroup$ – usεr11852 Feb 24 '16 at 2:15
  • $\begingroup$ Clearly it is a legit test in terms of the nested models. But given that it is a reduction-of-model test comparing a large model with a much smaller one, in what sense could it be in the same category as a test of a single contrast??? $\endgroup$ – rvl Feb 24 '16 at 2:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.