Can I run multiple regression with mostly categorical independent variables? I'd appreciate some examples to cite this approach as one of my reviewers is insisting that I justify the use of multiple regression for categorical IVs.

2 DVs (test scores) interval

IVs -
1 score (1-5 scale)
Gender dummy coded
Age dummy coded 3 levels
Member of ethnic minority dummy coded
Time spent out of country dummy coded 4 levels
Academic Discipline dummy coded 4 levels

Since the independent variables are mixed (mostly categorical) is multiple regression still acceptable? I want to explore the relationship of the IVs independently and in combinations on the test scores (DVs).

  • $\begingroup$ Just out of curiosity, what is the alternative your reviewer is suggesting you use instead? As @rolando2 correctly notes, an ANOVA is identical to a multiple regression with all categorical IVs. $\endgroup$ – gung - Reinstate Monica Oct 6 '12 at 19:17
  • $\begingroup$ ANCOVA but others are recommending Multiple Regression... The concern is the use of so many non-parametric variables for a parametric test $\endgroup$ – Dawn Oct 6 '12 at 21:00
  • $\begingroup$ ANCOVA is identical to multiple regression with some categorical and some continuous IVs, but no interactions with the continuous IVs. $\endgroup$ – gung - Reinstate Monica Oct 6 '12 at 22:49
  • $\begingroup$ Thanks gung, this is why I want to use MR and not Anova or Ancova or Manova $\endgroup$ – Dawn Oct 7 '12 at 22:32

Can you? Sure. But it'll no doubt be less work to use ANOVA. Mathematically, ANOVA is equivalent to regression; they're versions of the same general linear model. The mechanics vary depending on the software you're using, but ordering up an interaction (to show effects associated with combinations of predictors) is simpler if you use, e.g., Age*Academic Discipline in ANOVA than if you have to account for the many dummy variables you've created for regression.

  • $\begingroup$ Why would it be less work in ANOVA? All the statistics programs I am aware of create dummies automatically. $\endgroup$ – Peter Flom - Reinstate Monica Oct 6 '12 at 18:16
  • $\begingroup$ I created the dummies manually. AND there is one ordinal IV. Woudn't I want to use ANCOVA rather than ANOVA? And why not Multiple Regression if the categorical IVs are dummied? $\endgroup$ – Dawn Oct 6 '12 at 18:27
  • $\begingroup$ @PeterFlom - Not all stat programs do create dummies for a regression procedure, nor do they all automatically create, for regression, all the cross-product variables that would be necessary to comprehensively test interactions. But for ANOVA the procedure is often much simpler. E.g., entering Age*Academic Discipline in SPSS ANOVA tests all interactions of a 3-dummy variable and a 2-dummy variable: 6 product terms that for regression per se would need to be computed and then separately entered. $\endgroup$ – rolando2 Oct 6 '12 at 19:26
  • $\begingroup$ @rolando2 Huh. I didn't know that. In SAS or R regression would be just as easy as ANOVA. $\endgroup$ – Peter Flom - Reinstate Monica Oct 6 '12 at 20:51
  • $\begingroup$ According to Powers & Xie (2008) ANOVA is statistically equivalent to Multiple Regression but ANOVA coding constrains the sum of dummy variable effects to equal zero. The intercept term represents the overall mean with equal group sizes but with unequal group sizes, it is an arbitrarily standardized mean. Some of my categorical groups are unequal in size, so is regression better? $\endgroup$ – Dawn Oct 6 '12 at 20:51

I'm not sure a simple gaussian linear model is relevant because of the nature of the depednant variable. It seems to me that a test score is an "ordinal" variable (i.e. a discrete variable with ordered categories). I would therefore look into methods for ordinal regression instead of simple linear regression like anova. Such methods exist in all standard software.

  • 1
    $\begingroup$ It depends on the score. A score on a 100 item test with 1 point per item is probably close enough to interval to not need ordinal models (not to mention an ordinal model with 100 levels would be a mess). $\endgroup$ – Peter Flom - Reinstate Monica Oct 6 '12 at 18:17
  • $\begingroup$ The scores are 55-135 with decimal points. So you can have a score of 101.8 or 120.2 $\endgroup$ – Dawn Oct 6 '12 at 18:24
  • $\begingroup$ Ok. I guess each individual had to answer a series of questions each with a given scoring weight. The total is then normalized in some way, hence the decimals (?) But putting this problem within the general frame of generalized linear models in order to easily get asymptotic confidence intervals of the estimates, how could one qualify such data? It still is count data! $\endgroup$ – julien stirnemann Oct 6 '12 at 18:45
  • 2
    $\begingroup$ @julienstirnemann - Your point might partly apply to some very short, crude, and clumsily-constructed tests, but I agree with the other comments: this test score sounds as if it can quite safely be treated as an interval-level variable. $\endgroup$ – rolando2 Oct 6 '12 at 19:18
  • 1
    $\begingroup$ The dependent variable does not have to be Gaussian at all; OLS regression assumes the residuals are normal. Virtually nothing is really continuous, as measured. What the model does assume is that the difference between a score of 1 and a score of 2 is the same as between 101 and 102 (or whatever). $\endgroup$ – Peter Flom - Reinstate Monica Oct 6 '12 at 20:54

The answer to your question is YES you can apply ordinal logistic regression to predict DVs using categorical IVs. I am making a BIG ASSUMPTION here your DV have 2 interval something like LOW and HIGH.

I used it once to predict methods used for contraception methods. I assumed proportional odds assumption in our model. The trick is not just fitting all IVs in your model but to identify the IVs which influence DVs.

If you use R language then polr can help you. But in order to identify relevant IVs I used linear regression to find p-value for each IVs and remove IVs which had greater p-value. Ideally the rule I applied was eliminate IV one at a time with high p-value and then run the linear regression again till you find all IVs below p < 0.1.

Once I identified IVs I use polr

Suppose you have identified IVs : IV1 and IV2 and your Response variable is DV then

o_reg <- polr(DV ~ IV1+ IV2, data = mydata, Hess=TRUE)

summary(o_reg) would give you the output in units of ordered logits, or ordered log odds. Its slightly tricky to interpret polr output as it makes some assumption on the relationship between each pair of outcome groups. You need to read through the literature to find out.

Then comes the prediction part. Suppose you had test data as well ( we had testdata ) we used predict function in R to predict the probability of each DV value in your case it could be P(LOW) and P(HIGH).

predict(o_reg, testdata, type = "probs")

We prediction came out to be very accurate.

Again this was our experience this may or may not help.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.