4
$\begingroup$

If I have a continuous Dependent Variable and two Independent Variables, where one is categorical with three levels and the other is continuous, what assumptions do I need to check for multiple regression?

Scatter plots are for continuous variables and multicollinearity makes sense for continuous, but not for dummy variables.

| cite | improve this question | |
$\endgroup$
  • 7
    $\begingroup$ Multicollinearity makes sense for indicator variables (you say dummy). For example, if you had two indicator variables that were identical you can't distinguish between their effects. They are perfectly correlated too: a scatter point would give two blobs defining a perfect straight line. Approximations to this can be problematic too. Good software will catch the problem for you, but it's a misconception to assert that it makes no sense. In practice you use at most two indicators for a three-level categorical variable, or software does that for you. $\endgroup$ – Nick Cox Feb 23 '15 at 16:24
1
$\begingroup$

The most important assumptions to check are those for any multiple regression, as explained for example in Faraway's "Practical Regression and Anova using R," Chapter 7: tests for outliers and influential observations, a plot of residuals versus fitted values (an extremely useful scatter plot that incorporates both the categorical and the continuous predictor), tests of non-linearity and distributions of residuals, and so forth.

"Multicollinearity" would seem to be a bit of an overstatement with only 2 predictor variables. If you are concerned about collinearity, you could for example see how the values of the continuous predictor are distributed among the 3 levels of the categorical predictor. The Faraway reference noted above discusses collinearity in Chapter 9. As the answer from @jur notes, its practical importance depends on the intended use of the model.

| cite | improve this answer | |
$\endgroup$
1
$\begingroup$

Regression analysis treats the explanatory variables as fixed, and does not make any assumptions about their distribution. In particular, regression does not require the absence of collinearity in the explanatory variables, though that collinearity can affect the interpretation of the output.

As with any application of regression analysis, you need to check the assumptions concerning the error distribution, which is generally check via diagnostic plots of the residuals. It is not necessary to plot the explanatory variables against each other.

| cite | improve this answer | |
$\endgroup$
0
$\begingroup$

Important first question is: What do you want to do with the result of the model? When predicting is your key target, then multicollinearity is not that much an issue. Create dummies for 2 out of 3 levels of the categorical predictor and include them in the predictive model. In most software tools you can include collinearity statistics, to identify potential issues around multicollinearity (VIF / tolerance) to check if this might be impacting interpretation of (significance of) parameters, but that is mainly an issue for an explanatory model not for a purelypredictive model.

Furthermore, in multiple linear regression, it is important to adequately solve missing values and check the distribution of the dependent and independent(s) to identify outliers and evaluate (non-)normality of the distributions. Scatter plots can show to what extent the relation between the DV and IV is linear; an ANOVA or boxplots split on the categories of the IV can show you if there is a bivariate realtionship between the IV's and DV.

| cite | improve this answer | |
$\endgroup$
  • 5
    $\begingroup$ Non-normality of variables is never a problem. It is sometimes (small datasets) a problem when the residuals are very non-normal. $\endgroup$ – Maarten Buis Feb 23 '15 at 16:45
  • $\begingroup$ I used the ENTER method in SPSS where all the variables were entered simultaneously. Its an exploratory analysis. I wouldn't need to check collinearity among the dummy IV, right? They would be correlated. $\endgroup$ – user3096214 Feb 23 '15 at 17:43
  • $\begingroup$ @MaartenBuis: True that the assumption is about the distribution of the errors, however with only 2 predictors any non-normality in the dependent or (continuous) independent is very likely to result in non-normally distributed errors. $\endgroup$ – jur Feb 23 '15 at 21:31
  • 2
    $\begingroup$ Not necessarily. Try this out with a binary explanatory variable that has a strong effect: The marginal distribution of the dependent variable will be bimodal. $\endgroup$ – Maarten Buis Feb 24 '15 at 13:02
0
$\begingroup$

Collinearity can certainly be an issue with categorical variables.

Here is an example in R:

set.seed(123)
x1 <- c(rnorm(100,0,1), rnorm(100, 20, 2), rnorm(100, 5, 1))
x2 <- c(rep("A", 100), rep("B", 100), rep("C", 100))
Y <- 5*x1 + 5*as.numeric(factor(x2)) + rnorm(300,0, 10)
m1 <- lm(Y~x1+factor(x2))
summary(m1)

If you run this several times with different seeds you will get quite different parameter estimates - even reversing sign and significance.

| cite | improve this answer | |
$\endgroup$
  • 2
    $\begingroup$ Just a comment on coding: remove the second factor(). And I didn't see a co-linearity or the need for running more than once. Co-linearity would be from adding x3 <- x2 == 'C'. $\endgroup$ – Frank Harrell Mar 13 '18 at 12:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.