1
$\begingroup$

I have 1 dependent variable and 33 independent variables (continuous, categorical & dichotomous). Correlation analyses (2-tailed) show that the DV is only correlated to 7 of the IVs although most of the correlations are very weak, e.g. about 0.1 or less than 0.1.

Is it correct to put only IVs that are correlated with the DV into the regression model?

P.S. What's the use of the correlation matrix (1-tailed) produced with the regression analysis?

$\endgroup$
  • $\begingroup$ What kind of $R^2$ are you getting? Dropping predictors will not improve your accuracy, so you may want to re-think your model. $\endgroup$ – user75138 Aug 25 '15 at 4:23
  • $\begingroup$ Thanks for your reply, Bey. Sorry, what does it mean by what kind of R2? $\endgroup$ – statistics newbie Aug 25 '15 at 4:33
  • $\begingroup$ How much explained variance are you getting with your model? If you plot $Y_{pred}$ vs $Y_{actual}$, what is the coefficient of determination? $\endgroup$ – user75138 Aug 25 '15 at 4:35
  • $\begingroup$ I tried with the stepwise method. With the 7 IVs inputted, the best R Square I got is 0.340 and only 3 IVs remained in the model. $\endgroup$ – statistics newbie Aug 25 '15 at 6:42
  • $\begingroup$ Don't forget that regression works based on the combination of your predictor/independent variables. So, think about a) if there are strong correlations between your predictor variables, b) whether a predictor variable not so strongly correlated to your dependent variable (individually) might be important when other variables are present. $\endgroup$ – AntoniosK Aug 25 '15 at 10:47
2
$\begingroup$

Correlating the dependent variable with each of the potential regressors will not generally reveal all the useful information. It might very well be that a linear combination of a subset of regressors will be highly correlated with the dependent variable while each of the regressors in the subset will be only weakly correlated with the dependent variable. In other words, a regression with multiple regressors cannot be substituted with multiple pairwise regressions.
To answer your question, it is generally not wise to put only the IVs that are correlated with the DV into the regression.

For variable inclusion/exclusion strategies, you may check the posts under the feature-selection tag.

$\endgroup$
1
$\begingroup$

Generally, the answer is no, because the presence or absence of a third variable in the model may change the relationships between the independent and dependent variables. There are known phenomena like mediation and confounding, for instance. There are also sampling biases, which may mess up the correlations, hide relationships etc.

Having said this, the bi-variate correlations are informative. You should take them into account. It's just you shouldn't base your variable selection solely on bi-variate correlations.

$\endgroup$
  • $\begingroup$ My goal is to find out whether science background can predict money spent on organic food. Since I think taking Physics and taking Biology are quite different, I don't want to use "taking science" as a whole as one variable. Instead I use three different variables: "taking Biology", "taking Chemistry" and "taking Physics". I also thought about including the variables "taking science" and "no. of science subjects taken". $\endgroup$ – statistics newbie Sep 6 '15 at 11:33
  • $\begingroup$ However, as you can imagine, these variables are usually highly correlated, and I don't know if it's appropriate to put them all in the regression model. Moreover, for example, "taking physics" is only weakly correlated (< .1) with $ spent on organic food. So, is it still appropriate to include it in the model? I got stuck here and I'm really confused. $\endgroup$ – statistics newbie Sep 6 '15 at 11:33
  • $\begingroup$ I'm using the stepwise method. $\endgroup$ – statistics newbie Sep 6 '15 at 11:53
  • $\begingroup$ What's the purpose of the analysis? Multicollinearity is not an issue in forecasting (generally). So, if you're forecasting then you can throw in all variables and not worry too much about correlations. $\endgroup$ – Aksakal Sep 6 '15 at 21:15

Not the answer you're looking for? Browse other questions tagged or ask your own question.