I understand that correlation (I'm referring to Pearson product-moment correlation coefficient) and regression share some similarities from this post: What's the difference between correlation and simple linear regression?

However, I'm wondering the limitations of multiple regression.

Scenario: I would like to explore the associations between a number of continuous variables (e.g., 10 variables). Of course, I would first compute a correlation matrix to observe their inter-correlations.

Next, I would like to explore the effect of the variables on the one or two DVs I’m interested in. I am wondering whether I should run many multiple regression models. I understand that even if two variables (X1,X2) have a significant correlation with another variable (Y), entering X1 into the regression model first might make X2 insignificant. It means that X2 being insignificant in the model doesn't mean X2 is useless. Therefore, my question is: in general, why do we make a regression model even if our correlation matrix could address some of my questions?

  • 1
    $\begingroup$ If you are interested in multiple regressions of every variable on the other, you may want to model partial correlations instead, using a graphical model (e.g. graphical LASSO or ridge). This has the interpretation of regressing every variable on every other and results in a conditional independence network. You may also want to be careful with the use of the word 'significant' in this context, because of the combinatorial explosion when interested in the effect of everything on everything. I doubt any significance remains after multiple testing correction. $\endgroup$ – Frans Rodenburg Nov 1 '17 at 9:26
  • $\begingroup$ It seems you are trying to estimate a (causal) Bayesian network (you want to know how each variable affects each other as a system, you do not want only one "outcome regression"), is that it? $\endgroup$ – Carlos Cinelli Nov 20 '17 at 22:14
  • $\begingroup$ @CarlosCinelli Actually there are two DVs that I’m interested in, and my goal is to see how other variables would influence each of them. $\endgroup$ – JetLag Nov 21 '17 at 0:19
  • $\begingroup$ Could you draw a graph of the relationships you're thinking? For example, some models that might explain your description in your question is something like $x_2 \rightarrow x_1 \rightarrow y$ or $x_2 \leftarrow x_1 \rightarrow y$. $\endgroup$ – Carlos Cinelli Nov 21 '17 at 0:26
  • $\begingroup$ @CarlosCinelli Technically, the ultimate goal of my analysis is to see if Z1, Z2, Z3, Z4, ..., Z10 would moderated the relationship between X and Y. $\endgroup$ – JetLag Nov 21 '17 at 0:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.