I have a Dependent Variable that is to be modeled against 3 Independent Variables. After drawing a scatter plot for each IV with DV, I found out that two of the three IVs show no linear relationship with DV at all. Plus, the correlation analysis shows that the coefficients of these two IVs with the DV are below 0.3 with p-values that are much larger than my significance level(0.05). The question is, should I include these two variables in my analysis and perform a multiple linear regression? Or a simple linear regression between the one IV that shows a linear relationship with the DV is enough?

P.S. The question specifically asks for Linear Regression so other alternatives are not considered.

  • $\begingroup$ What is the goal of your regression equation? $\endgroup$
    – Dave
    Nov 16 '19 at 17:38
  • 2
    $\begingroup$ Your term non-linear variable is non-standard, but the explanation in the text is better. Even if a variable in an xy-scatterplot seems to show no relation, that only indicates no relation in that bivariate distribution. It could still be interacting with some other variable, modify the effect of some other variable, ... so only the no-relation plot you have described is not enough reason to leave the variable out. AND, even if you are restricted to linear regression, that includes interactions, square terms (and other polynomials), ... there is still a lot of possibilities! $\endgroup$ Nov 16 '19 at 22:08

Given the information in your question, the only answers are "maybe, maybe not" or "not necessarily".

If you exclude a variable from a multiple regression just because it has no bivariate linear relationship, then you are using a model building method called bivariate screening. Although this is a popular method, it can't be recommended. For full details, you can consult Regression Modeling Strategies by Frank Harrell (who is also a participant here and may chime in). Here are some problems with it:

  1. Unless all the independent variables are orthogonal, even a variable with a weak relationship to the dependent variable may be a valuable covariate.

  2. The variable may be involved in an interaction with another variable.

  3. A small effect may be interesting -- if theory predicts a large one, then a small effect may be more interesting than a large one.

  4. Any automated method denies you the opportunity to think.

  5. Doing this sort of screening biases all the results of the final model.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.