6
$\begingroup$

I am not a statistician, but a medical researcher and I have 5 outcomes that I want to identify independent predictor(s) for each using multiple regression. I have many potential variables that could be included in the multiple regression as independent variables (IV).

One colleague advises to run Spearman correlation matrix between all IV and DV, then to include only the significantly correlated IV in the multiple regression.

Questions

  • Is it appropriate to include only significant predictors with significant bivariate spearman correlations with the outcome?
  • Alternatively, what is a good way to determine inclusion of predictors in a multiple regression?
$\endgroup$
  • $\begingroup$ This approach has been called "double dipping". See my answer below. $\endgroup$ – Frank Harrell Jul 16 '13 at 22:25
8
$\begingroup$

The model should be formulated by subject matter expertise. It is not a good idea to use the data to tell you which data to use. The data are not information-rich enough to be able to reliably do this. Should you have too many events per variable (one rule of thumb is to have at least 15 subjects per parameter in the model), strongly consider data reduction methods that are blinded to $Y$. These include principal components, variable clustering, and redundancy analysis. Examples are in my course notes at http://biostat.mc.vanderbilt.edu/CourseBios330.

$\endgroup$
  • 1
    $\begingroup$ For many of the data sets I work with, subject-matter expertise is not well-developed either. There are many studies and guesses, but no consensus it seems. So although the advice is sound to me, I find it hard to use in many situations I encounter. $\endgroup$ – julieth Jul 17 '13 at 3:01
  • 1
    $\begingroup$ Then be clear that the project is worth doing. Modeling for the same of modeling is sometimes not a fruitful endeavor. Most interesting analyses are driven by interesting questions. Barring that, you can still use data reduction and penalization (lasso, elastic net) methods to find predictive signals. $\endgroup$ – Frank Harrell Jul 17 '13 at 11:44
1
$\begingroup$

There are lots of methods that can be used for variable selection. LASSO is one of the better data driven variable selection models. Do not, whatever you do, use forward stepwise. You'll be glad you didn't:

http://www.nesug.org/proceedings/nesug07/sa/sa07.pdf

$\endgroup$
0
$\begingroup$

It is probably important to not let the analysis drive the theory. Which variables are the best predictors should be based on previous research, or as a minimum, on a consensus of the opinions of subject matter experts. Some of the decision will rest on how large is your sample size. If the size is sufficiently large, you could take a subgroup and check for associations between the independent variables and the dependent variables. When you run multiple regression, you do risk an error with each step of the analysis, so it is important to not just throw everything you have into the regression. If you are able to work with a subgroup, you can then verify what you think you have found with a different group for confirmation. Could you tell us a little more about your sample?

$\endgroup$
-1
$\begingroup$

In conducting a regression analysis, it is useful to examine correlations between the independent variables to avoid the problem of multicolinearity. If you have multiple IVs that are highly correlated, this can indicate that different IVs are accounting for the same portion of variance in the dependent variable or outcome, which can bias the estimated correlation coefficients. One indication of this problem is that you can have a very high R^2 value with very few significant IVs. In other words, having highly correlated IVs in the regression model can mask their actual relationship with the DV. There are several remedies for the problem of multicolinearity, such as excluding one (or more) of the correlated IVs, combining the IVs (additive approach). It is useful to obtain and check the variance inflation factor (VIF) value for each predictor, as high VIF values can indicate variables contributing to multicollinearity.

In constructing regression models, it is acceptable to exclude non-significant IVs from your model, after running a regression model with all of the relevant IVs included, but the decision about whether to exclude variables from analysis is generally not based on the correlation matrix.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.