4
$\begingroup$

I have a continuous outcome (dependent) variable, which is body weight and I'm wondering which of my 20 candidate predictors (independent variables) are the most important ones for prediting body weight. Usually I'd just go for a linear regression and try to rationalize my variable selection by judging coefficient sizes, significance and subject matter knowledge. But after colliding with all threads about principal component analysis, I'm wondering whether the linear regression I usually go for is suboptimal for this purpose.

Variable selection is of inferior importance, since I am likely to select variables on subject matter knowledge. I am primarily interested in finding the strongest predictors of the dependent variable and a measure of their impact after multivariable adjustment. This is something I would typically carry out by means of linear regression and present the coefficients along with confidence limits.

What is your opinions on this? How should this analysis be carried out? I'm an R user.

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ Can you clarify whether you're interested in variable selection before fitting a (final) model or in apportionment of predictive power among predictors after fitting a model? Your title suggests the latter but talk of "candidate predictors" & "variable selection" in the body of your question suggests the former. $\endgroup$
    – Scortchi - Reinstate Monica ♦
    Commented May 6, 2015 at 15:21

1 Answer 1

Reset to default
4
$\begingroup$

What you used to be doing sounds fine, and you should not be confused and led astray by what you are reading about principal component analysis.

PCA does not use a dependent variable at all, and therefore can tell you nothing about the importance of your individual predictors.

All these issues have been discussed here before, and I can particularly recommend you to study the following threads:

  1. Using principal component analysis (PCA) for feature selection
  2. Detecting significant predictors out of many independent variables
  3. Importance of predictors in multiple regression: Partial $R^2$ vs. standardized coefficients

The last thread can initially appear quite confusing, but scroll down to my answer and look at its second part for a short summary of various existing measures of "predictor importance" in regression.

Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.