I want to create a regression model to predict state crime rate. There are two variables among 10 ( Vi= # of violent crimes per 100,000 population, Vi2 = # of violent crimes per 10,000 population) that are highly correlated, because Vi is 10 times Vi2, so I decided to use partial correlation. After controlling for Vi2, the correlation of Vi with the dependent variable is reduced to less than 0.4. Should O control for Vi2 in this way, or remove it altogether? Should I remove Vi?

Also, in general may I remove variables if their partial r is less than 0.4 (indicating a weak relationship with the dependent variable) or do need to use an automated variable selection method to get a final model?

  • 1
    $\begingroup$ Please see similar thread at stats.stackexchange.com/questions/11327/… and also the many questions tagged model-selection and feature-selection. $\endgroup$ – rolando2 Feb 10 '13 at 11:21
  • $\begingroup$ I made quite a few edits; please check to see that they are consistent with your intent. $\endgroup$ – rolando2 Feb 10 '13 at 11:40
  • 3
    $\begingroup$ If Vi2 is 10 times Vi then the correlation between them will be 1 and your model will not work. So, either you entered them incorrectly into your model or you described them incorrectly here. $\endgroup$ – Peter Flom - Reinstate Monica Feb 10 '13 at 12:00

(Edit: @Peter Flom is right that Vi and Vi2 cannot be assessed in a single model if one is simply a product of the other. But more generally,...) If Vi and Vi2 are both indicators of the incidence of violent crime, to check the relationship of Vi with the dependent variable while controlling for Vi2 will give an erroneous estimate of the former. You will in effect be partialing a portion of the first relationship right out of itself. Statistical control cannot be applied indiscriminately; one needs to assess a relationship X.Y while controlling for some distinct other relationship Z.Y. (Many of us have learned a lot from our own mistakes in this vein; textbooks and courses tend to emphasize the how of statistical control rather than the why or the when.)

Further, to exclude a variable from your model because its correlation with Y (or its partial correlation) falls below 0.4 seems to be a rule of thumb that will not serve you well in the long run. In many cases one wants to study relationships even when they are weaker than that; in selected cases one's cutoff point may need to be stronger. Criteria for inclusion should be dictated by subject-matter knowledge and perhaps also by statistical power. A few people will even recommend that no variable, once tested, should be removed, though that is a controversial position.

Automated variable selection methods are almost sure to cause harm to your research, unless they are applied in a very savvy way, by a researcher fully aware of their drawbacks.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.