Let's say I want to predict the value of Y using the value on my predictor X. X is correlated with Y with some strength $r$ (let's say 0.5). In order to correct for regression towards the mean I should take X's deviation form the mean (of the X distribution) divided by the standard deviation of that distribution. $$y=\frac{\bar{x}-x}\sigma*r_{xy}$$If X is 1 SD above the mean then I should predict Y to be: $1*0.5 = 0.5$, right? Now let's say I have another predictor, Z. It's in exactly the same situation as X, the same deviation and the same correlation, it yields a predicted Y-value of 0.5. How do I combine these two predictors? Taking the mean of the predicted values seems wrong, I mean the second predictor adds more information. If I had a hundred predictors all correlated with Y moderately and all 1 sd above the mean then I would assume that Y is 1 sd above the mean. Also, do I need to know to what degree my predictors are correlated? If so is there a way to work around that?
4
-
$\begingroup$ I was taken aback at "correct for regression to the mean." Could you explain how you expect that to improve the prediction? $\endgroup$– whuber ♦Commented Jun 10, 2016 at 22:47
-
$\begingroup$ Let's say we have SAT scores as X and an unknown "true" IQ score as Y. I know these things are correlated, so I can use SAT as a predictor of an individual IQ score. If I have a SAT score one standard deviation above the mean I should not translate this to an IQ score one standard deviation above the mean. I have to weigh this by the strength of the correlation between these meassures (using the word "correct" is probably super weird, sorry). SAT-deviation*correlation strength is the formula for that, I've heard. $\endgroup$– Vilgot HuhnCommented Jun 10, 2016 at 22:52
-
$\begingroup$ For most purposes, what you have heard is incorrect. For instance, this "correction" increases the variance of your predictions--in other words, it makes them worse on average. $\endgroup$– whuber ♦Commented Jun 11, 2016 at 13:41
-
$\begingroup$ @whuber I'm confused. If I make several predictions while doing this regressing-method on every data-point; shouldn't that decrease the variances of the predictions rather than increase them? Also, if it's incorrect, what's the alternative? Not adjusting for the correlation between X and Y seems weird (for example, what if the correlation is very weak). $\endgroup$– Vilgot HuhnCommented Jun 11, 2016 at 21:50
Add a comment
|