# Which is the strongest prediction in linear regression?

Assume a linear regression with metric predictors: y ~ x1 + x2 + x3

Assume all x are significant predictors.

Now I want to find out if predictors differ from each other, that is, if one predictor is a stronger predictor of y than the others, and on top of it, if it is significantly stronger.

Results:

X | Unstd. B | Std. Beta | t value | p value | CI lower | CI upper

1--- .140----- .170 ------ 9.806 --- .000 --- .112 --- .168 ------

2--- .022----- .035 ------ 2.252 --- .024 --- .003 --- .041 ------

3--- .256----- .152 ------ 9.898 --- .000 --- .210 --- .302 ------

Row 1 = x1, row 2 = x2, row 3 = x3. From the p values x2 is the weakest predictor, and also has the lowest std beta weight. But is it significantly weaker than the other predictors? Which of the other two is stronger?

EDIT:

So that we can understand this question, please tell us what it means for predictors to be "strong" or "weak."

I am predicting impairment/disability by different symptoms of a disorder, and want to find out whether one symptom is associated with more impairment than the other symptoms, that is, whether it explains more variance of impairment than the others, although all symptoms are significant predictors. It's about the degree of prediction, not about significance.

-
So that we can understand this question, please tell us what it means for predictors to be "strong" or "weak." – whuber Nov 21 '12 at 20:29
I edited my main post, I hope that makes it clear. Thank you! – Torvon Nov 22 '12 at 0:24
Of possible interest: Explanatory power of a variable. – chl Nov 24 '12 at 8:54
The concept of variable importance may be helpful. As may be the caret package: cran.r-project.org/web/packages/caret/vignettes/caretVarImp.pdf Conversely, I would not put too much stock in statistical significance, which measures something different than importance (however that concept may be defined). – Stephan Kolassa Nov 24 '12 at 19:42

I will decompose your question into two parts.

Which is the strongest predictor?

I assume the following form

$Y=\beta_1 X_1+\beta_2 X_{2}+\beta_3 X_{3}$

which does not include a bias. Now after fitting the $\beta$'s we want to find out the strength of each predictor. We will define this as

$S_1=\beta_1\sum_{i=1}^N x_{1,i}$

for the strength of $X_1$ where the lowercase symbols are the realized values present in your dataset. $N$ is the number of data points in your dataset. The same holds true for $S_2$ and $S_3$. The relative strength of predictor $X_1$ can then be defined as $\frac{S_1}{\sum_{j=1}^3 S_j}$.

Is one predictor significantly stronger/weaker than the others?

I don't believe there is an easy answer to this questions since it's hard to decide what significantly stronger/weaker means in this context. My advice is to seperate significance and strength and discuss them seperately.

I hope this helps you.

-
Thank you. Any chance you could explain this using the concrete parameter estimates I obtain using SPSS or R? I'm not very good with abstract formulae. I understand the differentiation between strong and significant. My problem is that it does not help to report in a paper that one predictor has a strength value of (I'm making these up) 3.8 and another of 3.5 when is it absolutely unclear to a reviewer or reader what these values imply, and if they are meaningfully different from each other. I hope this clarifies my problem. – Torvon Nov 22 '12 at 0:33
So if all the x variables have been centered (a common suggestion) then your strength measures are all 0 and the relative strengths are $\frac00$. This strength measure would give much higher strength (assuming positive slope) to temperature measured in Kelvin than to temperature measured in Celsius even though they give the exact same information. – Greg Snow Nov 22 '12 at 13:30
If you center the data then the beta value in itself can be used as the strength. Regarding a variable measured in kelvin or celsius the statement is incorrect since the beta values will change in the regression depending on which scale you use. The whole point of summing up the data is to see which variable drives the majority of the total sum of the response variable. This heuristic measure is of course far from perfect. – Dr. Mike Nov 23 '12 at 12:19
Why do the relative values of the $S_i$ deserve to be considered measures of "relative strength"? – whuber Nov 23 '12 at 15:49
@Torvon This is mainly used in sales modeling where the sales is measured in revenue or sold units etc. You don't have to center the variables to use this measurement. Your beta values will adapt to fit the data you use. The basic idea is that you decompose the total sum of the response variable in it's pieces by multiplying the corresponding beta value with the total sum of the corresponding variable. By definition adding all of the variables sums multiplied by their corresponding betas have to give the same as the sum of the response variable. But as whuber has pointed out, it has flaws. – Dr. Mike Nov 25 '12 at 20:50

That is what the std betas are for. They represent the expected number of stdev the dependent variable should move for a 1 stdev change in the independent variable.

It's reasonably close to an apples-to-apples comparison, so the variable with the highest beta should have the strongest effect.

-