How to interpret standardized regression coefficients and p-values in multiple regression?

Question

I've been using R to analyze my data (as shown in example below) and lm.beta from the QuantPsyc package to get the standardized regression coefficients.

My understanding is that the absolute value of the standardized regression coefficients should reflect its importance as a predictor. I was also under the impression (and the intuition) that the variable with the largest absolute value should be the most significant independent predictor and should have the lowest p-value. However, I'm not finding that in my data.

For example (taken from my data), I have a multiple regression with dependent variable y and 7 independent variables x1:x7.

    Call:
lm(formula = y ~ x1 + x2 + x3 + x4 + x5 + x6 + x7)

For 3 of the variables, the beta values and the p-values make sense to me (the greater the magnitude of beta, the lower p-value), but for 4 of them this is not the case. I'll show only the p-values and betas for those 4 to keep this short.

    x1          x2          x3          x7
p   0.006635    0.00004683  0.000152    0.022427
ß   0.15707977  0.24149287  0.27171665  0.16583391 

As you can see, x2 has a lower p-value than x3, but x3 has a larger value for beta. Similarly, x7 has a larger beta value than x1, but is less significant.

I've searched for an explanation but have found conflicting information. Is that because there's no straightforward answer to this question? Am I doing something wrong?

Answer 1

For the standard linear regression model the absolute value of the coefficient estimates and the p-value are not related in the way you describe. It is very possible to have absolutely large coefficients which are insignificant and absolutely small coefficients which are very significant. What your missing in your interpretation is the effect of the coefficient estimate standard errors.

The coefficients R reports (lets call them $b_1,b_2,b_3,...,b_k$) are the best linear unbiased estimators of the true parameters $\beta_1,\beta_2,\beta_3,...,\beta_k$ in that they minimize the sum of squared error or formally: $$ \{b_1,b_2,...,b_k\} = {\textrm{argmin} \atop \alpha}\left\{ \sum_{i=1}^{n}(y_i-\alpha_1x_{i,1}-. . .-\alpha_kx_{i,k})^2]\right\} $$

The p-value for the $i^{th}$ coefficient which R is reporting is the result of the following hypothesis test:

$H_0: \beta_i = 0$

$H_A: \beta_i \neq 0$

Assuming the regression is properly specified, it can be shown, with the central limit theorem, that each $b_i$ is a normally distributed random variable with mean $\beta_i$ and some standard deviation (also called standard error) $\sigma_i$. This is because the $b$'s are estimated with a random sample so they too are random variables (roughly speaking). What determines the $i^{th}$ p-value is where 0 "lands" in the normal distribution $N(\beta_i,\sigma_i^2)$ (technically the test is done using a t-distribution...but the difference is not so important for addressing your question). If zero is in the tails of $N(\beta_i,\sigma_i^2)$ the p-value is low, if it's more in the middle the p-value is high.

So given two estimates $b_i$ and $b_j$ where $b_i$ is "super far away" from zero and $b_j$ is "super close to" zero, the p-value of $b_i$ would be lower than $b_j$ assuming $\sigma_i$=$\sigma_j$. The part you are missing in your interpretation is that $\sigma_i$ and $\sigma_j$ can be very different. Essentially if $b_i$ is "huge" but $\sigma_i$ is also "huge" you see that you can get a high p-value. Conversely for "small" $b_i$ and "super small" $\sigma_i$, you see you can get a small p-value.

I hope that helps!

Answer 2

Standardized regression coefficients do not work for categorical variables or for nonlinear effects. You are assuming everything has a linear effect, which is unlikely. Standardization also assumes that the SD is the right scaling constant.

To me standardized coefficients are harder to interpret than the original coefficients, and the standardization is arbitrary. It can also be misleading. It is not always true that a variable should have more importance just because its standard deviation is different from another's.

Answer 3

Your question seems to reflect the mistaken understanding that the statistical "significance" of the p-value somehow means "meaningful", "important", or "relevant to real life". This is a false but very widely held misunderstanding.

P-values are a standardized representation of how reliable the effect size measures are. I say that the p-value is "standardized" in the sense that no matter what the test statistic, whether a t score, an F, a $\chi^2$, or whatever (in the case of linear regression, it is the t score), the p-value represents it on the same scale of a number from 0 to 1 such that values closer to 0 give greater confidence that the effect size measures are reliable, that is, are not mere haphazard statistical flukes. (Of course, by convention, 0.05 is the most common cutoff measure for an "acceptable" p-value, though that should not be taken to be a magic number. The "right" cutoff for a p-value actually depends on both the sample size and the strength of the expected effect size. So, in some cases, 0.05 might not be strict enough, and in other cases, it might be too stringent.)

Effect size measures are the most important results that actually reflect what is "meaningful", "important", or "relevant to real life". In the case of linear regression, these are the coefficients and standardized coefficients (or the adjusted $R^2$ for the overall regression model).

So, with that understanding, to directly answer your question:

• The p-values reflect how much confidence you should have in the reliability of the standardized coefficients. If you take the 0.05 traditional cutoff, the only thing that the p-values tell you in your results is that all your standardized coefficients are reliable. They absolutely do not tell you how important any specific variable might be, and they should never be misinterpreted in that way. So, it is irrelevant which p-values are larger than others, as long as they are all well below 0.05.
• The standardized coefficients are what you should focus on in trying to determine which variables are more important. In regression, what they mean is that one standard deviation increase in the given variable will give the specified number of standard deviations of change in the target variable. So, in your case, one standard deviation increase in x2 (measured in x2's units) will increase y by 0.24 standardard deviations (measured in y's units); one standard deviation increase in x3 (measured in x3's units) will increase y by 0.27 standardard deviations (measured in y's units). The relative p-values of x2 and x3 are irrelevant; they do not reflect importance; they only indicate that the estimates of x2 and x3 both have high confidence.

That said, considering that 0.24 and 0.27 as $\beta$ values are very close, you should certainly not trust that your results mean that x3 is conclusively more important than x2. The numbers are too close; they should be considered approximately equal; any apparent differences here might indeed be a statistical fluke. If you really want to test which one is more important, then there are specific tests for that.

Answer 4

p-value and $|\beta_i|$ have different meanings.

• p-value of $\beta_i$ relates to "the probability that $\beta_i = 0$". Full explanation is: if $\beta_i = 0$, p-value means the probability that "sample will be regressed to $\beta_i = \hat{\beta}_i$", where, $\hat{\beta}_i$ means the current coefficient of regression.

• $|\beta_i|$ means: the level of effect from $\beta_i$ to y, which value relies on variances.

In your case, I think $x_i$ may have different variances. It cause that several $x_i$ has high coefficient and p-value.

The second possibility is non-Gaussian distribution of variables. In case of unify distribution with little records:

> b = runif(50)
> a = runif(50)
> mo = lm(b ~ a + 1)
> summary(mo)

Call:
lm(formula = b ~ a + 1)

Residuals:
     Min       1Q   Median       3Q      Max 
-0.42894 -0.16301  0.01589  0.17515  0.49369 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  0.41346    0.07508   5.507 1.41e-06 ***
a            0.19266    0.12700   1.517    0.136    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.253 on 48 degrees of freedom
Multiple R-squared:  0.04575,   Adjusted R-squared:  0.02587 
F-statistic: 2.301 on 1 and 48 DF,  p-value: 0.1358