0
$\begingroup$

Is there a table of range for beta coefficients used in multiple linear regression that we can use to interpret if influence is strong, very strong, weak or very weak. Thanks!

$\endgroup$
1
$\begingroup$

The estimated $\hat\beta$ will range differently depending on your data (depending on both $X$ and $Y$)! So no, there is no table...

But if you do the regression on standardized variables (zero mean, unit variance), then your coefficients will also be standardized: each $\beta$ will belongs to $[-1,1]$ and will reflect their influence on your dependent variables a bit like a correlation coefficient does. Namely you get the direction of influence in the sign of the standardized $\beta$ and their strength in their magnitude in a standardize fashion (namely a magnitude from 0 to 1).

Moreover you can evaluate how significant each of these coefficients are, by computing their statistics, as shown below.


EDIT: (Below, my old answer that was not exactly about the influence of the coefficients but more on their statistical significance)

However if you used classic least-square to solve the linear regression, under classic assumptions (resiudals normally distributed), you can find statistics for the $\hat\beta$, then p-values, and confidence interval come along.

Namely, for $n$ data points, in the case of a simple linear regression (univariate, scalar dependent variable) the standard errors are: $$ s^2 = \frac{\sum_{i=1}^{n} (y_i - \hat y_i)^2}{n-2} $$ $$ SE(\hat\beta_0) = s\sqrt{\frac{\sum_{i=1}^{n} x_i^2}{n\sum_{i=1}^{n} x_i^2-(\sum_{i=1}^{n} x_i)^2}} $$ $$ SE(\hat\beta_1) = s\sqrt{\frac{n}{n\sum_{i=1}^{n} x_i^2-(\sum_{i=1}^{n} x_i)^2}} $$ where $s^2$ is the sample variance of $y$. Now each standard error will help you compute the t-statistics and get p-values for testing the hypothesis of having a $\beta \neq 0$ (the normalized t-statistics for e.g. $\hat\beta_1$: $t_{\hat\beta_1} = \frac{\hat\beta_1}{SE(\hat\beta_1)}$ follows a t-distribution with $n-2$ degrees of freedom).

For multiple regression (so dealing with data matrix $X$ and $Y$), you can read the standard errors of each $\beta$ by taking the square root of the diagonal elements of the variance covariance matrix of the joint distribution of all $\hat\beta$s: $$ s^2(X^TX)^{-1} $$

See this book chapter for better and detailed explanations.

$\endgroup$
  • 1
    $\begingroup$ In many fields, the "beta" coefficients refer to the coefficients in a regression where all variables have been standardized. Therefore they do not depend on the data in the way you seem to suppose. Moreover, strength of "influence" seems to be a completely different concept than "significance": it won't be measured with a $t$ statistic or its p-value. $\endgroup$ – whuber May 8 '17 at 17:10
  • $\begingroup$ True, but they still depend on the covariance structure between $X$ and $Y$, then in some fields it is also standard to whiten the data anyway you might say, but even though the estimated "beta" are still function of the data. Well, that was my only point to discard the idea of tables... Then indeed, my bad I thought about significance only, forgot the initial matter. Will edit accordingly. $\endgroup$ – H. Rev. May 8 '17 at 17:52
  • 1
    $\begingroup$ This discussion addresses whether standardized rates are between -1 and 1. stats.stackexchange.com/questions/120201/… $\endgroup$ – David Lane May 8 '17 at 18:15
1
$\begingroup$

Not directly, it would seem. However, it is these very moments where we would do well to consult with our dear friend Keith.

He advises:

enter image description here

However, if you're like me, you'll still want something reminiscent of Cohen's d, along with a rule of thumb for determining small, medium and large effect sizes. Later he adds:

enter image description here

Keith is such an accommodating fellow that he includes the formulae for calculating Cohen's f2 from R2 and change in R2:

enter image description here

enter image description here

Alternatively, you could try this effect size calculator.

Despite these recommended effect sizes, however, it's important to note that effect sizes are very much dependent upon the discipline in question.

References

1: Keith, T. Z. (2014). Multiple regression and beyond: An introduction to multiple regression and structural equation modeling. Routledge.

$\endgroup$
-1
$\begingroup$

For simple regression $\beta$ is like R. Thus I would use R rules of thumb... I use the follwoing with my Psychology students:

  • $\beta$ < 0.1 - Small effect size
  • $\beta \in $ [0.1; 0.5[ - Medium effect size
  • $\beta \geq $ 0.5 - Large effect size

For multiple regression these rules are not that straightfoward, but for Social Sciences they seem to hold (also following Cohen's d suggestions).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.