1
$\begingroup$

Is there a table of range for beta coefficients used in multiple linear regression that we can use to interpret if influence is strong, very strong, weak or very weak.

$\endgroup$
1
  • $\begingroup$ Faustus is right: beta is between -1 and 1 usually, but not necessarily (it is always between -1 and 1 in simple regression). $\endgroup$ Commented Nov 29, 2020 at 19:58

4 Answers 4

3
$\begingroup$

Not directly, it would seem. However, it is these very moments where we would do well to consult with our dear friend Keith.

He advises:

enter image description here

However, if you're like me, you'll still want something reminiscent of Cohen's d, along with a rule of thumb for determining small, medium and large effect sizes. Later he adds:

enter image description here

Keith is such an accommodating fellow that he includes the formulae for calculating Cohen's f2 from R2 and change in R2:

enter image description here

enter image description here

Alternatively, you could try this effect size calculator.

Despite these recommended effect sizes, however, it's important to note that effect sizes are very much dependent upon the discipline in question.

References

1: Keith, T. Z. (2014). Multiple regression and beyond: An introduction to multiple regression and structural equation modeling. Routledge.

$\endgroup$
1
$\begingroup$

The estimated $\hat\beta$ will range differently depending on your data (depending on both $X$ and $Y$)! So no, there is no table...

But if you do the regression on standardized variables (zero mean, unit variance), then your coefficients will also be standardized: each $\beta$ will belongs to $[-1,1]$ and will reflect their influence on your dependent variables a bit like a correlation coefficient does. Namely you get the direction of influence in the sign of the standardized $\beta$ and their strength in their magnitude in a standardize fashion (namely a magnitude from 0 to 1).

Moreover you can evaluate how significant each of these coefficients are, by computing their statistics, as shown below.


EDIT: (Below, my old answer that was not exactly about the influence of the coefficients but more on their statistical significance)

However if you used classic least-square to solve the linear regression, under classic assumptions (resiudals normally distributed), you can find statistics for the $\hat\beta$, then p-values, and confidence interval come along.

Namely, for $n$ data points, in the case of a simple linear regression (univariate, scalar dependent variable) the standard errors are: $$ s^2 = \frac{\sum_{i=1}^{n} (y_i - \hat y_i)^2}{n-2} $$ $$ SE(\hat\beta_0) = s\sqrt{\frac{\sum_{i=1}^{n} x_i^2}{n\sum_{i=1}^{n} x_i^2-(\sum_{i=1}^{n} x_i)^2}} $$ $$ SE(\hat\beta_1) = s\sqrt{\frac{n}{n\sum_{i=1}^{n} x_i^2-(\sum_{i=1}^{n} x_i)^2}} $$ where $s^2$ is the sample variance of $y$. Now each standard error will help you compute the t-statistics and get p-values for testing the hypothesis of having a $\beta \neq 0$ (the normalized t-statistics for e.g. $\hat\beta_1$: $t_{\hat\beta_1} = \frac{\hat\beta_1}{SE(\hat\beta_1)}$ follows a t-distribution with $n-2$ degrees of freedom).

For multiple regression (so dealing with data matrix $X$ and $Y$), you can read the standard errors of each $\beta$ by taking the square root of the diagonal elements of the variance covariance matrix of the joint distribution of all $\hat\beta$s: $$ s^2(X^TX)^{-1} $$

See this book chapter for better and detailed explanations.

$\endgroup$
3
  • 1
    $\begingroup$ In many fields, the "beta" coefficients refer to the coefficients in a regression where all variables have been standardized. Therefore they do not depend on the data in the way you seem to suppose. Moreover, strength of "influence" seems to be a completely different concept than "significance": it won't be measured with a $t$ statistic or its p-value. $\endgroup$
    – whuber
    Commented May 8, 2017 at 17:10
  • $\begingroup$ True, but they still depend on the covariance structure between $X$ and $Y$, then in some fields it is also standard to whiten the data anyway you might say, but even though the estimated "beta" are still function of the data. Well, that was my only point to discard the idea of tables... Then indeed, my bad I thought about significance only, forgot the initial matter. Will edit accordingly. $\endgroup$
    – H. Rev.
    Commented May 8, 2017 at 17:52
  • 2
    $\begingroup$ This discussion addresses whether standardized rates are between -1 and 1. stats.stackexchange.com/questions/120201/… $\endgroup$
    – David Lane
    Commented May 8, 2017 at 18:15
0
$\begingroup$

I believe the values posted by JPMD are comparable with Cohen's recommendations for the Pearson correlation coefficient r rather than d, which is a completely different statistic (where Cohen's recommendations for small/medium/large were 0.3/0.5/0.8). However I assume they would also apply to other measures of association constrained to the range -1 to +1 such as R, and maybe also standardised regression coefficients which is what this thread is about.

Even though Cohen was a psychologist, my impression of the conventional interpretation of correlations in psychology (my field) is that 0.1 is trivial, ~0.3 is small, ~0.5 is medium, and >0.6 is large.

$\endgroup$
1
  • $\begingroup$ Cohen's $d$ is not constrained to $[-1,1].$ $\beta$ can be construed as an effect size in its own right, but Cohen's (loose, heuristic, untested) cutpoints of 0.3/0.5/0.8 don't apply directly to it. $\endgroup$
    – whuber
    Commented Oct 23, 2022 at 14:21
-2
$\begingroup$

For simple regression $\beta$ is like R. Thus I would use R rules of thumb... I use the follwoing with my Psychology students:

  • $\beta$ < 0.1 - Small effect size
  • $\beta \in $ [0.1; 0.5[ - Medium effect size
  • $\beta \geq $ 0.5 - Large effect size

For multiple regression these rules are not that straightfoward, but for Social Sciences they seem to hold (also following Cohen's d suggestions).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.