0
$\begingroup$

I have a multivariable regression model that includes 4 independent variables. I would like to determine relative significance between the variables.

The 't stat' is given by the software. Can I simply use these values to compare relative significance?

Alternatively, I read that standardized regression coefficients should be found to determine relative significance (https://www.real-statistics.com/multiple-regression/standardized-regression-coefficients/)

The results are nearly but not exactly the same. Which approach is more appropriate?

$\endgroup$
1
  • $\begingroup$ Please explain what you might mean by "relative significance:" what do you hope such a quantity will help you do? $\endgroup$
    – whuber
    Nov 11 '21 at 19:50
1
$\begingroup$

Yes, you can use the $t$-statistic to compare relative significance. The $t$-statistic is scale-independent, so the absolute value of large vs. small values, i.e., $|t_j|$, across coefficients will reflect relative significance. For regression, the t-statistic for the $j$th predictor variable is

\begin{equation} t_j=\frac{\beta_j}{\mathrm{s.e.}(\beta_j)}, \end{equation}

where $\beta_j$ is the regression coefficient, and $\mathrm{s.e.}(\beta_j)$ is the standard error of $\beta_j$.

The t-statistics won't change if you run regression on standardized variables, but the regression coefficients will. So what you are asking, about standardized coefficients, will only result in a change in the coefficients, $\beta_j$, but not $t_j$ statistics.

For the sake of efficiency, I found this good example write-up on your question:

https://www.statology.org/standardized-vs-unstandardized-regression-coefficients/#:~:text=Standardized%20regression%20coefficients%20are%20useful%20when%20you%20want,has%20the%20greatest%20effect%20on%20the%20response%20variable.

$\endgroup$
1
  • 1
    $\begingroup$ Unfortunately, the meaning of "relative significance" is obscure in this context, because unless the variables are mutually orthogonal, all parameter estimates and their t-values depend on the context and have little independent meaning. For instance, the model might be extremely sensitive to the values of two mutually correlated explanatory variables that almost completely account for the response, yet the t-statistics for those two variables could be nearly zero due to that correlation. This shows how dramatically wrong a comparison of t-statistics can be. $\endgroup$
    – whuber
    Nov 11 '21 at 19:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.