What to report when p-values of Standardized and Unstandardized Beta disagree?

Question

I am testing the following models. Volumes were purposely log10 transformed to examine brain allometry.

Model_Age_by_Sex <- lm(Grey_Matter_Volume_log ~ TBV_log * Age * sex, data = Data_1)
Model_Age_by_Sex_Scaled <- lm(scale(Grey_Matter_Volume_log) ~ scale(TBV_log) * scale(Age) * sex, data = Data_1)

summary(Model_Age_by_Sex)$coefficients
                               Estimate   Std. Error    t value      Pr(>|t|)
(Intercept)                0.4874303644 0.1828868843  2.6652013  7.700986e-03
TBV_log                    0.8809958273 0.0302474221 29.1263111 2.398416e-182
Age                       -0.0024949875 0.0029438211 -0.8475337  3.967088e-01
sexMale                    0.6302910995 0.2639808852  2.3876392  1.696708e-02
TBV_log:Age                0.0003064347 0.0004871135  0.6290828  5.293027e-01
TBV_log:sexMale           -0.1043227758 0.0434873643 -2.3989216  1.645339e-02
Age:sexMale               -0.0100807480 0.0041958802 -2.4025347  1.629180e-02
TBV_log:Age:sexMale        0.0016541670 0.0006916066  2.3917747  1.677718e-02

summary(Model_Age_by_Sex_Scaled)$coefficients
                            Estimate  Std. Error     t value      Pr(>|t|)
(Intercept)                 0.068316452 0.004044650  16.8905709  1.601447e-63
scale(TBV_log)              0.949059885 0.004016238 236.3056916  0.000000e+00
scale(Age)                 -0.120918626 0.004028183 -30.0181524 2.733358e-193
sexMale                    -0.137960804 0.005930699 -23.2621491 5.528708e-118
scale(TBV_log):scale(Age)   0.002418004 0.003843698   0.6290828  5.293027e-01
scale(TBV_log):sexMale     -0.001284456 0.005791493  -0.2217833  8.244850e-01
scale(Age):sexMale         -0.009004298 0.005775597  -1.5590245  1.190079e-01
scale(TBV_log):scale(Age):sexMale  0.013052642 0.005457304   2.3917747  1.677718e-02

I want to analyze the standardized beta because my variables are on different scales and because I want to be able to say that the age effect for one brain volume was greater than for another, for instance.

When I scale my continuous variables with the scale function in R, the estimates, standard errors and p-values change. This is to be expected considering that I center my variables and am interested in an interaction (e.g. Standardized estimates give different p-value with a glmer/lmer).

However, some effects only become significant after I scale my variables. For instance, my age (p = 2.73e-193) and sex (p = 5.52e-118) main effects are not significant when my DV and IVs are not scaled but becomes very significant when my variables are scaled.

What should I do when the p-value is significant for my standardized output but not my unstandardized output?

Answer 1

You need to decide what tests you need to make, because although the models are the same, the tests that the software automatically conducts will differ.

To see why, consider the simplified version of the situation you originally proposed, where there are two regressors $x_1, x_2$ and their interaction $x_1x_2$. Let $\xi_i$ be the corresponding standardized versions of those regressors so that

$$\mu_i + \sigma_i \xi_i = x_i,\quad i\in\{1,2\}.\tag{*}$$

The model is

$$E[y] = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_{12}x_1x_2$$

which can be expressed in terms of the standardized versions by substituting $(*):$

$$\eqalign{ E[y] &= \beta_0 + \beta_1(\mu_1 + \sigma_1 \xi_1) + \beta_2 (\mu_2 + \sigma_2 \xi_2) + \beta_{12}(\mu_1 + \sigma_1 \xi_1)(\mu_2 + \sigma_2 \xi_2) \\ & (\beta_0 + \beta_1 \mu_1 + \beta_2 \mu_2 + \beta_{12}\mu_1\mu_2) \\ & \quad +\, (\beta_1 \sigma_1 + \beta_{12}\mu_2 \sigma_1)\,\xi_1 \\ & \quad +\, (\beta_2 \sigma_2 + \beta_{12}\mu_1 \sigma_2)\,\xi_2 \\ & \quad +\, \beta_{12}\sigma_1\sigma_2\, \xi_1 \xi_2. }$$

Thus, for instance, the default test of $\beta_1$ in the first version of the model compares it to $0$ while in the standardized version the software is comparing $\beta_1\sigma_1 + \beta_{12}\mu_2\sigma_1$ to $0,$ which is equivalent to comparing $\beta_1$ to $-\beta_{12}\mu_2.$ Unless $\mu_2=0,$ this is a different test.

(Note that the tests of the interaction are equivalent, though: both compare $\beta_{12}$ to $0.$ Compare the last lines of your outputs: although the coefficients differ, their p-values are identical.)

Most likely you are interested in testing the original coefficients $\beta_j$ because those will be meaningful.

---

The question of determining which variables are "significant" is complex, involving considerations of what your prior knowledge indicates, which models you have considered, how many variables you are considering simultaneously, the objectives of your study, and much more. See our threads related to significance and lack thereof; model building; and even regularization (key words for searches might include "Lasso" and "glm").