2
$\begingroup$

I am testing the following models. Volumes were purposely log10 transformed to examine brain allometry.

Model_Age_by_Sex <- lm(Grey_Matter_Volume_log ~ TBV_log * Age * sex, data = Data_1)
Model_Age_by_Sex_Scaled <- lm(scale(Grey_Matter_Volume_log) ~ scale(TBV_log) * scale(Age) * sex, data = Data_1)

summary(Model_Age_by_Sex)$coefficients
                               Estimate   Std. Error    t value      Pr(>|t|)
(Intercept)                0.4874303644 0.1828868843  2.6652013  7.700986e-03
TBV_log                    0.8809958273 0.0302474221 29.1263111 2.398416e-182
Age                       -0.0024949875 0.0029438211 -0.8475337  3.967088e-01
sexMale                    0.6302910995 0.2639808852  2.3876392  1.696708e-02
TBV_log:Age                0.0003064347 0.0004871135  0.6290828  5.293027e-01
TBV_log:sexMale           -0.1043227758 0.0434873643 -2.3989216  1.645339e-02
Age:sexMale               -0.0100807480 0.0041958802 -2.4025347  1.629180e-02
TBV_log:Age:sexMale        0.0016541670 0.0006916066  2.3917747  1.677718e-02

summary(Model_Age_by_Sex_Scaled)$coefficients
                            Estimate  Std. Error     t value      Pr(>|t|)
(Intercept)                 0.068316452 0.004044650  16.8905709  1.601447e-63
scale(TBV_log)              0.949059885 0.004016238 236.3056916  0.000000e+00
scale(Age)                 -0.120918626 0.004028183 -30.0181524 2.733358e-193
sexMale                    -0.137960804 0.005930699 -23.2621491 5.528708e-118
scale(TBV_log):scale(Age)   0.002418004 0.003843698   0.6290828  5.293027e-01
scale(TBV_log):sexMale     -0.001284456 0.005791493  -0.2217833  8.244850e-01
scale(Age):sexMale         -0.009004298 0.005775597  -1.5590245  1.190079e-01
scale(TBV_log):scale(Age):sexMale  0.013052642 0.005457304   2.3917747  1.677718e-02

I want to analyze the standardized beta because my variables are on different scales and because I want to be able to say that the age effect for one brain volume was greater than for another, for instance.

When I scale my continuous variables with the scale function in R, the estimates, standard errors and p-values change. This is to be expected considering that I center my variables and am interested in an interaction (e.g. Standardized estimates give different p-value with a glmer/lmer).

However, some effects only become significant after I scale my variables. For instance, my age (p = 2.73e-193) and sex (p = 5.52e-118) main effects are not significant when my DV and IVs are not scaled but becomes very significant when my variables are scaled.

What should I do when the p-value is significant for my standardized output but not my unstandardized output?

$\endgroup$
2
  • $\begingroup$ Welcome to CV, Camille Williams! Your question would be including by providing your exact code and output. You can edit your question with the "edit" link at lower left. $\endgroup$
    – Alexis
    Commented Jan 6, 2020 at 17:13
  • $\begingroup$ Thank you for your comment ! Please let me know if there is anything else I can provide. $\endgroup$ Commented Jan 6, 2020 at 17:25

1 Answer 1

5
$\begingroup$

You need to decide what tests you need to make, because although the models are the same, the tests that the software automatically conducts will differ.

To see why, consider the simplified version of the situation you originally proposed, where there are two regressors $x_1, x_2$ and their interaction $x_1x_2$. Let $\xi_i$ be the corresponding standardized versions of those regressors so that

$$\mu_i + \sigma_i \xi_i = x_i,\quad i\in\{1,2\}.\tag{*}$$

The model is

$$E[y] = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \beta_{12}x_1x_2$$

which can be expressed in terms of the standardized versions by substituting $(*):$

$$\eqalign{ E[y] &= \beta_0 + \beta_1(\mu_1 + \sigma_1 \xi_1) + \beta_2 (\mu_2 + \sigma_2 \xi_2) + \beta_{12}(\mu_1 + \sigma_1 \xi_1)(\mu_2 + \sigma_2 \xi_2) \\ & (\beta_0 + \beta_1 \mu_1 + \beta_2 \mu_2 + \beta_{12}\mu_1\mu_2) \\ & \quad +\, (\beta_1 \sigma_1 + \beta_{12}\mu_2 \sigma_1)\,\xi_1 \\ & \quad +\, (\beta_2 \sigma_2 + \beta_{12}\mu_1 \sigma_2)\,\xi_2 \\ & \quad +\, \beta_{12}\sigma_1\sigma_2\, \xi_1 \xi_2. }$$

Thus, for instance, the default test of $\beta_1$ in the first version of the model compares it to $0$ while in the standardized version the software is comparing $\beta_1\sigma_1 + \beta_{12}\mu_2\sigma_1$ to $0,$ which is equivalent to comparing $\beta_1$ to $-\beta_{12}\mu_2.$ Unless $\mu_2=0,$ this is a different test.

(Note that the tests of the interaction are equivalent, though: both compare $\beta_{12}$ to $0.$ Compare the last lines of your outputs: although the coefficients differ, their p-values are identical.)

Most likely you are interested in testing the original coefficients $\beta_j$ because those will be meaningful.


The question of determining which variables are "significant" is complex, involving considerations of what your prior knowledge indicates, which models you have considered, how many variables you are considering simultaneously, the objectives of your study, and much more. See our threads related to significance and lack thereof; model building; and even regularization (key words for searches might include "Lasso" and "glm").

$\endgroup$
5
  • $\begingroup$ The $\mu_i,$ by definition, are the means of the original variables, not the standardized ones. The analysis of the triple interaction is identical to the analysis of the interaction--it just involves more terms and a little more algebra, but the conclusions are the same. $\endgroup$
    – whuber
    Commented Jan 7, 2020 at 17:08
  • $\begingroup$ ok, thank you! I thought the 𝜇𝑖 was your standardized mean since when you have a two way interaction the p-values are the same in your standardized and non-standardized models (except for the intercept). But once you have more than one interaction, your p-values are no longer the same between the standardized and non-standardized models. $\endgroup$ Commented Jan 7, 2020 at 17:23
  • $\begingroup$ That is true, as you can work out (using the same ideas): but the p-value of the highest-order interaction will remain the same. $\endgroup$
    – whuber
    Commented Jan 7, 2020 at 17:31
  • $\begingroup$ Follow up question: If I used the unstandardized model, how do I calculate the effect sizes of my main effects and interactions if I can't use standardized beta. This is imperative as I (1) would like to quantify the magnitude of the significant effects and (2) compare the effects of my independent variables across dependent variables. e.g. There was an age effect for brain volume 1 which was greater than for brain volume 2. $\endgroup$ Commented Jan 7, 2020 at 19:03
  • 2
    $\begingroup$ @CamilleWilliams if you have significant interaction terms involving age then there is no single "age effect size" to compare among brain volumes. You can only compare effects of age among volumes if you specify the status of all the predictors involved in the age interactions. In your case interpretations with standardized age would be the easiest, as the reference conditions for the intercept and for predictors interacting with age would otherwise be for a newborn with age 0, presumably not your population of interest. $\endgroup$
    – EdM
    Commented Jan 29, 2020 at 17:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.