2
$\begingroup$

I've noticed that when standardizing or centering the regressor(s), the p value for the intercept becomes smaller. Does this always happen? If so, why?

I know that $$ var(\hat{\beta}) = \sigma^2(X^TX)^{-1} $$

and that the t-stat is

$$ t_i = \frac{\hat{\beta}_i}{SE(\hat{\beta}_i)} $$

I can't tell from this general equation, nor the ones for simple linear regression found in https://en.wikipedia.org/wiki/Simple_linear_regression why the $t_i$ will increase and thus result in a reduction in the p-value.

$\endgroup$
1

1 Answer 1

1
$\begingroup$

When you center the predictors, the only coefficient that changes is the intercept. See this answer.

Particularly of notice, is the fact that

\begin{matrix} \hat{\beta_L} = \left[\matrix{n^{-1}\mathbf {1}_n^T \\ (X^{*T}CX^*)^{-1}X^{*T}C}\right]y & \hat{\beta_X} = \left[\matrix{ n^{-1}\mathbf {1}_n^T+n^{-1} (\mathbf {1}_n^TX^*(X^{*T}CX^*)^{-1}X^{*T})C \\ (X^{*T}CX^*)^{-1}X^{*T}C }\right]y \end{matrix}

It's not particularly obvious that the p-value will diminish, it can actually increase. But, for non-trivial transformation of the independent variables, it's almost guaranteed to change.

$\endgroup$
2
  • $\begingroup$ I'll check out your derivation in the linked post in a little bit, but I just want to follow up on "it can actually increase." When can it increase? $\endgroup$
    – 24n8
    May 4, 2021 at 14:36
  • 1
    $\begingroup$ Suppose that the estimated intercept for the centered model is exactly zero (there exist values of the means and variances of x and y for which this will be true). Then the centered p-value is 1, regardless of the standard deviation ... $\endgroup$
    – Ben Bolker
    May 4, 2021 at 15:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.