Skip to main content
Bounty Ended with 100 reputation awarded by Peter Flom
deleted 16 characters in body
Source Link
Sextus Empiricus
  • 86.4k
  • 6
  • 115
  • 301

In this question/answer an image is shown that also presents this idea how the 95% confidence intervals do not tell much about the certainty of the coefficients (at least not intuitively) when the the errors in the estimates of the coefficients are correlated.

image

In this question/answer an image is shown that also presents this idea how the 95% confidence intervals do not tell much about the certainty of the coefficients (at least not intuitively) when the the errors in the estimates of the coefficients are correlated.

image

deleted 16 characters in body
Source Link
Sextus Empiricus
  • 86.4k
  • 6
  • 115
  • 301
                    2.5 %       97.5 % 

(Intercept)  1.602795e+03 1.621126e+03   1602       1621
T-348               7.871775e+0087       8.255498e+0026
(T-348)^2    2       0.897153e-030029 1    0.663665e-020166
                    2.5 %       97.5 % 

(Intercept)       -8.391779e+02839       816.42900118
T                   -3.519318e+0052      6.0510660605
T^2          2        0.897153e-030029    0.016636650166
                    2.5 %       97.5 %
(Intercept)  1.602795e+03 1.621126e+03
T-348        7.871775e+00 8.255498e+00
(T-348)^2    2.897153e-03 1.663665e-02
                    2.5 %       97.5 %
(Intercept) -8.391779e+02 816.42900118
T           -3.519318e+00   6.05106606
T^2          2.897153e-03   0.01663665
                 2.5 %      97.5 % 

(Intercept)      1602       1621
T-348               7.87       8.26
(T-348)^2           0.0029     0.0166
                  2.5 %     97.5 % 

(Intercept)       -839       816
T                   -3.52      6.05
T^2                  0.0029    0.0166
Source Link
Sextus Empiricus
  • 86.4k
  • 6
  • 115
  • 301

It was not so clear to me what sort of standardization was meant, and while looking for the history I picked up two interesting references.

This recent article has a historic overview in the introduction:

García, J., Salmerón, R., García, C., & López Martín, M. D. M. (2016). Standardization of variables and collinearity diagnostic in ridge regression. International Statistical Review, 84(2), 245-266

I found another interesting article that sort of claims to show that standardization, or centering, has no effect at all.

Echambadi, R., & Hess, J. D. (2007). Mean-centering does not alleviate collinearity problems in moderated multiple regression models. Marketing Science, 26(3), 438-445.


To me this criticism all seems a bit like missing the point about the idea of centering.

The only thing that Echambadi and Hess show is that the models are equivalent and that you can express the coefficients of the centered model in terms of the coefficients of the non-centered model, and vice versa (resulting in similar variance/error of the coefficients).

Echambadi and Hess' result is a bit trivial and I believe that this (those relations and equivalence between the coefficients) is not claimed to be untrue by anybody. Nobody claimed that those relations between the coefficients are not true. And it is not the point of centering variables.

The point of the centering is that in models with linear and quadratic terms you can choose different coordinate scales such that you end up working in a frame that has no or less correlation between the variables. Say you wish to express the effect of time $t$ on some variable $Y$ and you wish to do this over some period expressed in terms of years AD say from 1998 to 2018. In that case, what the centering technique means to resolve is that

"If you express the accuracy of the coefficients for the linear and quadratic dependencies on time, then they will have more variance when you use time $t$ ranging from 1998 to 2018 instead of a centered time $t^\prime$ ranging from -10 to 10".

$$Y = a + bt + ct^2$$

versus

$$Y = a^\prime + b^\prime(t-T) + c^\prime(t-T)^2$$

Of course, these two models are equivalent and instead of centering you can get the exact same result (and hence the same error of the estimated coefficients) by computing the coefficients like

$$\begin{array}{} a &=& a^\prime - b^\prime T + c^\prime T^2 \\ b &=& b^\prime - 2 c^\prime T \\ c &=& c^\prime \end{array}$$

also when you do ANOVA or use expressions like $R^2$ then there will be no difference.

However, that is not at all the point of mean-centering. The point of mean-centering is that sometimes one wants to communicate the coefficients and their estimated variance/accuracy or confidence intervals, and for those cases it does matter how the model is expressed.

Example: a physicists wishes to express some experimental relation for some parameter X as a quadratic function of temperature.

  T   X
  298 1230
  308 1308
  318 1371
  328 1470
  338 1534
  348 1601
  358 1695
  368 1780
  378 1863
  388 1940
  398 2047

would it not be better to report the 95% intervals for coefficients like

                    2.5 %       97.5 %
(Intercept)  1.602795e+03 1.621126e+03
T-348        7.871775e+00 8.255498e+00
(T-348)^2    2.897153e-03 1.663665e-02

instead of

                    2.5 %       97.5 %
(Intercept) -8.391779e+02 816.42900118
T           -3.519318e+00   6.05106606
T^2          2.897153e-03   0.01663665

In the latter case the coefficients will be expressed by seemingly large error margins (but telling nothing about the error in the model), and in addition the correlation between the distribution of the error won't be clear (in the first case the error in the coefficients will not be correlated).

If one claims, like Echambadi and Hess, that the two expressions are just equivalent and the centering does not matter, then we should (as a consequence using similar arguments) also claim that expressions for model coefficients (when there is no natural intercept and the choice is arbitrary) in terms of confidence intervals or standard error are never making sense.