I have the following python dataframe (5 rows out of nearly 15,000):

   N0_YLDF        MAT       MAP
0  6.286333  11.669069  548.8765
1  6.317000  11.669069  548.8765
2  6.324889  11.516454  531.5035
3  6.320667  11.516454  531.5035
4  6.325556  11.516454  531.5035

I run OLS using following formula: N0_YLDF ~ MAT + MAP

Here, MAP stands for mean annual precipitation. The coefficient of y-intercept for MAP is 0.0079. However, when I divide MAP by 100, the coefficient increases to 0.79

Does this mean that I should normalize my independent variables?

  • 3
    $\begingroup$ The coefficient must change when you scale, or the fitted values wouldn't be the same. In what way would normalizing help? $\endgroup$ – Glen_b -Reinstate Monica May 11 '14 at 3:30
  • $\begingroup$ Thanks @Glen_b, normalizing helps because otherwise the coefficient for MAT is much bigger than that of MAP (intuitively for this problem they should be similar). I have the opposite question, why is normalizing not done always? $\endgroup$ – user308827 May 11 '14 at 15:11
  • $\begingroup$ You normalize to make the results fit your intuition? That's fine, but why would people "do that always"? If I want to know the effect of a unit change in a predictor (and extra km/h of speed, say) on the response variable (running cost, say), I generally want that in the actual units, not normalized units. If my result's in some kind of normalized units, I'd only have to unscale all the results again to say something people would readily understand. $\endgroup$ – Glen_b -Reinstate Monica May 11 '14 at 20:59

@Alecos gave a very thorough mathematical answer.

More intuitively, suppose you regress weight on height. You measure weight in pounds and height in inches. Then you are told you should have measured weight in kilos and height in centimeters. Many of the numbers in the regression results will change, but the meaning will stay exactly the same.

So, in answer to your question: No. The fact you stated doesn't mean anything; it is an automatic consequence of what you did. Whether you should standardize your variables is a good question, but the changes you noted give no help in answering it.

  • $\begingroup$ Thanks Peter, the intuitive reasoning does make sense. I should have phrased my question as, why is standardization of variables not done always? In my case, the magnitude of the parameters makes much more sense once the variables are standardized. $\endgroup$ – user308827 May 11 '14 at 15:09

The "Partitioned Regression" result (Frisch–Waugh–Lovell theorem) states that in the linear regression model (matrix notation)

$$\mathbf y = \mathbf X\beta + \mathbf u$$

if we partition the regression matrix in two, $$X=\left[ \begin{matrix} \mathbf X_1 & \mathbf X_2\\ \end{matrix} \right]$$

Then the coefficients peratining to say $\mathbf X_2$ can be calculated by

$$\hat \beta_2 = \left((\mathbf M_1\mathbf X_2)'\mathbf M_1\mathbf X_2\right)^{-1}\left(\mathbf M_1\mathbf X_2\right)'(\mathbf M_1\mathbf y)$$

where $$\mathbf M_1 = \mathbf I - \mathbf X_1(\mathbf X_1'\mathbf X_1)^{-1}\mathbf X_1'$$ is the "annihilator" or "residual maker" matrix and it is symmetric and idempotent $$\mathbf M_1'=\mathbf M_1,\;\; \mathbf M_1\mathbf M_1=\mathbf M_1$$ So the formula for $\hat \beta_2$ simplifies to

$$\hat \beta_2 = \left(\mathbf X_2'\mathbf M_1\mathbf X_2\right)^{-1}\left(\mathbf X_2'\mathbf M_1\mathbf y\right)$$

Now multiply your $X_2$ regressors by a constant $\alpha$ to obtain

$$\hat \beta_2^* = \left(\alpha\mathbf X_2'\mathbf M_1\alpha\mathbf X_2\right)^{-1}\left(\alpha\mathbf X_2'\mathbf M_1\mathbf y\right) = \frac {\alpha}{\alpha^2}\left(\mathbf X_2'\mathbf M_1\mathbf X_2\right)^{-1}\left(\mathbf X_2'\mathbf M_1\mathbf y\right) = \frac 1{\alpha}\hat \beta_2$$

You multiplied by $\alpha =1/100$ so the coefficient is multiplied by $100$, which is exactly what you found.

  • $\begingroup$ Thanks Alecos, it was very useful following your derivation... $\endgroup$ – user308827 May 11 '14 at 15:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.