1
$\begingroup$

How to prove that OLS estimators do not depend on other parameters, given the same variables are included in the regression?

I generate some random data for 30 observations and 4 variables $x_1,x_2, x_3, x_4 $, then generate a dependent variable $y$ based on these variables plus an error term:

$ y = \alpha_1 x_1 + \alpha_2 x_2 + \alpha_3 x_3 + \alpha_4 x_4 + u $

I want to formally prove that, given the same data, regardless of the value of $\alpha_1$, the estimated coefficients, i.e. $\hat{\alpha}_2,\hat{\alpha}_3,\hat{\alpha}_3 $ will be the same.

If the text is not clear, here is an example in python.

I know the formula for the estimator is:

$\hat{\beta}_2 = \frac{\sum_{i=1}^{n} x_{i2} (y_i - \bar{y})}{\sum_{i=1}^{n} x_{i2}^2}$

which does not depend on the values of other variables, but I am confused, because from this formula it appears to me that if a don't include other variables in the regression, the estimated coefficient would be the same, which is clearly wrong.

Ps: This is not a homework, but self-study, if there is a problem in posting the solution, please provide me with the intuition on how to start.

Ps2: Apologies for a completely basic question, but I am studying linear regression and trying to understand formally the results.

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Please review your notation. First you say that you generate $x_1,x_2, x_3, x_4 $, but then it is $x_2$ and $\alpha_2$ three times in the equation for $y$. Then it's suddenly also $\hat\alpha_3$ afterwards. This makes it hard to see what your question precisely is. $\endgroup$
    – Christoph Hanck
    Commented Dec 8, 2023 at 14:57
  • 1
    $\begingroup$ What does it mean to be "given the same data" but to vary the value of $\alpha_1$?? $\endgroup$
    – whuber ♦
    Commented Dec 8, 2023 at 16:15

2 Answers 2

Reset to default
3
$\begingroup$

In matrix notation you have $Y=X\beta + U$ and OLS provides $$\widehat{\beta} = \left(X'X\right)^{-1}X'Y = \beta + \left(X'X\right)^{-1}X'U. $$ As long as $X$ and $U$ are the same, $\widehat{\beta}-\beta = \left(X'X\right)^{-1}X'U$ will always be the same. Now if you change the value of $\beta_1$, say, then the estimator for other parameters does not change.

Improve this answer
$\endgroup$
2
$\begingroup$

You formula for $\beta_2$ is wrong. Suppose you would have two independent variables, then you can see in this link that the estimates $\beta_1$ and $\beta_2$ depend on both $x_1$ and $x_2$.

Only when the two variables are uncorrelated in your sample data, the estimate $\beta_1$ (for $x_1$) would not change when adding independent $x_2$.

However, in sample data (simulated or real) there will almost always be some correlation, even when in the population (simulated or real) there is NO correlation.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ Could you please check my python example? Data are correlated but the estimate is always the same.. I think I`m not benig able to explain, it is not about depending on x1 and x2, but on the tru values of alpha1 and alpha2 $\endgroup$
    – Oalvinegro
    Commented Dec 8, 2023 at 14:56
  • 1
    $\begingroup$ Never used python, but just saw that you generate the data outside of the loop. If you do that inside the loop, you should get different estimates each iteration. $\endgroup$
    – BenP
    Commented Dec 8, 2023 at 15:11
  • 1
    $\begingroup$ Also, print all estimates using print(round(model.params,9)) instead of only one. $\endgroup$
    – BenP
    Commented Dec 8, 2023 at 15:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.