# Looking for a OLS-Equation if one Regressor is correlated with the error

How can I express a OLS-Estimator if I know about the correlation i.e. I know that $E(x_i u_i)=\rho$ (I'm not looking for IV or 2SLS).

I'll explain my problem with an example:

In a simple problem $\ \ \ y_i = \beta_0 + \beta_1 x_i + u_i \ \ \$ i assume

$E(u_i)=0$

$E(x_i u_i) = \rho$ $\ \ \ i=1,...,n$

With the MM, I can derive the following equations:

$\frac{1}{n}\sum_{i=1}^n(y_i - \hat{\beta}_0 - \hat{\beta}_1 x_i) = 0 \ \ \ \ \ \ \leftrightarrow \ \ \hat{\beta}_0 = \bar{y} - \hat{\beta}_1 \bar{x}$

$\frac{1}{n}\sum_{i=1}^n x_i(y_i - \hat{\beta}_0 - \hat{\beta}_1 x_i) = \rho \ \ \ \leftrightarrow \ \ \hat{\beta}_1 = \frac{\sum_{i=1}^n x_i(y_i - \bar{y})}{\sum_{i=1}^n x_i(x_i - \bar{x})} - \frac{\rho}{\sum_{i=1}^n x_i(x_i - \bar{x})}$

In terms of the ols-estimator and some factor, I can now write the following:

$\hat{\beta}_1 = {\hat{\beta}_1}_{ols} - \frac{\rho}{m_{xx}}$

$\hat{\beta}_0 = {\hat{\beta}_0}_{ols} + \bar{y}\frac{\rho}{m_{xx}}$

We can now see by what particular factor the $\boldsymbol{\beta}_{ols}$ estimator is biased. For this very simple case, I could derive the estimators very easily but for the more common case with k regressors, I can't find a good approach to express the new estimator. For instance, I thought I could use

$\boldsymbol{X^T(y-X\beta) = \rho} \ \$ instead of $\ \ \boldsymbol{X^T(y-X\beta) = 0}$

with $\ \boldsymbol{\rho} \$ a $\ \ k\times 1$-vector containing the information about the correlation between the regressors and the error-term.

But the solution

$\boldsymbol{\hat{\beta} = (X^TX)^{-1}X^Ty - (X^TX)^{-1}\rho}$

does not quite do the job.

• Ask a question. It is not clear what it is that you want to do. You cannot push analytics any further than your last equation (which you would find in most introductory econometrics textbooks when discussing endogeneity and introducing instrumental variables... which is something you may want to look up). Nov 17, 2012 at 12:01
• Hi. Question: How can I express a OLS-Estimator if I know about the correlation i.e. i know that E(xiui)=ρ. Sure a related topic is IV,2SLS but this is not what I want. My example shows that I can account for the correlation between $x_i$ and $u_i$ if i know about the correlation. I want to do the same thing for the k-regressor case. The reason for this is that im using smth. related in my MC-study. There I noticed that if I introduce heteroskedasticity i.e. increasing $S_{xx}$ the smaller the bias of $\boldsymbol{\beta}_{ols}$ gets. I want to conclude smth. similar in the more general case. Nov 17, 2012 at 14:15
• For my MC-Study I use the model: (1) $y_1 = \beta_0 + \beta_1 z_1 + \beta_2 x_2 + \beta_3 z_2 + u_1$ where $E(x_2 u_1) = \rho$ and $z_2=\begin{cases} 0, & \text{for }i = 1,..,\frac{n}{2}\\ 1, & \text{for }i = \frac{n}{2}+1,..,n \end{cases}$ All the other regressors are assumed to be exogenous. If I now introduce heteroskedasticity related to $z_2$ i.e. I estimate (1) m-times each time the variance of the group $y_1|z_2=1$ gets larger the overall bias of $\boldsymbol{\beta}_{ols}$ gets smaller each time. I want to conclude why. Nov 17, 2012 at 14:18

While this is not a situation that arises in practice, this is related to the so-called control function approach to dealing with endogeneity.

Let me rewrite your (simple) model $$Y_i = \beta_0 + \beta_1X_i + U_i$$ together with your assumptions $\mathbb{E}(U_i)=0$ and $\mathbb{E}(U_iX_i)=\rho$.

Then $$\mathbb{E}(X_i(U_i-\frac{\rho}{X_i}))=0$$ so that if I rewrite my model $$Y_i = \beta_0 + \beta_1X_i + \frac{\rho}{X_i} + \underbrace{U_i-\frac{\rho}{X_i}}_{\equiv V_i}$$ and estimate this model by OLS, constraining the coefficient of the $\frac{1}{X_i}$ term to be $\rho$, I should get consistent estimates of $\beta_1$.

So, consider the following Stata simulations

clear*
program simcont, rclass
drop _all
set obs 1000
g x1 = rnormal()
g x2 = rnormal()
g u = x1 + rnormal()
g x = x1 + x2

g y = 2 + 3*x + u  // ols
reg y x
mat mA = e(b)
return scalar ols = el(mA, 1, colnumb(mA, "x"))

g cont = 1/x
constraint define 1 cont = 1  // true correlation between error and regressor
cnsreg y x cont, constraints(1)  // constrained regression
mat mA = e(b)
return scalar cont = el(mA, 1, colnumb(mA, "x"))
end

simulate olsCoeff = r(ols) controlFuncCoeff=r(cont), reps(100): simcont

kdensity olsCoeff, xline(3, lcolor(green)) addplot(kdensity controlFuncCoeff) ///
legend(label(1 "KDE of OLS coeff. estimates") label(2 "KDE of control function coeff. estimates")) ///
xtitle("estimates") ytitle("density") title("Comparison of OLS and control function approaches")


which produces the following picture (based on 100 replications) However, I would think very carefully and experiment with more regressors and in general more data configurations before I put this estimation strategy to work on real data.

## Follow-up questions:

The OP has asked for some clarifications in the comments for which I am providing an updated answer.

Let me rewrite your model $$Y_i = \beta_0 + \beta_1 Z_{1i} + \beta_2 Z_{2i} + \beta_3 X_i + U_i$$ where $Z_{1i}$ and $Z_{2i}$ are exogenous, and $X_{i}$ is endogenous, that is $\mathbb{E}(X_iU_i) = \rho$. In addition, you want the variable $Z_{2i}$ to be constructed as $$Z_{2i} = \mathbf{1}_{[i\text{ is odd.}]}$$ You are simulating the OLS estimates of the coefficient on $X_i$, that is $\beta_3$. Here is a small Stata script to do that.

clear*
program simcont, rclass
syntax [, errorVariance(real 1.0)]
drop _all
set obs 1000
scalar beta0 = 5
scalar beta1 = 1
scalar beta2 = 2
scalar beta3 = 3
scalar rho = 0.1

g z1 = rnormal()
g z2 = mod(_n, 2)
g u = sqrt(errorVariance')*rnormal()
g x = rho*u/errorVariance' + rnormal()

g y = beta0 + beta1*z1 + beta2*z2 + beta3*x + u
reg y z1 z2 x
mat mA = e(b)
return scalar ols = el(mA, 1, colnumb(mA, "x"))

// return the results of the heteroskedasticity test
estat hettest, rhs iid
return scalar hettestPValues = r(p)
end

// simulate with error variance = 1
simulate olsCoeff = r(ols) hettestPValues = r(hettestPValues), reps(100): simcont
su olsCoeff hettestPValues // p-values have the correct mean; no heteroskedasticity

cap mat drop biasBeta
forvalues errorVariance = 2(1)6 {
simulate olsCoeff = r(ols), reps(100): simcont, errorVariance(errorVariance')
qui su olsCoeff
mat biasBeta = (nullmat(biasBeta), r(mean) - 3)
local colNames "colNames' errVar:errorVariance' "
}
mat colnames biasBeta = colNames'
mat list biasBeta


Now note that there are at least two discrepancies here.

• The first thing is that there is no heteroskedasticity in the model as you have written it. The mean of the p-values of an LM test of homoskedasticity from the simulations indicates that they are being drawn under the null.
. su olsCoeff hettestPValues // p-values have the correct mean; no heteroskedasticity

Variable |       Obs        Mean    Std. Dev.       Min        Max
-------------+--------------------------------------------------------
olsCoeff |       100    3.094182    .0326048   3.007202   3.179945
hettestPVa~s |       100    .4762952    .2885692   .0039824   .9844939

• The next thing to note is that the bias is that the asymptotic bias is the same for all values of the error variance, as long as the degree of endogeneity is the same. Here are the results of my simulation
. mat list biasBeta

biasBeta[1,5]
errVar:    errVar:    errVar:    errVar:    errVar:
2          3          4          5          6
r1  .09913641  .10274499   .0912232  .11309942  .09122764


If you are getting results different than this, then you should show us your code, and we can compare the two.

## Update to follow-up

Let me rewrite your model one more time $$Y_i = \beta_0 + \beta_1Z_{1i} + \beta_2 Z_{2i} + \beta_3 X_i + U_{1i}$$ where as before, $Z_{1i}$ and $Z_{2i}$ are exogenous and $X_i$ is endogenous. You assume the data generating process \begin{align} Z_{2i} &= \mathbf{1}_{[i\text{ is odd}]}\\ X_i &= \alpha_0 + \alpha_1Z_{1i} + U_{2i} \end{align} You introduce endogeneity by assuming that $U_{1i}$ and $U_{2i}$ are correlated. $$\mathbb{C}(U_{1i}, U_{2i}) = \rho$$

Also, you assume that the errors in the reduced form equation are heteroskedastic: \begin{align} \mathbb{V}(U_{2i}\mid Z_{2i} = 0) &= 1 \\ \mathbb{V}(U_{2i}\mid Z_{2i} = 1) &= \tfrac{1}{q} \\ \end{align}

Here is the Stata code to simulate this DGP.

clear*
program simcont, rclass
syntax [, Q(real 1.0)]

drop _all
set obs 1000
scalar beta0 = 5
scalar beta1 = 1
scalar beta2 = 2
scalar beta3 = 3

scalar alpha0 = 1
scalar alpha1 = 4

scalar rho = 0.1

g z1 = rnormal()
g z2 = mod(_n, 2)
scalar a11 = 1
scalar a12 = rho*sqrt(1)*sqrt(1)
scalar a13 = rho*sqrt(1)*sqrt(1/q')
scalar a21 = rho*sqrt(1)*sqrt(1)
scalar a22 = 1
scalar a23 = 0
scalar a31 = rho*sqrt(1)*sqrt(1/q')
scalar a32 = 0
scalar a33 = 1/q'

mat corrMatrix= (a11, a12, a13 \ a21, a22, a23 \a31, a32, a33)
drawnorm u1 u3 u4, cov(corrMatrix)
g u2 = cond(z2, u3, u4)

g x = alpha0 + alpha1*z2 + u2
g y = beta0 + beta1*z1 + beta2*z2 + beta3*x + u1
reg y z1 z2 x
mat mA = e(b)
return scalar ols = el(mA, 1, colnumb(mA, "x"))

// return the results of the heteroskedasticity test
qui reg x z2
estat hettest, rhs iid
return scalar hettestPValues = r(p)
end

// simulate to check for heteroskedasticity
simulate olsCoeff = r(ols) hettestPValues = r(hettestPValues), reps(100): simcont, q(5)
su hettestPValues // strong evidence of heteroskedasticity in the reduced form

cap mat drop biasBeta
forvalues q = 2(1)5 {
simulate olsCoeff = r(ols), reps(1000): simcont, q(q')
qui su olsCoeff
mat biasBeta = (nullmat(biasBeta), r(mean) - 3)
local colNames "colNames' q:q' "
}
mat colnames biasBeta = colNames'
mat list biasBeta

• Note that the heteroskedasticity is where you put it in the model, and the simulation results are now able to find it.
. su hettestPValues // strong evidence of heteroskedasticity in the reduced form

Variable |       Obs        Mean    Std. Dev.       Min        Max
-------------+--------------------------------------------------------
hettestPVa~s |       100    1.53e-26    8.12e-26   3.09e-38   6.22e-25

• Also, I can confirm that what you claim is actually true, that as the heteroskedasticity in the reduced form equation increases, the bias in the OLS estimator also increases.
biasBeta[1,4]
q:         q:         q:         q:
2          3          4          5
r1  .11242463  .11808594  .12122701  .12210455
`

A simple explanation of this is that as $q$ increases, the conditional (on $Z_{2i}=1$) and hence the unconditional variance of the reduced form error $U_{2i}$ decreases (check through a variance decomposition) which means that the variance of the endogenous regressor decreases, which means that the $(\mathbf{X}'\mathbf{X})^{-1}$ increases in magnitude, and the overall bias increases (this is a very rough description -- I am sure it can be formalized).

• Sry for the late response. Thx for your detailed answer but it was not quite what I was looking for. I awarded you anyway^^. I was more looking for a analytic way to determine how the impact of correlation biases the ols-estimator because in a simulation i want to calculate this particular bias as exact as possible. For my simple case i could easily caclulate that bias i was looking for. For a more advanced problem not so much. Nov 22, 2012 at 11:01
• @Druss2k Do you want to show that if you use the estimator $\widetilde{\boldsymbol{\beta}} = (\mathbf{X}'\mathbf{X})^{-1}\mathbf{X}'\boldsymbol{Y} - (\mathbf{X}'\mathbf{X})^{-1}\widehat{\boldsymbol{\rho}}$, where $\widehat{\boldsymbol{\rho}} = \mathbf{X}'\boldsymbol{U}$ -- you will get an unbiased estimator? This is not particularly hard to prove. Nov 22, 2012 at 12:48
• Furthermore, the asymptotic bias of the OLS estimator is easily written as $\mathbb{E}(\widehat{\boldsymbol{\beta}} \mid \mathbf{X}) - \boldsymbol{\beta} = (\mathbf{X}'\mathbf{X})^{-1}\boldsymbol{\rho}$, where $\boldsymbol{\rho} = \mathbb{E}(\boldsymbol{X}U)$. Nov 22, 2012 at 12:52
• @Druss2k I will update my answer to include this information. Nov 23, 2012 at 11:43
• @Druss2k I have updated the answer to include the DGP from the commentspace. I think this answers your question in full. Nov 28, 2012 at 19:59