5
$\begingroup$

How can I express a OLS-Estimator if I know about the correlation i.e. I know that $E(x_i u_i)=\rho$ (I'm not looking for IV or 2SLS).

I'll explain my problem with an example:

In a simple problem $\ \ \ y_i = \beta_0 + \beta_1 x_i + u_i \ \ \ $ i assume

$E(u_i)=0$

$E(x_i u_i) = \rho$ $ \ \ \ i=1,...,n$

With the MM, I can derive the following equations:

$\frac{1}{n}\sum_{i=1}^n(y_i - \hat{\beta}_0 - \hat{\beta}_1 x_i) = 0 \ \ \ \ \ \ \leftrightarrow \ \ \hat{\beta}_0 = \bar{y} - \hat{\beta}_1 \bar{x}$

$\frac{1}{n}\sum_{i=1}^n x_i(y_i - \hat{\beta}_0 - \hat{\beta}_1 x_i) = \rho \ \ \ \leftrightarrow \ \ \hat{\beta}_1 = \frac{\sum_{i=1}^n x_i(y_i - \bar{y})}{\sum_{i=1}^n x_i(x_i - \bar{x})} - \frac{\rho}{\sum_{i=1}^n x_i(x_i - \bar{x})}$

In terms of the ols-estimator and some factor, I can now write the following:

$\hat{\beta}_1 = {\hat{\beta}_1}_{ols} - \frac{\rho}{m_{xx}}$

$\hat{\beta}_0 = {\hat{\beta}_0}_{ols} + \bar{y}\frac{\rho}{m_{xx}}$

We can now see by what particular factor the $\boldsymbol{\beta}_{ols}$ estimator is biased. For this very simple case, I could derive the estimators very easily but for the more common case with k regressors, I can't find a good approach to express the new estimator. For instance, I thought I could use

$\boldsymbol{X^T(y-X\beta) = \rho} \ \ $ instead of $ \ \ \boldsymbol{X^T(y-X\beta) = 0}$

with $ \ \boldsymbol{\rho} \ $ a $\ \ k\times 1$-vector containing the information about the correlation between the regressors and the error-term.

But the solution

$\boldsymbol{\hat{\beta} = (X^TX)^{-1}X^Ty - (X^TX)^{-1}\rho}$

does not quite do the job.

$\endgroup$
  • 2
    $\begingroup$ Ask a question. It is not clear what it is that you want to do. You cannot push analytics any further than your last equation (which you would find in most introductory econometrics textbooks when discussing endogeneity and introducing instrumental variables... which is something you may want to look up). $\endgroup$ – StasK Nov 17 '12 at 12:01
  • $\begingroup$ Hi. Question: How can I express a OLS-Estimator if I know about the correlation i.e. i know that E(xiui)=ρ. Sure a related topic is IV,2SLS but this is not what I want. My example shows that I can account for the correlation between $x_i$ and $u_i$ if i know about the correlation. I want to do the same thing for the k-regressor case. The reason for this is that im using smth. related in my MC-study. There I noticed that if I introduce heteroskedasticity i.e. increasing $S_{xx}$ the smaller the bias of $\boldsymbol{\beta}_{ols}$ gets. I want to conclude smth. similar in the more general case. $\endgroup$ – Druss2k Nov 17 '12 at 14:15
  • $\begingroup$ For my MC-Study I use the model: (1) $y_1 = \beta_0 + \beta_1 z_1 + \beta_2 x_2 + \beta_3 z_2 + u_1$ where $E(x_2 u_1) = \rho$ and $z_2=\begin{cases} 0, & \text{for }i = 1,..,\frac{n}{2}\\ 1, & \text{for }i = \frac{n}{2}+1,..,n \end{cases}$ All the other regressors are assumed to be exogenous. If I now introduce heteroskedasticity related to $z_2$ i.e. I estimate (1) m-times each time the variance of the group $y_1|z_2=1$ gets larger the overall bias of $\boldsymbol{\beta}_{ols}$ gets smaller each time. I want to conclude why. $\endgroup$ – Druss2k Nov 17 '12 at 14:18
8
+50
$\begingroup$

While this is not a situation that arises in practice, this is related to the so-called control function approach to dealing with endogeneity.

Let me rewrite your (simple) model $$ Y_i = \beta_0 + \beta_1X_i + U_i $$ together with your assumptions $\mathbb{E}(U_i)=0$ and $\mathbb{E}(U_iX_i)=\rho$.

Then $$ \mathbb{E}(X_i(U_i-\frac{\rho}{X_i}))=0 $$ so that if I rewrite my model $$ Y_i = \beta_0 + \beta_1X_i + \frac{\rho}{X_i} + \underbrace{U_i-\frac{\rho}{X_i}}_{\equiv V_i} $$ and estimate this model by OLS, constraining the coefficient of the $\frac{1}{X_i}$ term to be $\rho$, I should get consistent estimates of $\beta_1$.


So, consider the following Stata simulations

clear*
program simcont, rclass
    drop _all
    set obs 1000
    g x1 = rnormal()
    g x2 = rnormal()
    g u = x1 + rnormal()
    g x = x1 + x2

    g y = 2 + 3*x + u  // ols 
    reg y x
    mat mA = e(b)
    return scalar ols = el(mA, 1, colnumb(mA, "x"))

    g cont = 1/x
    constraint define 1 cont = 1  // true correlation between error and regressor
    cnsreg y x cont, constraints(1)  // constrained regression
    mat mA = e(b)
    return scalar cont = el(mA, 1, colnumb(mA, "x"))
end

simulate olsCoeff = r(ols) controlFuncCoeff=r(cont), reps(100): simcont 

kdensity olsCoeff, xline(3, lcolor(green)) addplot(kdensity controlFuncCoeff) ///
    legend(label(1 "KDE of OLS coeff. estimates") label(2 "KDE of control function coeff. estimates")) ///
    xtitle("estimates") ytitle("density") title("Comparison of OLS and control function approaches") 

which produces the following picture (based on 100 replications)

enter image description here

However, I would think very carefully and experiment with more regressors and in general more data configurations before I put this estimation strategy to work on real data.


Follow-up questions:

The OP has asked for some clarifications in the comments for which I am providing an updated answer.

Let me rewrite your model $$ Y_i = \beta_0 + \beta_1 Z_{1i} + \beta_2 Z_{2i} + \beta_3 X_i + U_i $$ where $Z_{1i}$ and $Z_{2i}$ are exogenous, and $X_{i}$ is endogenous, that is $\mathbb{E}(X_iU_i) = \rho$. In addition, you want the variable $Z_{2i}$ to be constructed as $$ Z_{2i} = \mathbf{1}_{[i\text{ is odd.}]} $$ You are simulating the OLS estimates of the coefficient on $X_i$, that is $\beta_3$. Here is a small Stata script to do that.

clear*
program simcont, rclass
    syntax [, errorVariance(real 1.0)]
    drop _all
    set obs 1000
    scalar beta0 = 5
    scalar beta1 = 1
    scalar beta2 = 2
    scalar beta3 = 3
    scalar rho = 0.1

    g z1 = rnormal()
    g z2 = mod(_n, 2)
    g u = sqrt(`errorVariance')*rnormal() 
    g x = rho*u/`errorVariance' + rnormal()

    g y = beta0 + beta1*z1 + beta2*z2 + beta3*x + u
    reg y z1 z2 x
    mat mA = e(b)
    return scalar ols = el(mA, 1, colnumb(mA, "x"))

    // return the results of the heteroskedasticity test
    estat hettest, rhs iid
    return scalar hettestPValues = r(p)
end

// simulate with error variance = 1
simulate olsCoeff = r(ols) hettestPValues = r(hettestPValues), reps(100): simcont 
su olsCoeff hettestPValues // p-values have the correct mean; no heteroskedasticity

cap mat drop biasBeta
forvalues errorVariance = 2(1)6 {
    simulate olsCoeff = r(ols), reps(100): simcont, errorVariance(`errorVariance') 
    qui su olsCoeff
    mat biasBeta = (nullmat(biasBeta), r(mean) - 3)
    local colNames "`colNames' errVar:`errorVariance' "
}
mat colnames biasBeta = `colNames'
mat list biasBeta

Now note that there are at least two discrepancies here.

  • The first thing is that there is no heteroskedasticity in the model as you have written it. The mean of the p-values of an LM test of homoskedasticity from the simulations indicates that they are being drawn under the null.
. su olsCoeff hettestPValues // p-values have the correct mean; no heteroskedasticity

    Variable |       Obs        Mean    Std. Dev.       Min        Max
-------------+--------------------------------------------------------
    olsCoeff |       100    3.094182    .0326048   3.007202   3.179945
hettestPVa~s |       100    .4762952    .2885692   .0039824   .9844939
  • The next thing to note is that the bias is that the asymptotic bias is the same for all values of the error variance, as long as the degree of endogeneity is the same. Here are the results of my simulation
. mat list biasBeta

biasBeta[1,5]
       errVar:    errVar:    errVar:    errVar:    errVar:
            2          3          4          5          6
r1  .09913641  .10274499   .0912232  .11309942  .09122764

If you are getting results different than this, then you should show us your code, and we can compare the two.

Update to follow-up

Let me rewrite your model one more time $$ Y_i = \beta_0 + \beta_1Z_{1i} + \beta_2 Z_{2i} + \beta_3 X_i + U_{1i} $$ where as before, $Z_{1i}$ and $Z_{2i}$ are exogenous and $X_i$ is endogenous. You assume the data generating process $$ \begin{align} Z_{2i} &= \mathbf{1}_{[i\text{ is odd}]}\\ X_i &= \alpha_0 + \alpha_1Z_{1i} + U_{2i} \end{align} $$ You introduce endogeneity by assuming that $U_{1i}$ and $U_{2i}$ are correlated. $$ \mathbb{C}(U_{1i}, U_{2i}) = \rho $$

Also, you assume that the errors in the reduced form equation are heteroskedastic: $$ \begin{align} \mathbb{V}(U_{2i}\mid Z_{2i} = 0) &= 1 \\ \mathbb{V}(U_{2i}\mid Z_{2i} = 1) &= \tfrac{1}{q} \\ \end{align} $$

Here is the Stata code to simulate this DGP.

clear*
program simcont, rclass
    syntax [, Q(real 1.0)]

    drop _all
    set obs 1000
    scalar beta0 = 5
    scalar beta1 = 1
    scalar beta2 = 2
    scalar beta3 = 3

    scalar alpha0 = 1
    scalar alpha1 = 4

    scalar rho = 0.1

    g z1 = rnormal()
    g z2 = mod(_n, 2)
    scalar a11 = 1
    scalar a12 = rho*sqrt(1)*sqrt(1)
    scalar a13 = rho*sqrt(1)*sqrt(1/`q')
    scalar a21 = rho*sqrt(1)*sqrt(1)
    scalar a22 = 1
    scalar a23 = 0
    scalar a31 = rho*sqrt(1)*sqrt(1/`q')
    scalar a32 = 0
    scalar a33 = 1/`q'

    mat corrMatrix= (a11, a12, a13 \ a21, a22, a23 \a31, a32, a33)
    drawnorm u1 u3 u4, cov(corrMatrix)
    g u2 = cond(z2, u3, u4)

    g x = alpha0 + alpha1*z2 + u2
    g y = beta0 + beta1*z1 + beta2*z2 + beta3*x + u1
    reg y z1 z2 x
    mat mA = e(b)
    return scalar ols = el(mA, 1, colnumb(mA, "x"))

    // return the results of the heteroskedasticity test
    qui reg x z2
    estat hettest, rhs iid
    return scalar hettestPValues = r(p)
end

// simulate to check for heteroskedasticity
simulate olsCoeff = r(ols) hettestPValues = r(hettestPValues), reps(100): simcont, q(5) 
su hettestPValues // strong evidence of heteroskedasticity in the reduced form

cap mat drop biasBeta
forvalues q = 2(1)5 {
    simulate olsCoeff = r(ols), reps(1000): simcont, q(`q') 
    qui su olsCoeff
    mat biasBeta = (nullmat(biasBeta), r(mean) - 3)
    local colNames "`colNames' q:`q' "
}
mat colnames biasBeta = `colNames'
mat list biasBeta
  • Note that the heteroskedasticity is where you put it in the model, and the simulation results are now able to find it.
. su hettestPValues // strong evidence of heteroskedasticity in the reduced form

    Variable |       Obs        Mean    Std. Dev.       Min        Max
-------------+--------------------------------------------------------
hettestPVa~s |       100    1.53e-26    8.12e-26   3.09e-38   6.22e-25
  • Also, I can confirm that what you claim is actually true, that as the heteroskedasticity in the reduced form equation increases, the bias in the OLS estimator also increases.
biasBeta[1,4]
            q:         q:         q:         q:
            2          3          4          5
r1  .11242463  .11808594  .12122701  .12210455

A simple explanation of this is that as $q$ increases, the conditional (on $Z_{2i}=1$) and hence the unconditional variance of the reduced form error $U_{2i}$ decreases (check through a variance decomposition) which means that the variance of the endogenous regressor decreases, which means that the $(\mathbf{X}'\mathbf{X})^{-1}$ increases in magnitude, and the overall bias increases (this is a very rough description -- I am sure it can be formalized).

$\endgroup$
  • $\begingroup$ Sry for the late response. Thx for your detailed answer but it was not quite what I was looking for. I awarded you anyway^^. I was more looking for a analytic way to determine how the impact of correlation biases the ols-estimator because in a simulation i want to calculate this particular bias as exact as possible. For my simple case i could easily caclulate that bias i was looking for. For a more advanced problem not so much. $\endgroup$ – Druss2k Nov 22 '12 at 11:01
  • 1
    $\begingroup$ @Druss2k Do you want to show that if you use the estimator $\widetilde{\boldsymbol{\beta}} = (\mathbf{X}'\mathbf{X})^{-1}\mathbf{X}'\boldsymbol{Y} - (\mathbf{X}'\mathbf{X})^{-1}\widehat{\boldsymbol{\rho}}$, where $\widehat{\boldsymbol{\rho}} = \mathbf{X}'\boldsymbol{U}$ -- you will get an unbiased estimator? This is not particularly hard to prove. $\endgroup$ – tchakravarty Nov 22 '12 at 12:48
  • 1
    $\begingroup$ Furthermore, the asymptotic bias of the OLS estimator is easily written as $\mathbb{E}(\widehat{\boldsymbol{\beta}} \mid \mathbf{X}) - \boldsymbol{\beta} = (\mathbf{X}'\mathbf{X})^{-1}\boldsymbol{\rho}$, where $\boldsymbol{\rho} = \mathbb{E}(\boldsymbol{X}U)$. $\endgroup$ – tchakravarty Nov 22 '12 at 12:52
  • 1
    $\begingroup$ @Druss2k I will update my answer to include this information. $\endgroup$ – tchakravarty Nov 23 '12 at 11:43
  • 1
    $\begingroup$ @Druss2k I have updated the answer to include the DGP from the commentspace. I think this answers your question in full. $\endgroup$ – tchakravarty Nov 28 '12 at 19:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.