OLS: $E[\epsilon_{it}^T\epsilon_{it}] \not= 0$ in 1st equation biases standard errors in 2nd equation?

Question

Suppose ${X_{it}},{Y_{it}}$ are time series with $X_{it}\sim N(0.1,1)$, ($\sigma^2(Y_{it}) = 1$ and $mean(Y_{it})$ is similar to that for $X_{it}$, but changes when the dummy = 1). and $t \in \{1,2,...,200\}$, $i \in \{1,2,...,N\}$. In a real world setting this will be periodic stock market returns over $N$ firms (but you can ignore this). There is a dummy, $D_t$ that's equal to unity over $t \in \{150,151,...,200\}$ and equal to zero otherwise. The time series model to be estimated with OLS $\forall i$ is:

$(1) Y_{it} = \alpha_i + \beta_i X_{it} + \gamma_i D_{t} + \epsilon_{it}$

This model generally adheres to the Gauss-Markov assumptions for each $i$. However, we have $E[\epsilon_{it}^T \epsilon_{jt}] \not= 0$ for all $i$ and $j$.

The next step is to construct a vector of gammas using the $N$ estimates of model $(1)$. Call this vector $\bf{\hat{\gamma}}$. We then use this in the cross-sectional model:

$(2) \hat{\gamma}_i = a + b Z_i + u_i$

where $Z_i$ is some cross-sectional variable that doesn't cause any violations in OLS assumptions and is relevant for explaining $\hat{\gamma}_i$.

---

The claim in the applied econometrics literature is that $E[\epsilon_{it}^T \epsilon_{jt}] \not= 0$ in model $(1)$ leads to (i) No problem for the OLS coefficient estimates in $(2)$, but (ii) Biased standard errors in $(2)$.

• Can someone please post ideas about why this is the case?

• I don't understand what $\epsilon_{it}^T$ is in the expression $E[\epsilon_{it}^T \epsilon_{jt}] \not= 0$. Of course $\epsilon_{it}$ is a scalar and you can't transpose a scalar. This is seen HERE, where they apply this methodology.

Answer 1

In order to be sure you need to go into the details, this implies comparing the true variance covariance matrix with the one you get in the second ols stage.

The true one:

This can be obtained by replacing eq.2 into eq.1, pooled OLS follows, and from it, the true $\hat a , \hat b$ variance covariance matrix:

$Y_{it} = \alpha_i + \beta_i X_{it} + aD_t + bD_tZ_{i} +D_t u_{i} + \epsilon_{it}$

Using matrix notation to split the equation in $\gamma$ parameters and others leads to:

$Y = X\theta + Z\gamma + \varepsilon$

where we are interested in $V(\hat \gamma)$ , $\gamma=[a \; b] $, Z is a two column vector $Z=[D_t \; D_tZ_i]_{[i=1,..,N;t=1,...,T]}$ ( a similar structure defines X but this is not of interest) and where $V(\varepsilon) =\Sigma$ has a full structure of between firms covariances that's why it's not diagonal ($\sigma^2I_{NT}$) like in the GAUSS-MARKOV assumptions. By Frish-Waugh we can express $\gamma$ ols as :

$\hat \gamma = (Z'M_{X}Z)^{-1}Z'M_{X}Y$ where $M_X= I-X(X'X)^{-1}X' $

which implies the following true variance:

$V(\hat \gamma) = H\Sigma H'$ where $H = (Z'M_{X}Z)^{-1}Z'M_{X}$

The other one

Under the assumption of non correlated firms (and time periods but this is not the issue), $\Sigma$ has a simplier diagonal structure $\Delta$. This means that $\Delta$ triangular terms are 0. Under an even simpler specification,( the one that is estimated by default by econometric and statistical software for OLS) $\Sigma$ follows GAUSS-Markov assumptions meaning that even the diagonal terms are equal thus $\Sigma$ is downgraded to $\sigma^2I$

This implies that not considering between firms correlation would lead to $V(\hat\gamma) $as:

$V(\hat \gamma) = H\Delta H'$ or $V(\hat \gamma) = H\sigma^2I H' \equiv \sigma^2(Z'M_xZ)^{-1}$

which, as it can be seen, are not equal to the true one.

Answer 2

I am putting up another answer with more details.

In standard linear regression model (in matrix form):

$$Y=X\beta+\varepsilon$$

the OLS estimate is the following

$$\hat\beta=(X^TX)^{-1}X^TY.$$

Its variance then is

$$Var(\hat\beta)=(X^TX)^{-1}X^TVar(Y)X(X^TX)^{-1}.$$

The usual assumption for regression is that

$$Var(Y)=\sigma^2I,$$

where $I$ is the identity matrix. Then

$$Var(\hat\beta)=\sigma^2(X^TX)^{-1}.$$

Now in your case you have two models:

$$Y_{i}=M_i\delta_i+\epsilon_i$$

and

$$\Gamma=Lc+u,$$

where

• $Y_i^T=(Y_{i1},...,Y_{iT})$,
• $M_i=[1,X_i,D]$, with $X_i^T=(X_{i1},...,X_{iT})$, $D^T=(D_1,...,D_T)$
• $\delta_i^T=(\alpha_i,\beta_i,\gamma_i)$
• $\epsilon_i^T=(\epsilon_{i1},...,\epsilon_{iT})$
• $\Gamma^T=(\gamma_1,...,\gamma_n)$
• $L=[1,Z]$, with $Z^T=(Z_1,...,Z_n)$
• $c^T=(a,b)$
• $u^T=(u_1,...,u_N)$.

Note that you state second model for the estimates of $\gamma$, which is not usual, hence I restate it in usual form, for the "true" $\gamma$.

Let us write down the covariance matrix for OLS estimates of coefficients $c$:

$$Var(\hat{c})=(L^TL)^{-1}L^TVar(\Gamma)L(L^TL)^{-1}$$

The problem is that we do not observe $\Gamma$. We observe the estimates $\hat\Gamma$. $\hat\gamma_i$ is part of vector

$$\hat\delta_i=\delta_i+(M_i^TM_i)^{-1}M_i^T\epsilon_i.$$

Assume that $\delta_i$ are random and independent with $\epsilon_i$ and $M_i$. This surely holds for $\gamma_i$ so we do not lose anything if we extend this for other elements of $\delta_i$.

Let us stack all $\hat\delta_i$ on top of each other:

$$\hat\delta^T=[\delta_1^T,...,\delta_N^T]$$

and explore the variance of $\hat\delta$:

$$Var(\hat\delta)=\begin{bmatrix} Var(\hat\delta_1) & cov(\hat\delta_1,\hat\delta_2) & \dots & cov(\hat\delta_1,\hat\delta_N)\\ \dots & \dots & \dots & \dots\\ cov(\hat\delta_n,\hat\delta_1) & cov(\hat\delta_n,\delta_2) & \dots & Var(\hat\delta_N) \end{bmatrix}$$

Assume that $Var(\epsilon_i)=\sigma^2_\epsilon I$ and that $E\epsilon_i\epsilon_j^T=0$. For $i\neq j$ we have

\begin{align} cov(\hat\delta_i,\hat\delta_j)&=cov(\delta_i,\delta_j)+cov((M_i^TM_i)^{-1}M_i^T\epsilon_i,(M_j^TM_j)^{-1}M_j^T\epsilon_j)\\ &=(M_i^TM_i)^{-1}M_i^TE(\epsilon_i\epsilon_j^T)M_j(M_j^TM_j)^{-1}\\ &=0 \end{align}

For diagonal elements we have

$$ Var(\hat\delta_i)=Var(\delta_i)+\sigma_\epsilon^2(M_i^TM_i)^{-1} $$

Let us turn back to variance of $\hat c$. Since we substitute $\hat\Gamma$ instead of $\Gamma$ the variance is the following

$$Var(\hat{c})=(L^TL)^{-1}L^TVar(\hat\Gamma)L(L^TL)^{-1},$$

We can extract $Var(\hat\Gamma)$ from $Var(\hat\delta)$ by selecting appropriate elements:

$$Var(\hat\Gamma)=Var(\Gamma)+diag(g_1,...,g_n)$$

where $g_i$ is the element of $\sigma_\epsilon^2(M_i^TM_i)^{-1}$ corresponding to the $Var(\hat\gamma_i)$. Each $g_i$ is different from $g_j$ since they correspond to different $X_{it}$ and $X_{jt}$ which are not assumed to be equal.

So we get the surprising result, that algebraically even if we assume all the necessary properties, the resulting covariance matrix at least algebraically will not be equal to usual OLS covariance matrix, since for that we need that $Var(\hat\Gamma)$ is constant times identity matrix which it is clearly not.

All the formulas above were derived assuming that $X_{ij}$ are constant, so they are conditional on $X_{ij}$. This means that we actually calculated $Var(\hat\Gamma|X)$. By putting additional assumptions on $X_{ij}$, I think it would be possible to show that unconditional variance is OK.

The independence assumption placed on $\epsilon_i$ can also be relaxed to uncorrelatedness.

It would also be possible to use simulation study to see how covariance matrix differ if we use $\hat\Gamma$ instead of $\Gamma$.

Answer 3

I think the problem lies in definition of the second model. I think it is assumed that

$$\gamma_i=a+bZ_i+u_i$$

with the usual assumption that

$$cov(\gamma_i,\gamma_j|Z_1,...,Z_N)=0,$$

i.e. that the $\gamma_i$ are not correlated if we control for $Z_i$. Now when you substitute $\hat{\gamma}$ instead of $\gamma$, you need to check whether the assumption holds, i.e. if

$$cov(\hat{\gamma_i},\hat{\gamma}_j|Z_i)=0.$$

Now

$$\hat{\gamma}_i=\gamma_i+L(\epsilon_{it}),$$

where $L$ is some linear function. It is safe to assume that $\epsilon_{it}$ is independent of $Z_i$, but if $E\epsilon_{it}\epsilon_{jt}\neq0$, the necessary assumption does not hold.

Since the uncorrelatedness assumption is central to calculation of usual OLS statistics, this gives the reason why the standard errors are biased.

This was a rough outline, but I think the idea should work if you'd get into nitty gritty details of OLS machinery.