Take a standard regression framework: $$ Y_{it} =\beta X_{it} + \epsilon_{it}$$ Assume for simplicity that no omitted variables exist, nor are there simultaneity or measurement problems.

In several economic disciplines I have seen literature where occasionally statements of the following nature are made: The regression only implies correlation, but does not imply causation. We cannot rule out that Y causes X rather than X causing Y.

I am currently in a situation like this myself and I am wondering: Isn't this a far bigger problem than most of these authors make it out to be ? If Y causes X and Y by very definition is correlated with its own error term, wouldn't X be endogeneous and the estimated coefficient biased ? Thereby making it impossible to say anything about both correlation and causation, rather than just causation ?

Edit: The regression implies correlation, it does not show it. Thanks @ Tim

  • $\begingroup$ Regression does not show correlation... correlation shows correlation. Correlation is symmetric, while regression estimates are asymmetric. $\endgroup$
    – Tim
    Dec 2, 2015 at 9:59
  • $\begingroup$ This may be just a misunderstanding on my part, but: A regression coefficient is defined as the covariance over the variance, whereas correlation is the covariance over the standard deviations. Assuming non-zero variances a regression coefficient does imply existence and sign of the correlation (not size)? $\endgroup$
    – Peter
    Dec 2, 2015 at 10:08
  • $\begingroup$ Yes, they are related but not the same and the relation is that simple only with simple regression - in multivariate case it resembles rather parial correlation... $\endgroup$
    – Tim
    Dec 2, 2015 at 10:20
  • $\begingroup$ @Tim (since you are in the discussion already), isn't ruling out simultaneity targetted to ensuring that $X_{it}$ is not endogenous (which would remove the whole problem of this post)? If not, then where do we utilize the absence of simultaneity? $\endgroup$ Dec 2, 2015 at 19:51
  • $\begingroup$ The point is that there are no other factors except for the reverse causality that cause a problem. Simultaneity would suggest a factor Z causes both Y and X, reverse causality suggests that Y causes X and thereby creates a problem. The question boils down to: does reverse causality cause a bias in my estimation of the coefficient or just a lack of causal inference. $\endgroup$
    – Peter
    Dec 2, 2015 at 20:01

1 Answer 1


Assume that the true causal relation is

$$x_i = ay_i + u_i \tag{1}$$

with the $u$-vector independent of the $y_i$-vector, but we mispecify

$$y_i = bx_i + \epsilon_i \tag{2}$$

And we get the theoretical relationship (substituting $(1)$ in $(2)$ and applying expected values)

$$b = \frac 1a \tag{3}$$

Attempting an OLS estimation for $b$ we get

$$\hat b = \frac {\sum x_iy_i}{\sum x_i^2}$$

What does this estimate in reality?
We need to plug in eq.$(1)$ to find out (since this is the true causal relationship, by assumption, while $(2)$ is just a figment of our imagination). We get

$$\hat b = \frac {\sum (ay_i + u_i)y_i}{\sum (ay_i + u_i)^2} = \frac {a\sum y_i^2 + \sum u_iy_i}{a^2\sum y_i^2 + 2a\sum u_iy_i + \sum u_i^2}$$

This is certainly a biased estimator. Asymptotically, given the independence between $y_i$ and $u_i$ (orthogonality would suffice) we will get (multiplying up and down by $(1/n)$)

$$\hat b \xrightarrow{p} \hat b_p = \frac {aE(y^2)}{a^2E(y^2) + \sigma_u^2} = \frac 1a \cdot \left(\frac {E(y^2)}{E(y^2)+ (\sigma_u/a)^2}\right) \tag{4}$$

This shows that $\hat b$ is an inconsistent estimator for $1/a$. The term in the big parenthesis is always positive and smaller than unity, so we get the "attenuation bias" (bias towards zero) phenomenon, i.e. the plim of $\hat b$ will be closer to the zero value than $1/a$ irrespective of whether $a$ is positive or negative.

Can we do anything else? Well what if we attempt to estimate the variance using the residuals? We have

$$\hat \sigma^2_{\epsilon} = \frac 1n \sum \hat \epsilon_i^2 = \frac 1n \sum [y_i-\hat b(ay_i +u_i)]^2 = \frac 1n \sum [(1-\hat b a)y_i-\hat bu_i)]^2$$

$$= (1-\hat ba)^2\frac 1n \sum y_i^2 -2(1-\hat b a)\hat b\frac 1n\sum y_iu_i + \hat b^2\frac 1n\sum u_i^2$$

The probability limit of this is

$$\hat \sigma^2_p = (1-\hat b_pa)^2E(y^2) + \hat b_p^2 \sigma^2_u \tag{5}$$

Now note that
a) for the left-hand sides of $(4)$ and $(5)$ we have consistent estimates from the estimation procedure (since they are the actual probability limits of the estimators we used) b) We can estimate consistently $E(y^2)$

So if you rearrange $(4)$ to solve for $a$, rearrange $(5)$ to solve for $\sigma^2_u$, and use $\hat b$ instead of $\hat b_p$, $\hat \sigma^2_{\epsilon}$ instead of $\hat \sigma^2_p$, and $(1/n)\sum y_i^2$ instead of $E(y^2)$ you have a system of two equations in two unknowns ($a$ and $\sigma^2_u$).

Does it give a solution?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.