8
$\begingroup$

Let $Y$ and $X$ be random variables. $E(Y|X)$ is the conditional mean of $Y$ given $X$. We say $Y$ is not causally related to $X$ if $E(Y|X)$ does not depend on $X$, which implies it is equal to $E(Y)$. Now, let's go along with this definition of causality for a second. By the law of iterated expectations, $E(XE(Y|X)) = E(E(XY|X)) = E(XY)$. This means that if $E(Y|X)$ does not depend on $X$, if it is equal to $E(Y)$, then $E(X)E(Y) = E(XY)$.

In other words:

If $X$ and $Y$ are not causally related, then $X$ and $Y$ are uncorrelated! - This makes no sense and I know this must be wrong. Have I defined causality incorrectly? What have I done wrong?

In econometrics we generally assume $E(Y|X) = b_0 + b_1X$. So $E(Y|X) = E(Y)$ is equivalent to $b_1 = 0$. The logic applies in this specific scenario too.

$\endgroup$
  • 2
    $\begingroup$ You said that $E(XE(Y|X))=E(E(XY|X))=E(XY)$. I believe this is wrong. E(Y|X) is a constant. Therefore, $E(XE(Y|X))$ is equal to $E(Y|X)E(X)$. Another point, $E(Y|X)=b0+b1*X$ comes from the simple linear regression model. $\endgroup$ – Budhapest Sep 12 '13 at 1:36
  • $\begingroup$ Let E(Y|X) = b, where b is a constant. Then take expectations of both sides. One finds that E(E(Y|X)) = E(b) = b. By law of iterated expectations, E(E(Y|X)) = E(Y). Therefore, if E(Y|X) is constant, it must be equal to E(Y). $\endgroup$ – Christian Sep 12 '13 at 1:38
  • $\begingroup$ If E(Y/X)=b, that's implies Y does not depend on X, and E(Y)=b, you are confusing yourself. $\endgroup$ – SAAN Sep 12 '13 at 5:39
  • $\begingroup$ I don't understand why "this makes no sense". You are starting off with a definition of causality that is I think equivalent to definition of independence in statistics. And independent variables have zero covariance, where is the story? $\endgroup$ – January Sep 12 '13 at 6:17
  • $\begingroup$ January, no, they are not the same thing! X and Y are independent if the joint distribution factors into the product of the marginals, and this is definitely not the same thing. I don't see what your point is? Azeem, aside from restating what I previously said, do you have anything to contribute? Instead of saying I am wrong, can you explain WHY I am wrong? $\endgroup$ – Christian Sep 12 '13 at 6:41
17
$\begingroup$

You have defined causality incorrectly, yes. Probably, you have heard the saying "correlation isn't causation." You have essentially defined causality as correlation. The problem is worse than that, though. Causality is not a statistical or probabilistic concept at all, at least as those topics are normally taught. There is no statistical or probabilistic definition of causality: nothing involving conditional expectations or conditional distributions or suchlike. It is hard to pick up this fact from courses in statistics or econometrics, though.

Unfortunately, we tend to do a better job saying what causality isn't than what causality is. Causality always and everywhere comes from theory, from a priori reasoning, from assumptions. You mentioned econometrics. If you have been taught instrumental variables competently, then you know that causal effects can only be measured if you have an "exclusion restriction." And you know that exclusion restrictions always come from theory.

You said you wanted math, though. The guy you want to read is Judea Pearl. It's not easy math, and the math sometimes wanders off into philosophy, but that's because causality is a hard subject. Here is a page with more links on the subject. Here is a free online book I just came across. Finally, here is a previous question where I gave an answer you might find useful.

$\endgroup$
  • $\begingroup$ Thank you sincerely. I will have a read of his work and get back to you when I have time. $\endgroup$ – Christian Sep 12 '13 at 13:45
  • 4
    $\begingroup$ Excellent answer. The Morgan & Winship book is quite a bit easier than Pearl, with a focus on social science problems. $\endgroup$ – Dimitriy V. Masterov Sep 12 '13 at 16:53
8
$\begingroup$

We say $Y$ is not causally related to $X$ if $E(Y|X)$ does not depend on $X$, which implies it is equal to $E(Y)$.

This is wrong. Causal relations are about functional/structural dependencies, not statistical/associational dependencies. You should take a look here.

Have I defined causality incorrectly? What have I done wrong?

Yes, you have defined it incorrectly, you can check causal inference books/references here. More formally, in a structural equation model the causal effect of $X$ on the distribution of $Y$, which we can denote by $P(Y|do(X = x))$ --- that is, how changing $X$ affects the distribution of $Y$ --- is mathematically defined as the probability distribution induced by the modified structural equation model where the equation for $X$ is substituted for $X = x$.

For example, suppose your causal model is defined by the following structural equations:

$$ U = \epsilon_u\\ X = f(U, \epsilon_x)\\ Y = g(X,U, \epsilon_y) $$

Where the disturbances are mutually independent and have some probability distribution. This corresponds to the DAG:

$\hskip2in$enter image description here

Then $P(Y|do(X = x))$ is the probability distribution of $Y$ induced by the modified structural equations:

$$ U = \epsilon_u\\ X = x\\ Y = g(X, U, \epsilon_y) $$

Which corresponds to the mutilated DAG:

$\hskip2in$enter image description here

The average causal effect would be simply the expectation of $Y$ using the causal cdf $P(Y|do(X=x))$.

$$ E[Y|do(X =x)] = \int Y dP(Y|do(X = x)) $$

This is the mathematical definition, whether you can identify the effect with observational data depends on whether you can re-express $P(Y|do(X=x))$ in terms of the observational distribution without the $do()$ operator.

$\endgroup$
2
$\begingroup$

A Counterexample

The problem doesn't seem to be that mean independence (the condition where $E[Y|X] = E[Y]$) implies that $Y$ and $X$ are uncorrelated. If $X$ and $Y$ are not correlated, it is not generally true that they are mean independent. So this doesn't seem problematic so far.

However, suppose you had a relationship (we can call it causal) defined as $Y = WX$, where $X$ is distributed with a standard normal distribution and $W$ is distributed with a Rademacher distribution so that $W = 1$ or $-1$, each with probability $1/2$ (see this Wikipedia article). Then notice that $E[Y|X] = E[Y]$. Under your definition, this relationship would not be causa even though $Y$ clearly depends on $X$.

An Example of a Formal Way of Thinking About Causality

To give you maybe a clearer and more mathematical way to look at causality, take the following example. (I borrow this example from the book "Mostly Harmless Econometrics.") Suppose you want to analyze the effect of hospitalization on health. Define $Y_i$ as some health measure of individual $i$ and $D_i \in \{0,1\}$ to indicate whether or not that individual was hospitalized. In our first attempt, suppose we look at the average difference in health of the two kinds of individuals: $$ E[Y_i | D_i=1] - E[Y_i|D_i=0]. $$ On first look at the data, you might notice, counter intuitively, that individuals that have been hospitalized actually have worse health than those that have not. However, going to the hospital certainly does not make people sicker. Rather, there is a selection bias. People who go to the hospital are those people that are in worse health. So this first measure does not work. Why? Because we are not interested in just the observed differences, but rather in the potential differences (we want to know what would happen in the counter-factual world).

Define the potential outcome of any individual as follows: $$ \text{Potential Outcome} = \left \{ \begin{array}{ll} Y_{1,i} & \text{if } D_i = 1 \\ Y_{0,i} & \text{if } D_i = 0. \end{array} \right . $$ $Y_{0,i}$ is the health of individual $i$ if he had not gone to the hospital, regardless of whether he actually went or not (we want to think about counterfactuals) and in the same way, $Y_{1,i}$ is the health of the individual is he did go. Now, write the actual observed outcome in terms of the potentials, $$ Y_i = \left \{ \begin{array}{ll} Y_{1,i} & \text{if } D_i = 1 \\ Y_{0,i} & \text{if } D_i = 0. \end{array} \right. $$ Thus, $Y_i = Y_{0,i} + (Y_{1,i} - Y_{0,i}) D_i$. Now, we can define the causal effect as $Y_{1,i} - Y_{0,i}$. This works because it is in terms of potentials. Now, suppose we again look at the observed differences in average health: \begin{align*} E[Y_i | D_i=1] - E[Y_i|D_i=0] &= E[Y_{1,i}|D_i = 1] - E[Y_{0,i}|D_i = 1] \\ & \qquad + E[Y_{0,i}|D_i=1] - E[Y_{0,i}|D_i=0]. \end{align*} Notice that the term $E[Y_{1,i}|D_i = 1] - E[Y_{0,i}|D_i = 1]$ can be interpreted as the average treatment effect on the treated and $E[Y_{0,i}|D_i=1] - E[Y_{0,i}|D_i=0]$ as the bias in selection. Now, if the treatment $D_i$ is assigned randomly, then we have \begin{align*} E[Y_i | D_i=1] - E[Y_i|D_i=0] &= E[Y_{1,i}|D_i] - E[Y_{0,i}|D_i=0] \\ &= E[Y_{1,i}|D_i] - E[Y_{0,i}|D_i=1] \\ &= E[Y_{1,i} - Y_{0,i}|D_i=1] \\ &= E[Y_{1,i} - Y_{0,i}], \end{align*} where we see that $E[Y_{1,i} - Y_{0,i}]$ is the average causal effect that we are interested in. This is a basic way of thinking about causality.

$\endgroup$
1
$\begingroup$

I went through your proof, and I think it is correct (at least, I checked all the steps for discrete definition of $E()$). If $E(Y|X) = E(Y)$, then $E(X\cdot Y) = E( X )\cdot E( Y )$. Also, it works the other way.

However, I don't see where is your problem?

  1. If $X$ and $Y$ are independent, then they have zero covariance. But
  2. If $X$ and $Y$ have zero covariance, then they are not necessarily independent.

Example: consider following table:

     Y
 X | -1      0      1
 --+---------------------
-1 | 0.25    0     0.25
 1 |   0    0.5      0

The values are probabilities, i.e. $P(X=1 \wedge Y=0) = 0.5$ etc. Marginal probabilities for Y are 0.25, 0.5, 0.25, and 0.5 and 0.5 for X.

It is easy to see that $E(Y) = E(X) = E(X \cdot Y) = 0$ and that $E(Y|X=-1)=E(Y|X=1)=0$ and therefore $E(Y|X)=E(X)$, therefore by your definition the variables are causally unrelated.

The covariance is zero because $E(X\cdot Y) = E(X)\cdot E(Y)$.

However, the two variables are not independent, because $P(X = 1 \wedge Y = 0 ) = 0.5 \ne 0.5 \cdot 0.5 = P(X=1)\cdot P(Y=0)$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.