A Counterexample
The problem doesn't seem to be that mean independence (the condition where $E[Y|X] = E[Y]$) implies that $Y$ and $X$ are uncorrelated. If $X$ and $Y$ are not correlated, it is not generally true that they are mean independent. So this doesn't seem problematic so far.
However, suppose you had a relationship (we can call it causal) defined as $Y = WX$, where $X$ is distributed with a standard normal distribution and $W$ is distributed with a Rademacher distribution so that $W = 1$ or $-1$, each with probability $1/2$ (see this Wikipedia article). Then notice that $E[Y|X] = E[Y]$. Under your definition, this relationship would not be causa even though $Y$ clearly depends on $X$.
An Example of a Formal Way of Thinking About Causality
To give you maybe a clearer and more mathematical way to look at causality, take the following example. (I borrow this example from the book "Mostly Harmless Econometrics.") Suppose you want to analyze the effect of hospitalization on health. Define $Y_i$ as some health measure of individual $i$ and $D_i \in \{0,1\}$ to indicate whether or not that individual was hospitalized. In our first attempt, suppose we look at the average difference in health of the two kinds of individuals:
$$
E[Y_i | D_i=1] - E[Y_i|D_i=0].
$$
On first look at the data, you might notice, counter intuitively, that individuals that have been hospitalized actually have worse health than those that have not. However, going to the hospital certainly does not make people sicker. Rather, there is a selection bias. People who go to the hospital are those people that are in worse health. So this first measure does not work. Why? Because we are not interested in just the observed differences, but rather in the potential differences (we want to know what would happen in the counter-factual world).
Define the potential outcome of any individual as follows:
$$
\text{Potential Outcome} = \left \{
\begin{array}{ll}
Y_{1,i} & \text{if } D_i = 1 \\
Y_{0,i} & \text{if } D_i = 0.
\end{array}
\right .
$$
$Y_{0,i}$ is the health of individual $i$ if he had not gone to the hospital, regardless of whether he actually went or not (we want to think about counterfactuals) and in the same way, $Y_{1,i}$ is the health of the individual is he did go. Now, write the actual observed outcome in terms of the potentials,
$$
Y_i = \left \{
\begin{array}{ll}
Y_{1,i} & \text{if } D_i = 1 \\
Y_{0,i} & \text{if } D_i = 0.
\end{array}
\right.
$$
Thus, $Y_i = Y_{0,i} + (Y_{1,i} - Y_{0,i}) D_i$. Now, we can define the causal effect as $Y_{1,i} - Y_{0,i}$. This works because it is in terms of potentials. Now, suppose we again look at the observed differences in average health:
\begin{align*}
E[Y_i | D_i=1] - E[Y_i|D_i=0] &= E[Y_{1,i}|D_i = 1] - E[Y_{0,i}|D_i = 1] \\
& \qquad + E[Y_{0,i}|D_i=1] - E[Y_{0,i}|D_i=0].
\end{align*}
Notice that the term $E[Y_{1,i}|D_i = 1] - E[Y_{0,i}|D_i = 1]$ can be interpreted as the average treatment effect on the treated and $E[Y_{0,i}|D_i=1] - E[Y_{0,i}|D_i=0]$ as the bias in selection. Now, if the treatment $D_i$ is assigned randomly, then we have
\begin{align*}
E[Y_i | D_i=1] - E[Y_i|D_i=0] &= E[Y_{1,i}|D_i] - E[Y_{0,i}|D_i=0] \\
&= E[Y_{1,i}|D_i] - E[Y_{0,i}|D_i=1] \\
&= E[Y_{1,i} - Y_{0,i}|D_i=1] \\
&= E[Y_{1,i} - Y_{0,i}],
\end{align*}
where we see that $E[Y_{1,i} - Y_{0,i}]$ is the average causal effect that we are interested in. This is a basic way of thinking about causality.
