How do I intuitively understand that independence is always symmetric?

Question

Independence between two events, $A$ and $B$, is a symmetric relation, that is, if $P(A \mid B) = P(A)$, then $P(B \mid A) = P(B)$. The proof is very simple and can be found at the ProofWiki.

Intuitively, when I think about dependence, I think about relations like $A = 2*B$. In this case, $A$ clearly depends on $B$ (it is always the double of $B$). However, in this case, the equation doesn't say that the value of $B$ depends on the value of $A$. Indeed, if that was the case, this would be a circular definition. So, in general, when I think about $A$ being dependent on $B$, I don't really think about $B$ being also dependent on $A$ (even in our every day life, this is not usually the case). However, in this case, we are talking about dependence and not independence. Nonetheless, if event $A$ is not dependent on $B$, then event $A$ is independent of $B$ (trivial). But does this imply that the dependence relation is also symmetric, that is, if $P(A \mid B) \neq P(A)$, then $P(B \mid A) \neq P(B)$?

In general, if dependence is also symmetric, how do we intuitively understand that this is the case? Also, in general, how do we intuitively understand that independence is symmetric?

Answer 1

$A=2B$ is also symmetric, you can equivalently say that $B=\frac12 A$. "Equality" is a symmetric relation.

The usual definition of independence of events is $\DeclareMathOperator{\P}{\mathbb{P}}\P(A \cap B)=\P(A)\P(B)$ so symmetry is clear. But this is a mathematical explanation, and you ask for intuition.

First, in your example I depend on water to survive, but water does not depend on me you use depend in a colloquial sense, not a technical sense. In that example, there is no probability model, so the concept of stochastic independence do not apply. So I guess what you need is an analysis of different uses (and senses) of the word depend. Looking at OED the senses given there, none is the probabilistic one! and none of those are symmetric.

This intuition from colloquial use simple cannot be taken over to the technical sense. But, more deeper, often in uses of statistics we are really more interested in causal dependence which is a much more difficult concept than stochastic dependence (defined as simply $\P(A \cap B)\not=\P(A)\P(B)$.)

But it is possible for causally dependent events to be stochastically independent (maybe unusual.) For this see Is there an example of two causally dependent events being logically (probabilistically) independent?.

Answer 2

You can convert $A=2B$ to $B=A/2$. These are not programmatic variable definitions, so you shouldn't think one-way. If $A$ is equal to $2B$, then, if I give you $A$, you can certainly deduce $B$, by halving it. For daily events, $A$ and $B$, say you know $B$ depends on $A$ without no doubt, e.g. (A) if it rains, (B) ground will be wet. If we know that it is raining ($A$), we're going to have quite definite idea about the situation (wet or not) of the ground, i.e. $B$. Conversely, if we know $B$, i.e. that the ground is wet, it's certainly going to give us some idea about the probability that it was raining, $A$. So, when thinking both ways, you don't only consider causality, and you're going to ask the following question: does knowing one imply something about the other?

Answer 3

Intuitively, when I think about dependence, I think about relations like A=2∗B. In this case, A clearly depends on B (it is always the double of B). However, in this case, the equation doesn't say that the value of B depends on the value of A. Indeed, if that was the case, this would be a circular definition.

You seem to be using the non-symmetric 'causation' in place of 'statistical dependence' or functional relationship.

Say you throw a ball at an angle 45 degrees. Then the distance $x$ in meters and speed $v$ in meters per second could be related by:

$$x = \frac{2}{9.8} v^2$$

There is nothing circular to say that '$x$ depends on $v$' as well as '$v$ depends on $x$'. At least, not in a statistical sense. The dependence here is that you can predict the speed at which a ball has been thrown based on the distance that it travels. And vice versa, you can predict the distance that a ball travels based on the speed at which it has been thrown.

With causation you indeed have non symetric relations. E.g. you have if-then relationships which are depicted by a nonsymmetric arrow. See also https://en.wikipedia.org/wiki/Affirming_the_consequent

---

If two events are independent, thus neither one influences the probability of the other, then the probability for both events to occur together is P(a,b) = P(a)P(b), the product of the individual proabilities. From that you can deduce P(a|b) = P(a,b)/P(b) = P(a)

Answer 4

Dependence is also a symmetric relation.

If event $A$ depends on event $B$, then $P(A, B) = P(A|B)P(B) \iff P(A, B)/P(B) = P(A|B)$. However, given that $P(A, B) = P(B, A)$, then $P(B, A) = P(B|A)P(A)$, which implies $P(A, B)/P(A) = P(B|A)$.