3
$\begingroup$

I've just spent the last 3 hours reading every post, question, Medium article, and textbook entry on conditional independence, and I still don't really understand it. Can somebody explain what it means in layman's terms? I think I'm getting tripped up on the semantics.

One example I frequently come upon is: a person's height and vocabulary are conditionally independent, given age. What I don't understand is, what are they when not given age? Dependent? Wikipedia says so.

This is preposterous to me. Height is not dependent upon vocabulary, nor vice versa. (This brings up another question: does dependent mean "caused by?" Does it mean "correlated with?") In any case, it seems to me that any two events are either empirically independent or they are empirically dependent. Their relationship to each other cannot magically change. Bart and Lisa are still brother and sister regardless of whether or not we know that Homer is father to both.

The existence of a notion called "conditional" independence doesn't make sense to me. And the semantics confuse me. Indeed, it seems that height and vocabulary are conditionally dependent--on age! In other words, age is a variable that binds these things (making them less independent).

My head is in knots.

Improve this question
$\endgroup$
2

1 Answer 1

Reset to default
1
$\begingroup$

It may be useful for you to initially think of 'dependence' and 'independence' here as statements about correlation or association, rather than as direct statements of causality. They are useful in developing a framework in which to think about causality, though this is easier to comprehend once the basic idea is in place.

In your example, once we have accounted for our knowledge about the age of an individual, height tells us nothing about their vocabulary and vice versa. In other words: at any particular age, being taller or shorter should have no association with vocabulary (I'm going to ignore rare medical conditions that may cause reductions in both). This makes height and vocabulary conditionally independent.

But now assume that we do NOT know anything about age. If I told you that the individual was 40 cms tall, you could probably make a better than random guess about their vocabulary - simply because (i) very short individuals are likely to be infants and (ii) infants are likely to have a smaller vocabulary than adults.

Similarly, if I gave told you than an individual knew ~100 words in total, you would probably guess that they are very short - because (i) individuals with a tiny vocabulary are likely to be children, and (ii) children are likely to be shorter than adults.

This makes height and vocabulary conditionally dependent on each other. This means that though A and B do not affect each other directly, they both affect C, and so information about C and one of A or B therefore tells you something about the other.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks for the response! So it's a description of a situation in which we have super limited knowledge. It's not an empirical state, it's a state that describes only what we can surmise at a particular moment. Also, A and B can only ever be described as conditionally (in)dependent of each other but in relation to a third variable. It's meaningless to talk about conditional independence without a 3rd variable. Which would mean that A & B can be conditionally independent with respect to C, but not with respect to D. Is this all correct? $\endgroup$
    – NaiveBae
    Sep 28, 2022 at 17:01
  • 1
    $\begingroup$ @NaiveBae I'll try to address those one at a time. (i) it's a description of a situation in which we have super limited knowledge. I don't think our level of knowledge is particularly relevant. We know a lot about heights, vocabularies, and age, after all! (ii) It's not an empirical state, it's a state that describes only what we can surmise at a particular moment. That may overstate things a little bit. It describes the nature of the relationships, which governs what it is possible to infer. $\endgroup$
    – mkt
    Sep 28, 2022 at 17:06
  • 1
    $\begingroup$ @NaiveBae (iii) A and B can only ever be described as conditionally (in)dependent of each other but in relation to a third variable. It's meaningless to talk about conditional independence without a 3rd variable. Which would mean that A & B can be conditionally independent with respect to C, but not with respect to D. Yes, this is correct. $\endgroup$
    – mkt
    Sep 28, 2022 at 17:08
  • 1
    $\begingroup$ @NaiveBae Does that address your questions? $\endgroup$
    – mkt
    Sep 28, 2022 at 17:09
  • 1
    $\begingroup$ Thank you, I'm grateful for your responses. This largely clarifies things. $\endgroup$
    – NaiveBae
    Sep 29, 2022 at 23:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.