2
$\begingroup$

I have 2 random variables, A and B, both of which are categorical. I understand that I could calculate their mutual information, which would give me a measure of the information that they share. However, I would like to know how much information is gained about B by knowing that A has a specific value, a. The only name I could think of for something like this would be 'Conditional Mutual Information', i.e. the mutual information of A and B given that A=a.

I am very new to stats, but it seems to me like it would be possible to condition the standard MI formula on A=a, but I don't know if doing that even makes sense. Googling yielded nothing for that name, so I asked here.

$\endgroup$
2
$\begingroup$

The term "conditional mutual information" is reserved for mutual informations between at least three variables, and refers to the shared information between two variables when a third is known. Its labelled $I(A;B|C)$. What you want sounds more like $H(B) - H(B|A=a)$ (i.e. the entropy of $B$ minus the conditional entropy of $B$ given $A = a$), which isn't strictly a mutual information, but would represent the reduction in uncertainty when $A=a$. The mutual information is just the expectation of this $\forall a \in A$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.