Is there such a thing as conditional mutual information?

Question

I have 2 random variables, A and B, both of which are categorical. I understand that I could calculate their mutual information, which would give me a measure of the information that they share. However, I would like to know how much information is gained about B by knowing that A has a specific value, a. The only name I could think of for something like this would be 'Conditional Mutual Information', i.e. the mutual information of A and B given that A=a.

I am very new to stats, but it seems to me like it would be possible to condition the standard MI formula on A=a, but I don't know if doing that even makes sense. Googling yielded nothing for that name, so I asked here.

Answer 1

The term "conditional mutual information" is reserved for mutual informations between at least three variables, and refers to the shared information between two variables when a third is known. Its labelled $I(A;B|C)$. What you want sounds more like $H(B) - H(B|A=a)$ (i.e. the entropy of $B$ minus the conditional entropy of $B$ given $A = a$), which isn't strictly a mutual information, but would represent the reduction in uncertainty when $A=a$. The mutual information is just the expectation of this $\forall a \in A$.