# Information Entropy - Ambiguous Notation

I'm constantly confused by the ambiguous notation when discussing Entropy (H) and Mutual Information (I). For example, here's a formula from "Elements of Information Theory": $$I(X;Y|Z) = H(X|Z) − H(X|Y,Z)$$

In this, $$H(X|Y,Z)$$ can mean two things:

a) Entropy of $$X$$ given $$(Y,Z)$$ or

b) Entropy of $$(X|Y)$$ and $$Z$$

These two are completely different things!

Which of these two is correct? Logically I can see that it must be a) and not b), but this is only because of intuition. In many other formulas elsewhere, the formula is ambiguous.

Is there some rule to tell exactly what it means?

$$\text{H}(X|Y,Z)$$ means the entropy of $$X$$ when both $$Y$$ and $$Z$$ are given. This is true for probabilities/densities, i.e. $$\text{P}(X|Y,Z)$$ means the distribution of $$X$$ given $$Y,Z$$. So, you're correct; it is (a) and I've never seen the usage of (b) anywhere.
• $H(X,Y|Z)$ is the entropy of $(X,Y)$ given $Z$. The rules are simple. You'll assume all the variables to the right of the given sign, i.e. $\vert$, is given, and to the left of it is not given. Nov 2, 2019 at 14:26
• I haven't seen a single reference to your situation in either your book or another one, including prob & stats. And, imho (since I can't find a discussion on the topic) it's not a possible one. When a RV is given, it's given for all the RVs. Otherwise, you're referring to different sample spaces. For example, $P(X|Y, Z)$ would mean probability of $X$ happening when $Y$ happened, and $Z$ happened (when $Y$ is not known). You won't be able to calculate it. An event either occurs or not. Nov 3, 2019 at 6:34
• @BhagwadJalPark Well, I meant what you are trying to find doesn't exist in the literature because it doesn't make sense. If something (an event, a RV) is given, it is given for all the RVs in the space. For example, $P(A|B\cap C)$ means prob. of A happening if $B$ and $C$ are given. The following doesn't have a meaning: $P((A|B)\cap C)$, e.g. prob. of $A$ when $B$ happened and $C$. Did $B$ happen or not? Nov 3, 2019 at 17:51