# Is mutual information meaningful for specific values of two random variables?

The mutual information of two random variables $X$ and $Y$ tells us how much knowing one reduces uncertainty about the other (roughly speaking). Mathematically, the mutual information is $$I(X;Y) = \sum_{x,y} p(x,y) \log \frac{p(x,y)}{p(x)p(y)}.$$ I was wondering whether it is meaningful to consider the mutual information for specific values of $X$ and $Y$, i.e. $$I(X=x;Y=y) = p(x,y) \log \frac{p(x,y)}{p(x)p(y)}.$$ If so, what's the right interpretation of this quantity?

$$i(x;y) = \log \frac{p(x \mid y)}{p(x)}.$$
$$i(x;y) = h(x) - h(x \mid y) = h(y) - h(y \mid x)$$ and $$I(X \mid Y) = \mathrm{E}[i(x;y)],$$ where $h$ measures the Shannon information content and $I$ is the mutual information. Intuitively $i(x;y)$ quantifies “the amount of information provided by the occurrence of the event represented by $y$ about the occurrence of the event represented by $x$" (Fano, "Transmission of information: a statistical theory of communications," 1961).
Note that $i(x;y)$ is positive iff $p(x \mid y) > p(x)$. As a concrete example, consider the two "random variables" Weather and Forecast which can each take on two possible values: rain or shine. Let's suppose that forecasters generally do a good job so that $$P(Weather = rain \mid Forecast = shine) < P(Weather = rain).$$ This implies that $$i(Weather = rain \texttt{ };\texttt{ } Forecast = shine) < 0.$$ We consider it less likely to rain when the forecast says shine than we would if we didn't hear the forecast at all. If it does in fact rain, then the forecast was misleading or misinformative; this is what is meant by $i(x;y) < 0.$