Difference between mutual and conditional information I have studied about the basic probability theory. Now I am studing about entropy from information theory point of view from Bose's Information Theory, Coding and Cryptography.
I have not understood the difference between mutual and conditional information.
Suppose there are two random variabes $X\;,Y$ with outcomes $x_i$ where $i=1,2,...,n$ and $y_j$ where $j=1,2,...m$.
The Mutual Information $I(x_i;y_j)$ between $x_i$ and $y_j$ is defined as $I(x_i;y_j)=\log \frac{P(x_i|y_j)}{P(x_i)}$.
The conditional information between $x_i$ and $y_j$ is defined  as $I(x_i|y_j)=\log \frac{1}{P(x_i|y_j)}$
They give an example for mutual information in the book.

We can see that if $p=0$ (meaning the channel is ideal), then knowing the input is $0$ or $1$ gives us all the information about the output (which is also $0$ or $1$ respectively).
For $p=0$, $I(Y=0;X=0)=\log_2 2(1-p)=1$.
Also, $I(Y=0|X=0)=-\log_2(P(Y=0|X=0))=-\log_2(1-p)=-\log_2(1)=0$
Q-1 I am not able to understand intuitively what the difference is between $I(Y=0;X=0)=1$ and $I(Y=0|X=0)=0$. What is the difference between mutual and conditional information?
Q-2 For $p<0.5$, $I(Y=0;X=0)=\log_22(1-p)<0$. What does the negative information mean?
 A: I think using $\operatorname{I}$ for both conditional information and pointwise mutual information might confuse things, so I'll use $h$ for conditional information and $\operatorname{pmi}$ for PMI.
Q1: pointwise mutual information versus conditional entropy
The 'conditional information' $h(y_0 \mid x_0)$ is a measure of uncertainty: how surprised are we that $Y = 0$ given that $X = 0$? In a noiseless channel, we're not surprised at all, which is why the value is $0$. If we already know that $X = 0$ in a noiseless channel, then finding out that $Y=0$ gives no additional information. As the noise increases, so will our surprise if $Y=0$ given that $X=0$.
By contrast, $\operatorname{pmi}(x_0; y_0)$ is a measure of the shared information content. Given what we already know about the individual events $x_0$ and $y_0$, how much does one tell us about the other?
The two quantities can be related by this equality: $\operatorname{pmi}(x_0; y_0) = h(y_0) - h(y_0 \mid x_0)$. Knowing that the event $x_0$ occurs may make $y_0$ more or less surprising. The $\operatorname{pmi}$ function quantifies this change in surprise in bits.
Q2: meaning of negative PMI
When the pointwise mutual information is negative, this means that $x$ and $y$ co-occur less frequently than would be expected if they were independent. This can be made clearer by rewriting the formula:
$$
\operatorname{pmi}(x_0; y_0) = \log \frac{p(y_0 \mid x_0)}{p(y_0)} = \log \frac{p(y_0 \mid x_0) p(x_0)}{p(y_0)p(x_0)} = \log \frac{p(y_0, x_0)}{p(y_0)p(x_0)}
$$
The fraction is less than 1 when the two events don't tend to co-occur as often as if they were independent; the log of the fraction is thus negative.
Because of the very-high noise in the channel, we don't expect $x$ and $y$ to have the same value; it's more likely expect the channel to corrupt $x$.
