I am trying to compute the mutual information between a channel with two inputs and two outputs. I know that for a channel with 2 inputs (U,V) and 1 output (Y), the mutual information takes the following form:

\begin{equation} I(U,V;Y) = H(U,V)+H(Y) - H(U,V,Y) \end{equation}

I, on the other hand, have 2 input distributions and 2 output distributions. I also have marginal probability distributions of the inputs and the outputs and also the conditional probabilities between inputs and outputs (for convenience, I will label input distributions with $I_1$ and $I_2$ and similarly output distributions by $O_1$ and $O_2$). So What I have is the following:

\begin{equation} P(I_1),P(I_2), P(O_1), P(O_2),P(O_1,O_2|I_1,I_2). \end{equation}

What I am looking for is: \begin{equation} I(I_1,I_2;O_1,O_2)? \end{equation}

What would be the correct expression for the expression above?


https://www3.nd.edu/~jnl/ee80653/Fall2005/tutorials/sunil.pdf https://en.wikipedia.org/wiki/Multivariate_mutual_information


After reading William J. McGill's 1954 paper 'Multivariate Information Transmition', the answer seems to be:

\begin{equation} I(I_1,I_2;O_1,O_2) = H(I_1,I_2) + H(O_1,O_2) - H(I_1,I_2,O_1,O_2) \end{equation}

Please correct me if I am wrong.

Reference: http://link.springer.com/article/10.1007%2FBF02289159


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.