For simplicity, you may assume everything is discrete/finite.

$x\in\mathcal{X}$ goes through a channel and produces $y\in\mathcal{Y}$.

The mutual information between them is $i=I(x\Vert y)$.

My questions are with regards to the following scenario: what if we subject the channel to erasures with a probability $p$? That is to say, if an erasure happens, then instead of getting $y$, you get a useless erasure $e$, and glean no information. Let $\widehat{y}_p\in\mathcal{Y}\cup \{e\}$ denote this $y$ subject to erasures.

  1. What will be the new mutual information $I(x\Vert\widehat{y}_p)$ in terms of the original $i$? Or is there no simple form?
  2. Is there some straightforward way to degrade the channel (without introducing an outright erasure) such that the resulting mutual information is the same as in the erasure case? By degrading the channel, I mean that instead of $y$, you'll get a noisy $\widetilde{y}_p$ that still belongs to the domain $\mathcal{Y}$ (doesn't involve erasures). But it should be the case that $I(x\Vert\widetilde{y}_p)=I(x\Vert\widehat{y}_p)$.

For (2.), an example noisy $\widetilde{y}_p$ would be to replace $y$ with noisy uniform over $\mathcal{Y}$ w.p. $p$. I don't know if this will necessarily result in the same mutual information though.

If necessary, you can just consider the binary domain case. If possible, I'd like to know for the continuous case.

Even if there is no simple/straightforward answer, I'd like to know what an answer would be (even if it's intractable to compute).



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.