All Questions
Tagged with communication information-theory
3 questions
0
votes
0
answers
14
views
How does source and channel coding work without cancelling out?
Source coding is basically compressing raw message bits. Channel coding is adding redundancy. Both are counter to one another. Yet, in a digital communication block diagram, they are always next to ...
- 1
0
votes
0
answers
26
views
What is the non-asymptotic counterpart of the mutual information?
The mutual information of a joint probability distribution $p(x,y)$ tells us, if we send $n$-letter messages with each letter drawn from the marginal distribution $p_X(x)$, that we can use roughly $2^{...
- 393
5
votes
1
answer
636
views
Find the input/output probability distributions that realize the capacity of communications channel?
Consider two random variables $X,Y$. For simplicity, they are discrete and finite. Let $Q(y|x)$ be the conditional probability of $Y$ given $X$. Their mutual information is defined as:
$$I=H(Y)-H(Y|X)...
- 4,552