I consider a circular complex normal variable $x = x_r + i x_i \sim \mathcal{CN}(0, \sigma^2)$. I know that the PDFs of the magnitude $r$ of that variable and the squared magnitude $s = r^2$ are given by a Rayleigh distribution $f(r;\sigma) = \frac{x}{\sigma^2} e^{-x^2/(2\sigma^2)}$ and exponential distribution $f(s,\sigma) = \frac{1}{2\sigma^2}e^{-s/(2\sigma^2)}$, respectively (c.f. for example http://www.math.uah.edu/stat/special/Rayleigh.html).

Now, if I look at the entropies of these PDFs (wiki) I find that they scale with $\log(\sigma)$ and $\log(\sigma^2)$. Here, I do not care about terms that do not depend on $\sigma$.

How can this difference be explained intuitively? In my intuition both representations should carry the same Shannon information as they are just different representations of the same quantity, completely characterized by $\sigma$. Note that - in general - a difference of two entropies will not absorb the scaling: $\log(\sigma_1)-\log(\sigma_2)\neq \log(\sigma_1^2)-\log(\sigma_2^2)$. Hence, also mutual information would scale differently with $\sigma$. As far as I know mutual information is supposed to be invariant under coordinate transformation.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.