Intuition for Rayleigh PDF

Question

We have ground-truth data $\mathbf{x}^* = (0,0)^T \in \mathbb{R}^{2}$. Furthermore, we have $N$ measurements $\mathbf{x}_i \in \mathbb{R}^{2}, i\in \{1,\ldots,N\}$. We calculate the $N$ error vectors $\mathbf{e}_i = \mathbf{x}_i - \mathbf{x}^*$. We assume that the two components of $\mathbf{e}_i = (e_{x,i}, e_{y,i})^T $ are independent of each other and normally distributed with equal mean $\mathbf{\mu} = (0,0)^T$ and variance $\sigma^2$. The absolute errors $e_i = \sqrt {e_{x,i}^2 + {e}_{y,i}^2} $ follow a Rayleigh distribution, is that correct?

The probability density function (PDF) of the Rayleigh function is zero at a value of zero. Why is this the case (intuitively)? I would assume that our errors should most likely be 0 (i.e. PDF should be highest at 0)...

Answer 1

Here's some intuition:

The bivariate distribution of the errors has its maximum at 0. However, the distribution of the distance from the center does not, since the only point contributing density there is the one at the center.

As you move out a little the bivariate density has decreased only a little but you get a little "circle" of contributions to the density of distance from the center, making a small distance relatively more probable than a 0 distance.

Then as you keep increasing the radius, the larger circumference "picks up" more density while the joint density of the error at that distance decreases more rapidly. Eventually you reach a peak, where the rate of loss from the second and gain from the first are cancelling, and then after that the increasing radius loses out and the density of distance from the center of the error distribution starts to fall.

Answer 2

No, the OP's claim that

"the two components of $\mathbf{e}_i = (e_{x,i}, e_{y,i})^T $ are independent of each other and normally distributed with equal mean $\mu$ and variance $\sigma^2$"

is incorrect: the means are $0$ because the OP subtracted off $\mu$ when you defined $\mathbf{e}_i $ as being $ \mathbf{x}_i - \mathbf{x}^*$. Edit: After I wrote the above, the OP corrected the statement in the question.

As to the question as to why the pdf of the Rayleigh random variable is not maximum at $0$ because the errors are more likely to be close to $0$, it is true that the joint pdf of $e_{x,i}$ and $e_{y,i}$ has a peak at $(0,0)$ but the probability that $e_i = \sqrt {e_{x,i}^2 + {e}_{y,i}^2}$ is small, say $\leq \epsilon$, is the volume under the joint pdf in a very slim (diameter $2\epsilon$) cylinder, and this is converging to $0$ as $\epsilon \to 0$. More generally, for $r \in [0,\infty)$, the event $\{r \leq e_i \leq r+\Delta r\}$ occurs whenever the point $(e_{x,i},e_{x,i})$ is in the annular region that lies between the circles of radius $r$ and $r+\Delta r$ centered at the origin. In this region, the pdf has value $\approx \frac{1}{2\pi \sigma^2}\exp(-r^2/2\sigma^2)$ while the area of the region is $\pi (r+\Delta r)^2 - \pi r^2 \approx 2\pi r\Delta r$ giving that $$P\{r \leq e_i \leq r+\Delta r\} \approx f_{e_i}(r)\Delta r \approx \frac{1}{2\pi \sigma^2}\exp(-r^2/2\sigma^2)\cdot 2\pi r\Delta r,$$ that is, $$f_{e_i}(r) = \frac{r}{\sigma^2}\exp(-r^2/2\sigma^2), r \geq 0$$

which is the Rayleigh pdf. Since $r$ increases steadily while $\exp(-r^2/2\sigma^2)$ decreases (slowly at first, but then very rapidly) as $r$ increases from $0$ to $\infty$, the Rayleigh pdf increases from $0$ at first, but soon reaches a peak and then declines rapidly towards $0$. The location of the peak is $\sigma$ as we can determine via the standard calculus methods (or looking up the answer on Wikipedia), but notice that the peak of the Rayleigh pdf is at the point where its CDF $1-\exp(-r^2/2\sigma^2)\mathbf 1_{\{r\colon r \geq 0\}}$ has maximum derivative. Recalling that the normal density function $\frac{1}{\sigma\sqrt{2\pi}}\exp(-r^2/2\sigma^2)$ has inflection points at $r = \pm \sigma$, we can deduce the location of the peak using only statistical knowledge instead of mindless mathematical calculations.

Answer 3

Your "absolute error" random variable is Rayleigh distributed, with PDF $$\frac{x}{\sigma^2} e^{-x^2/(2\sigma^2)}, \quad x \geq 0$$ only if the mean of $e_i$ is centered on the point $x^*$. (If not, then you do have a "systematic error" which, if you can't factor out, would require you to use the Rice distribution to model the absolute error.)

Note that your variable e is non-negative, and any "error" represents a positive value. So yes, if variance is non-zero then the cumulative probability at exactly zero (i.e., "no error") is zero.

I find it convenient to think of this graphically: You're defining "absolute error" as a distance from some point in two dimensions. The only way for the distance to be zero is for a sample to exactly match $x^*$.