Variable frequency redistribution

Question

I know that there is a way to "redistribute the frequencies of a variable" as stated here:

Slide number 14 and 15 about redistribution

and here:

Dorian Pyle book chapter 7 section 2 paragraph 3 (7.2.3):

The easiest way to adjust distribution density is simply to displace the high-density points into the low-density areas until all points are at the mean density for the variable. Such a process ends up with a rectangular distribution.

Illustrating this passage are figures like these:

In both the references the problem is clear and solved but does not provide sufficient information for writing an algorithm and in particular does not say how to calculate the displacement (the way to move the frequencies on the left or right of the center of the distribution).

Please can you help me to understand it ??

Answer 1

This is an empirical version of the "probability integral transform."

Suppose you have $n$ data values. Sorting them in ascending order, index them as

$$x_1 \le x_2 \le \cdots \le x_n.$$

Replacing each $x_i$ by $i$ gives as uniform a distribution as possible. That's all there is to it.

If you would like the replacement values to lie within a given range, such as the range $[0,1]$ in the illustration, then rescale the $i$. In general, if the range is to be $[a,b]$, then rescale $i$ to $a + (b-a)(i-1/2)/n$. To see that this works, note that the rescaling is linear in $i$ and by sending $i=1$ to $a + (b-a)/(2n)$ and $i=n$ to $b - (b-a)/(2n)$ it places the extreme values of $x$ at small, equal distances just inside the interval. Alternatively, rescale $i$ to $a + (b-a)(i-1)/(n-1)$ to place the extrema exactly at $a$ and $b$.

Example (in R)

Generate and plot some sample data.

n <- 100
x <- 10*rbeta(n, 1, 3)
hist(x, col="Gray")

Compute the "displacement" (as the difference between $x$ and the rescaled index $i$, given here as $y$) and plot its graph:

x <- sort(x)
x.min <- min(x)
x.max <- max(x)
y <- x.min + (x.max - x.min) * ((1:n)- 0.5) / n
displacement <- y - x
plot(x, displacement, type="l", lwd=2, main="'Displacement Graph'")

Finally, confirm the effect by plotting a histogram of the "displaced" values of $x$:

n.breaks <- 10
breaks <- (0:n.breaks) * (x.max - x.min)/n.breaks + x.min
hist(x + displacement, breaks=breaks, col="Gray")