In the paper Prioritized Experience Replay, the authors introduced a rank-based way to compute the priority of a transition. They said:
For the rank-based variant, we can approximate the cumulative density function with a piecewise linear function with $k$ segments of equal probability. The segment boundaries can be precomputed (they change only when $N$ or $α$ change).
I don't very understand what they were trying to convey in this sentence: where does the cumulative density function come from?
I have a simple thought about the rank-based implementation based on what I can extract from the paper-- not so sure if it approaches what the authors talked about.
We set the range of a segment in inverse proportion to the priorities of transitions in it. For example, if one segment has the average priority of $p$ and another has $2p$, then the range of the first is two times more than that of the second.
There is an obvious deficiency in this method: I just assumed the priorities are uniformly distributed, things will be more tricky if they're not. For now, however, I cannot get any further.