Converting arbitrary distribution to uniform one

Question

I have 500,000 values for a variable derived from financial markets. This variable has a arbitrary distribution. I need a formula that will allow me to select a range around any value of this variable such that an equal (or close to it) amount of values fall within that range. From what I understand, this means that I need to convert it from arbitrary distribution to uniform distribution. I have read (but barely understood) that what I am looking for is called "probability integral transform."

Can anyone assist me with some code (Matlab preferred, but it doesn't really matter) to help me accomplish this?

Edit: I uploaded my dataset as a .csv file and compressed .rar file

I used the empirical distribution function within MatLab and got the following plot: Does this look about right? Here is a histogram of the raw data for reference:

Answer 1

If $X$ has the (cumulative) distribution function $F(x)=P(X<x)$, then $F(X)$ has a uniform distribution on $[0,1]$. You don't know what $F$ is, but with N = 500,000 data points you could simply use the empirical distribution function:

$$\hat{F}(x) = \frac{1}{N} \sum_{i=1}^N 1[x_i\leq x]$$

where $1[A]$ is the indicator function, $1[A]=1$ is $A$ is true and $1[A]=0$ if $A$ is false. The function $F$ is often also called the quantile function.

---

In coding terms, once you've written your function F you now have two objects, x containing your data and q containing the transformed data, so you could write a function Finv which takes a number in [0,1] and returns the value of your sample distribution at that quantile (using linear interpolation or some other appropriate method for filling in the gaps).

Now if you want to take e.g. 5% of the data either side of the value x0, your range will be Finv(F(x0) - 0.05) to Finv(F(x0) + 0.05).

Answer 2

Suppose you have a cumulative distribution function $F$ of the variable in question. Suppose the value given is $x$, and the range is $[r_1,r_2]$ with $x\in[r_1,r_2]$. Then if you select the amount of values falling into that range $N$, the following should hold:

$$F(r_1)-F(r_2)=\frac{N}{500 000}$$

This is an equation with 2 unknown variables, so we need some restrictions to solve it. The popular one would be setting $r_1=x-\varepsilon/2$ and $r_2=x+\varepsilon/2$. This would give us the equation of one variable, which could be solved pretty easily, using any optimisation algorithm.

The only thing you need is the function $F$. You can either model it, or use some non-parametric estimate. In the latter case probably optimisation algorithm may be unnecessary, as it should be possible to work out the solution.

Note: This is only one possible approach, depending on data it might not work.