I lack knowledge of statistical terminology, so I'll try to thoroughly explain my predicament and hope that it is understandable:
I am programming a software for technical analysis of financial markets.
This software will receive a variable who's value represents current market conditions.
In this particular case the variable represents a measurement, in standard deviations, of the financial vehicle's price in respect to its moving average. The exact nature of this variable is not relevant to my question - the important part is that it has non-uniform distribution.
What I mean by this is that at any given time the variable has a much greater chance of being somewhere around 0 than of being at one of the extremes.
Now, the point of my software is to find and evaluate similar conditions in the market history. If I look for the exact same value (ex. 2.0021) I will not find it; so what I must do is create an interval around 2.0021 and evaluate whatever values fall within the interval.
Then, when I receive a different input value, I will need to create an interval of a different size, such that approximately the same amount of values will be found within this interval.
Here is a concrete example:
CURRENT VALUE X
a= 2.0021
Lets say I found that 100 values of X
a fall within the interval of 1.9992 < X < 2.0050
...
(The size of the interval (obviously) is 0.0058
)
Next I receive another value for X:
CURRENT VALUE X
b= 4.5010
Values around 4.5 are much more rare than values around 2.08. Therefore, the interval surrounding 4.5010
must also be larger in order for my software to find the same number of results as (close to) Xa ;
I am trying to create intervals in my data such that each interval contains the same number of samples within its bounds.
In other words:
The CLOSER TO 0
, the more data points are present, the SMALLER
the interval needs to be.
The farther from 0, the fewer data points are present, the greater the interval needs to be in order to contain the same amount of results.
And now to the heart of my question:
How can I formulaically calculate the size of the interval around X
based on the value of X
?
It is also crucial to mention that I have a ton (500,000) of real X
values to work from.
This is what I tried - I feel like I'm close but not quite there:
I created a histogram out of my 1D data. Since the data is symmetric, and centered on 0, I created another histogram based on the data's absolute value.
I do not know where to go from here :(
I tried doing an exponential regression to the data; it fits very well but I do not know how to use the formula to create the sort of bounds within my data that I want.
I seriously regret not taking a Stats class in college...
Can anyone help me out please?