3
$\begingroup$

I lack knowledge of statistical terminology, so I'll try to thoroughly explain my predicament and hope that it is understandable:

I am programming a software for technical analysis of financial markets.
This software will receive a variable who's value represents current market conditions.

In this particular case the variable represents a measurement, in standard deviations, of the financial vehicle's price in respect to its moving average. The exact nature of this variable is not relevant to my question - the important part is that it has non-uniform distribution.

What I mean by this is that at any given time the variable has a much greater chance of being somewhere around 0 than of being at one of the extremes.

Now, the point of my software is to find and evaluate similar conditions in the market history. If I look for the exact same value (ex. 2.0021) I will not find it; so what I must do is create an interval around 2.0021 and evaluate whatever values fall within the interval.

Then, when I receive a different input value, I will need to create an interval of a different size, such that approximately the same amount of values will be found within this interval.

Here is a concrete example:

CURRENT VALUE Xa = 2.0021

Lets say I found that 100 values of Xa fall within the interval of 1.9992 < X < 2.0050...
(The size of the interval (obviously) is 0.0058)

Next I receive another value for X:

CURRENT VALUE Xb = 4.5010

Values around 4.5 are much more rare than values around 2.08. Therefore, the interval surrounding 4.5010 must also be larger in order for my software to find the same number of results as (close to) Xa ;

I am trying to create intervals in my data such that each interval contains the same number of samples within its bounds.

In other words: The CLOSER TO 0, the more data points are present, the SMALLER the interval needs to be. The farther from 0, the fewer data points are present, the greater the interval needs to be in order to contain the same amount of results.

And now to the heart of my question: How can I formulaically calculate the size of the interval around X based on the value of X?

It is also crucial to mention that I have a ton (500,000) of real X values to work from.

This is what I tried - I feel like I'm close but not quite there:

I created a histogram out of my 1D data. Since the data is symmetric, and centered on 0, I created another histogram based on the data's absolute value.

My Histograms

I do not know where to go from here :(

I tried doing an exponential regression to the data; it fits very well but I do not know how to use the formula to create the sort of bounds within my data that I want.

I seriously regret not taking a Stats class in college...

Can anyone help me out please?

Improve this question
$\endgroup$
3
$\begingroup$

If your data really is exponentially distributed, find the maximum likelihood estimate of the rate $\lambda$, then transform the samples $X_n$ to a sequence $Y_n$ uniformly distributed in $[0,1]$ using the equation $Y_n=1-\mathrm e^{-\lambda X_n}$

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks so much for the speedy response! I am terrible at reading math notation but it looks straightforward enough :) Also - with Matlab I get the best fit when I do regression with this formula: a*x^b+c (99% fit) Would there happen to be a more general way to do this that might for that formula? $\endgroup$ – Mike Furlender May 10 '11 at 2:58
  • $\begingroup$ Is there a name for what I am trying to do, in case I need to find out how to do it for another distribution? Is it "confidence interval," or is that something else? $\endgroup$ – Mike Furlender May 10 '11 at 3:03
  • $\begingroup$ Read the linked article to learn how to estimate the rate. What you want to do is called "Transformation of a random variable". $\endgroup$ – Emre May 10 '11 at 3:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.