0
$\begingroup$

I have a list of values which start and continue close to 0 before increasing towards the end (see attached plot of these values). I am interested in the value at which this increase starts - I could do this by manually observing the plot and values, but I would like to explore a more robust method, and find the point at which a statistically significant increase in values starts. This seems like it would be a fairly simple procedure but I haven't found previous examples of this yet, and do not know the proper term for this sort of procedure.

Any suggestions as to statistical methods I could investigate would be much appreciated!

For reference, my values are derived from digitised coordinates on a curve, and each value is the distance from a coordinate to the circumference of a circle. So I am trying to find the point at which my coordinates deviate away from a circular progression.

plot of values

Improve this question
$\endgroup$
3
  • $\begingroup$ A naive approach would be to order the values as they appear and compute the differences. If your values do not "jitter" but only increase monotonically, then you only have wait until the first positive value shows up. If the values jitter, by that I mean, if there is some random noise to them, then you need to estimate the variation of that noise, for example using the standard deviation of past n values, and then wait until price difference increase more than 3 time the standard deviation. $\endgroup$
    – ChrisL
    Commented Nov 14 at 20:10
  • $\begingroup$ Do you know anything about the nature of the data generating system? $\endgroup$
    – Michael Lew
    Commented Nov 14 at 20:10
  • $\begingroup$ What, precisely, would a "circular progression" be? $\endgroup$
    – whuber ♦
    Commented Nov 14 at 22:15

2 Answers 2

Reset to default
1
$\begingroup$

This is a problem that falls under the category Change detection. For this particular problem, the change detection method called CUSUM seems very well suited because only too positive values are a problem.

There are more or less complicated ways of using CUSUM depending on what distribution to use. The simplest is to assume that both the steady states and the inflated states are normally distributed. In that case CUSUM becomes the cumulated sum and to detect when the cumulative sum is significantly different, one could use the V-shapes (see slides)

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you, very useful as I wasn't aware of change detection or CUSUM before. Currently working through the qcc package for R. $\endgroup$
    – Emily S
    Commented Jul 4, 2016 at 13:56
0
$\begingroup$

Finding the threshold consistently is always difficult because the rate of change at the threshold is minute. Depending on your analytical and inferential objectives it might be worth considering, instead of "the value at which this increase starts", the value at which the increase reaches a specified value.

That is effectively what pharmacologists do when finding the $EC_{50}$ of a concentration-response curve. The measurement noise in the $EC_{50}$ is much less than that for threshold estimates because the curve is much steeper at the $EC_{50}$ than it is where it just becomes non-zero.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.