1
$\begingroup$

Most of the time-series data I'll be looking at is linear and uniform - a straightish light parallel to the x-axis.

The exceptions I need to find are those that deviate recently, the last one, or two entries. These will probably be quadratic or exponential in their deviation from the straight line.

Running scipy's curve fit for a straight line isn't going to work very well, with a slope of 0. Even matching to a quadratic with such a long, flat base-line won't work well.

What is the recommended way of dealing with this? As an example, here are three data sets, the first normal, the second, and third, showing the deviation:

a: (0,34) (305,40) (630,46) (920,33)


b: (0,87) (50,87) (200,84) (311,83) (644,90) (828,110)


c: (0,187) (150,170) (203,188) (321,176) (644,190) (828,44)

In an ideal world, I'd like to get an answer that shows the level of deviation from linear, taking into account the noise level of the early readings. If the readings have a high standard deviation, then the final one being different is less significant than if they're tightly clustered to the line.

$\endgroup$
0
$\begingroup$

If you curve fit straight lines both with and without the last data point, the relative change in slope may pinpoint when the deviation from linear occurs.

$\endgroup$
  • $\begingroup$ Yes, it will. The problem is, that the slope, for a line parallel to the x-axis is 0 - y = mx + c. $\endgroup$ – Peter Brooks Feb 17 '17 at 0:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.