16
$\begingroup$

I have some data that I smooth using loess. I'd like to find the inflection points of the smoothed line. Is this possible? I'm sure someone has made a fancy method to solve this...I mean...after all, it's R!

I'm fine with changing the smoothing function I use. I just used loess because that's what I was have used in the past. But any smoothing function is fine. I do realize that the inflection points will be dependent on the smoothing function I use. I'm okay with that. I'd like to get started by just having any smoothing function that can help spit out the inflection points.

Here's the code I use:

x = seq(1,15)
y = c(4,5,6,5,5,6,7,8,7,7,6,6,7,8,9)
plot(x,y,type="l",ylim=c(3,10))
lo <- loess(y~x)
xl <- seq(min(x),max(x), (max(x) - min(x))/1000)
out = predict(lo,xl)
lines(xl, out, col='red', lwd=2)

enter image description here

Improve this question
$\endgroup$
2
  • 3
    $\begingroup$ Maybe you want to have a look at change-point analysis. $\endgroup$
    – nico
    Commented Nov 19, 2013 at 7:25
  • $\begingroup$ I have found this line of code very useful: infl <- c(FALSE, diff(diff(out)>0)!=0) But this code finds all turning points regardless of it turning up or down. How can I tell which points bend up and which bend down in a timeseries? For example, plot and color upward turning point green and downward ones red. $\endgroup$
    – user3511894
    Commented Aug 15, 2016 at 3:19

6 Answers 6

Reset to default
17
$\begingroup$

From the perspective of using R to find the inflections in the smoothed curve, you just need to find those places in the smoothed y values where the change in y switches sign.

infl <- c(FALSE, diff(diff(out)>0)!=0)

Then you can add points to the graph where these inflections occur. 

points(xl[infl ], out[infl ], col="blue")

From the perspective of finding statistically meaningful inflection points, I agree with @nico that you should look into change-point analysis, sometimes also referred to as segmented regression.

Improve this answer
$\endgroup$
3
  • $\begingroup$ This seems to do the job somewhat well. I understand it's not ideal and the result it gives is certainly not ideal. Thanks for the contribution though. It covers most cases except for things like a straight line. $\endgroup$
    – user164846
    Commented Nov 19, 2013 at 17:32
  • 3
    $\begingroup$ I'm not sure I understand, @user164846. A straight line has no inflection points. $\endgroup$
    – Jean V. Adams
    Commented Jan 2, 2014 at 12:12
  • $\begingroup$ Also, you may want to have a look at smoothers that immediately provide derivatives, e.g. Savitzky-Golay-filters. However the choice of the smoother should always be decided by your data and application, not the other way round! $\endgroup$
    – cbeleites
    Commented Jan 2, 2014 at 12:20
4
$\begingroup$

There are problems on several levels here.

First off, loess just happens to be one smoother and there are many, many to choose from. Optimists argue that just about any reasonable smoother will find a real pattern and that just about all reasonable smoothers agree on real patterns. Pessimists argue that this is the problem and that "reasonable smoothers" and "real patterns" are here defined in terms of each other. To the point, why loess and why do you think that a good choice here? The choice is not just of a single smoother or a single implementation of a smoother (not all that goes under the name of loess or lowess is identical across software), but also of a single degree of smoothing (even if that is chosen by the routine for you). You do mention this point but that is not addressing it.

More specifically, as your toy example shows, basic features like turning points may easily not be preserved by loess (not to single out loess, either). Your first local minimum disappears and your second local minimum is displaced by the particular smooth you show. Inflexions being defined by zeros of the second derivative rather than the first can be expected to be even more fickle.

Improve this answer
$\endgroup$
4
  • $\begingroup$ I chose loess because I grabbed it from the internet. I'm very experienced in smoothing in general, so I simply grabbed code online. Do you have a better suggestion? $\endgroup$
    – user164846
    Commented Nov 19, 2013 at 17:33
  • $\begingroup$ Sorry, but I don't understand your comment. If you are very experienced in smoothing, you should have arguments for which smoothers best preserve inflexions while suppressing noise. That seems a contradictory aim to me, but I would be happy to hear technical arguments why I am wrong. $\endgroup$
    – Nick Cox
    Commented Nov 19, 2013 at 17:46
  • $\begingroup$ Sorry, I mean't "inexperienced" haha $\endgroup$
    – user164846
    Commented Nov 19, 2013 at 17:46
  • 1
    $\begingroup$ I see. Life is short, and you can't try every possible method. We can't see your real data, but your toy example does underline that smoothing can remove identifiable structure. $\endgroup$
    – Nick Cox
    Commented Nov 19, 2013 at 17:48
2
$\begingroup$

There are a bunch of great approaches to this issue. Some include. (1) - changepoint- package (2) - segmented - package. But you are required to choose the number of changepoints. (3) MARS as implemented in the -earth- package

Depending on your bias/variance tradeoff, all will give you slightly different information. -segmented- is well worth a look. Different number of changepoints models can be compared with AIC/BIC

Improve this answer
$\endgroup$
1
$\begingroup$

You could perhaps use the fda library, and once you have estimated an appropriate continuous function, you can easily find the places where the second derivative is zero.

FDA CRAN

FDA Intro

Improve this answer
$\endgroup$
1
  • $\begingroup$ Zeros of the first derivative define minima and maxima. I think you mean the second. What is "easily", any way? There is more than one way to differentiate numerically. $\endgroup$
    – Nick Cox
    Commented Nov 19, 2013 at 14:59
0
$\begingroup$

I have received many visits to the blog about the changepoint package (>650 as of 11 Nov 2014), so here is an updated post using CausalImpact. http://r-datameister.blogspot.com/2014/11/causality-in-time-series-look-at-2-r.html

Improve this answer
$\endgroup$
0
$\begingroup$

see https://play128.shinyapps.io/inflection_points/

this will show your data in plot and you can download as pdf or svg enter image description here

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.