I have a noisy readout of a curve that is monotonically increasing or decreasing for a narrow range of points and then quickly saturates. I don't know exactly where the saturation point is, but from the underlying (biological) process I know that it exists and once it's reached, strong deviations from a flat line are very likely to be outliers.
I am looking for the best way to smooth these data. Something like this (the data is actually not for time series, but it's a good analogy):
So far I have tried using:
- the Hampel filter
- loess regression (including its robust version, family="symmetric" in R)
- a combination of both
The problem with loess is that since the non-saturated bit is so small, loess tends to "underfit" it, "dragging" the trendline up or down towards the saturation point. The problem with the Hampel filter is that while it does a great job in filtering out strong outliers, it's not that great for smoothing as such, i.e., polishing out small deviations from the trendline.
It seems like the best thing would be to find the saturation point and then fit two separate models "before" and "after" it. This would be straightforward without the noise, but I'm not sure how to do it in a noisy setting. I'm also open to any other suggestions.
This is part of a large project in R, so if a suggested algorithm has an R implementation this would be particularly great.