Detecting outliers in non-linear data

Question

Is there a way to detect outliers in data that would fit a non-linear shape? For example, I have data that fits an exponential decay with an obvious outlier. I have simulated data with an example of this:

I want to know the index locations of where these outliers occur in the real data and currently I am doing this by applying an exponential decay fit to the data then finding the residual of each point the fit. I then find if any points go over a certain threshold that I find as some multiple of the standard deviation of the residuals and find any point with a residual over that threshold. Is there a better way to find outliers in a model like this?

Here are the points used in the above array:

 [10., 8.46481725, 7.16531311, 6.0653066,
 5.13417119, 4.34598209, 3.67879441, 3.11403224,
 2.63597138, 2.2313016 , 1.88875603, 1.59879746,
 1.35335283, 1.14558844, 0.96971968, 0.82084999,
 0.69483451, 0.58816472, 25, 0.42143844]

Answer 1

Residuals can be useful, but there are a few things to watch out for.

Your data might be heteroskedastic, i.e. the magnitude of residuals varies with the x-value. In this case, what counts as a "large" residual might depend on the x-value. (This is an issue even with linear relationships.)

You might also need to think about the potential for errors in the measurement of the x-value. For example, if y=2x, mismeasuring x by 1 will always cause an error of 2 in your prediction of y. But if y=2^x, mismeasuring x by 1 can cause very small or very large errors in predicting y, depending on what x is. Strictly speaking this isn't heteroskedasticity, but the results look very similar.

One common trick is transformation - either of the whole function, or of the residuals. In the example you give, I'd consider a log transformation, which will convert your exponential relationship to a linear one; depending on the sort of errors you encounter, it may or may not remove heteroskedasticity.

You can also look at neighbour-based ways to identify outliers. For example, for any given data point P, pick the closest n neighbours to the left and to the right, characterise the range of y-values in that neighbourhood, and then check whether P is consistent with that range.

...and there are many other methods, all the way from "eyeball it on a scatter plot" up to elaborate machine-learning methods like neural nets and SVMs. It's hard to give a definitive answer to your question since it depends on so many factors.