# Automated detection of outliers in one dimensional data

I have several datasets that I need to be able to fit (the goal is to find the outliers). The datasets were created by groups of images and the x is an index number of the image and y is a focus measure of an image.

When I am plotting x vs y(focus measure) every set looks like the trend has a different shape (it could be 2 peaks (Gaussians or Lorentzians) or one peak (Gaussian) because of the structure of the image (these are complicated images and they have different structures which accounts for the differences). When I look at the plots there are several points that are very far away from the main trend, it is very easy to see them by eye, but what I want is to fit the data so that I can find the data points (using a program) that are further away than 3 standard deviations from the fit line. Because there are thousands of those datasets, I can not look at them one at a time and decide which curve shape fits the best, I need a more generic way to fit the data. I am using Matlab to do that.

Is there a way to find the best curve fit programmatically without knowing the shape of the curve? I am not very familiar with Machine Learning but this might be something that I can do using ML but I am hoping of a suggestion of a specific method to do that.

• Here's an approach you may find useful: stats.stackexchange.com/questions/151659/…. May 15, 2015 at 1:24
• It is unclear what you are trying to do. Are you fitting univariate distributions? Or are you performing nonlinear regression on paired data and looking for outlying residuals?
– whuber
May 15, 2015 at 17:49
• @whuber:i added a more detailed explanation
– AL B
May 15, 2015 at 18:07
• Thank you. I am still as mystified as before, however, because your description sounds like it's simultaneously several different things. Could you explain what you mean by "all in a straight line" and elaborate on this "structure"? In what sense can a sequence of "focus measures" look like it has "peaks"? Your definition of "outlier" is nonstandard, too: it is equivalent to what most people call the "minimum." This is confusing.
– whuber
May 15, 2015 at 18:13
• @whuber: When I am plotting x vs y(focus measure) every set looks different (it could be 2 peaks (Gaussian or Lorentzian) or one peak (Gaussian) because of the structure of the image (these are complicated images and they have different structures which accounts for the differences). When i look at the plots there are several points that are very far away from the main distribution, it is very easy to see them by eye. I want to fit the data so that I can find the data points (using a program) that are further away than 3 sigma from the fit line.
– AL B
May 15, 2015 at 18:31

You will need to use some kind of non-parametric scatter plot smoothing to do this. There are many available algorithms, loess, kernel smoothers, smoothing splines, etc. I'll use smoothing splines because they are amenable to your second request for confidence bands. It may be possible to use other smoothing algorithms though.

Note: I apologize that I cannot offer a solution in Matlab, as I do not have access to that software. I will explain the procedure in R, and hopefully you can translate it into Matlab.

Here's the step by step:

• Fit a smoothing spline to your $x, y$ data, and let it choose the appropriate smoothing parameter by cross validation. The matlab documentation for this kind of fit is here. The R documentation is here.
• Calculate the confidence bands around the smoothing spline.
• Use these confidence bands to detect your outliers.

First, I created a signal that seems like what you're describing:

sig <- function(x) {x - 2*x^2 - 1*exp(-(x-.5)^2/.05)}


Next I generated some data. $95$ observations are tightly varied around the signal, $5$ are given much higher variance, these are the outliers I'm attempting to identify:

x <- runif(n=95, min=0, max=1)  # Tight data
y <- sig(x) + rnorm(95, 0, .1)
xn <- runif(n=5, min=0, max=1)  # Outlier data
yn <- sig(xn) + rnorm(5, 0, 1)


I concatenated these two sets to give an approximation of what you're seeing for your data sets:

x <- c(x, xn); y <- c(y, yn)


Next I fit a smooth.spline to the data. This fit comes with a smoothing parameter $\lambda$ that controls how closely you want the spline to fit the observed data - between fitting a trend line at one extreme, to interpolating the data completely at the other extreme. The default behavior of smooth.spline in R is to determine a best smoothing level $\lambda$ by cross validation, I'm just going to let it do that:

library(splines)
sp <- smooth.spline(x, y)


Now we need to determine the confidence bands. The mathematical details of how to do this with smoothing splines takes some careful study. Practically, I followed this answer:

res <- (sp$yin - sp$y)/(1-sp$lev) # Get jacknife residual at each data point sigma <- sqrt(var(res)) # Calculate stddev of jackknife residuals upper <- sp$y + 5.0*sigma*sqrt(sp$lev) # Create confidence bands lower <- sp$y - 5.0*sigma*sqrt(sp$lev)  You can see, I increased it to a$5\sigma$band, you can tune this parameter to your liking. Plotting the bands gives a a good indication we're on the right track: Now I can attempt to detect my outliers by finding which data points lie outside of these confidence bands: is_outlier <- sp$yin > upper | sp$yin < lower xoutliers <- sp$x[is_outlier]
youtliers <- sp\$yin[is_outlier]


Plotting these along with the fit and the confidence bands shows that we got it right:

• Thank you very much! This is exactly what I was looking for. I am learning R right now, so it will not be a problem to translate the solution.
– AL B
May 17, 2015 at 5:39