# Smoothing algorithm for irregular time interval

I have various sets of irregular interval time series data to which I want to apply some sort of smoothing algorithm to produce a good fit.

I have attempted various methods which all were unsatisfactory.

1. Loess - Too much of a tendency to overshoot/overreact to outliers
2. Moving Average - The lag is unacceptable

I have read about the "Improved Holt Method for Irregular Time Series", but the paper was too difficult for me to understand and implement in C#.

Can someone point me to a good method / algorithm which produces good smoothing?

The method must be able to calculate the smoothed point at time $t$, without requiring $t+1$, etc., data. It also must be capable of dealing with multiple $y$ values for a given $x$ time.

• With loess, you can adjust the amount of smoothing, did you try different amounts? – Peter Flom Apr 20 '13 at 14:12
• yes, but the overshooting problem persists if a sudden dip or change in trend occurs. Too large a smoothing factor or window creates lag similar to MA. – Ying Apr 20 '13 at 14:16
• Seems like what you want is to use any typical smoothing but just disable it for large steps. If you do not do that it looks like you will always a get lag. – Bitwise Apr 20 '13 at 15:04

The simplest algorithm is the median filter. You can find an C++ implementation in the R package robfilter. That implementation also include an 'online' version that only uses past data and implements some algorithmic short-cuts.

Of course you will still have to set the "width" argument yourself, but this is the counter part of looking for a simple algorithm (this package also contains more sophisticated smoothing algorithms).

The median-filter is essentially a rolling window median, so it inherits the good behaviour of the median in terms of insensitivity to outliers and non-parametric interpret-ability.

So, considering the dataset you posted, the median filter would yield:

and the code:

a1<-read.table("sodat.txt",header=TRUE)
library("robfilter")
d1<-med.filter(a1[,2],width=10,online=TRUE)
plot(d1)


I think what you want is a median filter: "Median filtering is very widely used in digital image processing because, under certain conditions, it preserves edges while removing noise".