1
$\begingroup$

Possible Duplicate:
A reliable measure of series similarity - correlation just doesn't cut it for me

I'm trying to determine a method to compare one particular time series against about 10,000+ reference time series programmatically, and shortlist those reference time series which can be of interest.

The method I was using was Pearson Correlation. For each of the reference time series, I would calculate their correlation coefficients, and then sort the entire list of reference time series in descending order based on the correlation coefficient. I would then visually analyze the top N time series which have the highest correlation coefficients, which should be the best matches to the given time series.

The trouble is that I wasn't getting reliable results. Quite often the series in the top N range didn't visually resemble anything like the given time series. Finally when I read the complete article below I understood why: One can't use correlation alone to determine if two time series are similar.

Anscombe's quartet

Now this is a problem with all matching algorithms which calculate some sort of distance between two time series. For instance, the two groups of time series below can result in the same distance, yet one is obviously a better match than the other.

A => [1, 2, 3, 4, 5, 6, 7, 8,  9]
B1 => [1, 2, 3, 4, 5, 6, 7, 8, 12]
distance = sqrt(0+0+0+0+0+0+0+0+9) = 3
B2 => [0, 3, 2, 5, 4, 7, 6, 9,  8]
distance = sqrt(1+1+1+1+1+1+1+1+1) = 3

So my question is, is there a mathematical formula (like correlation) that can better suit me in these kind of situations? One which does not suffer from the problems mentioned here?

Please feel free to ask for any further clarification or improve the question text if needed. Thanks! =)

$\endgroup$
0

0

Browse other questions tagged or ask your own question.