6
$\begingroup$

I have time-series data of different houses. Assume it is power consumption data. Now, I want to cluster the houses following similar power consumption pattern utmost. So, the various distance metrics I can think of to measure the similarity include:

  • Euclidean distance
  • DTW distance
  • Frechet distance

With Euclidean distance, I found an outlier in one of the series leads to a huge difference. So, I do not want to use Euclidean distance in my case.

With DTW distance, I found that it tries to map the similar patterns/shapes first in given two series, and then computes the similarity between two series. I do not want to use this because I do not want to shift the consumption pattern at one instant of time to another in order to match the two input series.

Hand drawn graphs in support to above points are:

Using Euclidean distance enter image description here

Using DTW distance enter image description here

Now my question is:

  1. Which other distance metric is best for my case apart from DTW or Euclidean?
  2. Can you point me to some reference which explains Frechet distance more clearly. I found some papers, but I was not able to get the concept clearly. Does it consider only the corresponding points of two series or like DTW it compares one point of one series with more than one point of another series?

UPDATE: When we compare two series, I think we look from two perspectives:

  1. We consider only the magnitude of two series (i.e, peak value, lowest values, etc). Therefore, if the two series are within same peak values, then we consider two series to be same, otherwise series can be considered different.
  2. We consider only the shape of two series (i.e., try to compare crests and troughs). We do not consider how far or close the two series are in terms of magnitude. This essentially means that although I do not cluster the homes which consume same amount of net power, but I will get homes in a cluster which follow similar pattern (increase/decrease) in power consumption.

I want a similarity metric with respect to perspective number 2. I have summed up both the perspectives in below figure. enter image description here

$\endgroup$
1
$\begingroup$

Answering question 1:

You critic of DTW is met by introducing global constraints to the warping path. This effectively restrains both computational effort (since warping paths which are not allowed do not have to be computed) and prevents pathological warping.

Therefore the answer is: DTW with global constraints

There are several variants of such constraints such as the Sakoe-Chiba band and the Itakura Parallelogram as you can see in the following image. The image originates from a presentation, which is available online in a presentation done by Chotirat Ratanamahatana and Eamonn Keogh.

enter description here

Another possibly relevant time series distance measure is:

LCSS - Longest Common Subsequence - has been originally developed to analyse string similarity but can also be used for numerical time series.

$\endgroup$
0
$\begingroup$

To most users, this "outlier" is a remarkable difference and should yield a measurable difference.

But compared to an entirely different series, it should still only contribute a little, unless you did not preprocess your data well.

We cannot give you better recommendations, because it is impossible to tell what you want. We don't have your data, and we don't know your problem. In order to figure out how to solve this, you need to formalize your requirements, i.e. what should be similar, what should be different, and what should be more similar than the other. Just complaining that you did not "like" the results of the measures is not enough, you need to be much more precise.

$\endgroup$
  • $\begingroup$ I updated my question with a bit more details clearly. Please let me know If the question is not clear enough. I tried to be precise. Also, can you please tell me about preprocessing of data. Did you mean smoothing or curve fitting? $\endgroup$ – Haroon Rashid Dec 10 '15 at 5:47
  • $\begingroup$ To obtain 2, you may need to preprocess your data, e.g. by centering and standardization, prior to usin e.g. Euclidean. $\endgroup$ – Anony-Mousse Dec 10 '15 at 7:32
  • $\begingroup$ Thanks. and what about the distance metric apart from Euclidean. I can try different metrics, but I do not know how should I decide metric x is better than y $\endgroup$ – Haroon Rashid Dec 10 '15 at 9:48
  • 1
    $\begingroup$ Sure you can. Either by looking at their theory and checking if it aligns with your theory on your data; or if you have labeled data by empirical evaluation. E.g. if you assume your time series are perfectly aligned, then you don't need time warping! Frechet distance is for multivariate series (think of the dog-and-owner model, does that match your problem?) $\endgroup$ – Anony-Mousse Dec 10 '15 at 10:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.