0
$\begingroup$

I'm using DTW as a distance measure for comparing two multivariate time-series. I want to be able to cluster data using DTW as distance measure, since time-series may be shifted, skewed.

Since there are a couple of parameters I should normalize the series so that all the parameters have the same influence when trying to determine whether time-series are similar. I'm using Euclidean distance as local distance for DTW.

My question is - how to determine whether I should use normalization (subtract min and divide by max) or standardization (subtract mean and divide by standard deviation)?

Moreover, can anyone explain to me what is the point with standardization? I understand that it can help me determine how many standard deviations are values far from their mean, but why would that improve my similarity measure when comparing two time-series?

I'm not a statistician, so any explanation would be great. I understand that normalization would give me values in range [0,1] so that all parameters have values in same range, but what will I get by standardization?

Finally, should I divide each time-series by the standard deviation of the whole dataset, or only by standard deviation of the time-series I'm standardizing?

I must also emphasize that my data does not belong to a normal distribution.

$\endgroup$
  • $\begingroup$ Normalization in your sense would appear to be (value $-$ min) / (max $-$ min), not as described. $\endgroup$ – Nick Cox Apr 22 '19 at 9:13
1
$\begingroup$

I recommend reading http://www.dataminingblog.com/standardization-vs-normalization/

A couple of points, I usually prefer standardization over normalization since normalization suffers even more from outliers. However since you stated that your data is not normal, I would recommend trying (X-median)/IQR or winsoring (clipping the top and bottom 0.1%) before doing standardization.

If you know the distribution of your data, you could use the appropriate functions. For example, geometric mean for log-normal.

As always, it is best to test the methods to see which one performs best.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.