3
$\begingroup$

I am working on clustering some time series and have decided to try to cluster based on features rather than the raw series (for now). I found this paper by Hyndman that chooses a series of features (serial correlation, non-linearity, skewness, kurtosis, periodicity) and discusses some of the algorithms used for clustering.

At one point he mentions the need to bring the features to [0,1] and he proposes 3 different normalization methods. The first two:

  • $f1$ mapping Q $\in$ [0,$\infty$) to q $\in$ [0,1]: $q = \frac{e^{aQ}-1}{b+e^{aQ}}$

  • $f2$ mapping $\in$ [0,1] to q $\in$ [0,1]: $q = \frac{(e^{aQ}-1)(b+e^{a})}{(b+e^{aQ})(e^a -1)}$

For $f1$ and $f2$ the constants a and b are chosen "such that $q$ satisfies the conditions: $q$ has 90th percentile of 0.10 when $Y_t$ (the time series) is standard normal white noise, and $q$ has value of 0.9 for a well-known benchmark dataset with the required feature."

For each feature, and depending on whether the original values lie in the [0,$\infty$) or [0,1] range, Hyndman gives the values of $a$ and $b$.

I am having trouble understanding this normalization technique. Why would we want a 90th percentile of 0.1 fo $q$? Are there any reasons why we can't simply use z-normalization? Is this a common and standard procedure?

Any hints, tips, insights, and links for further reading are greatly appreciated. Thanks!

$\endgroup$
3
  • $\begingroup$ 1) z-normalization would not result in target range of [0,1]. 2) q set to the 90th percentile only for a time series of standard white noise - thereby effectively asking that 90% of standard white noise results in an influence of less than 0.1 for q. 3) It is not a common precodure (to my knowledge) $\endgroup$ Nov 18, 2016 at 15:46
  • $\begingroup$ Thanks for the answer Nikolas. I know that z-normalization won't bring the features to [0,1] but it will bring all values to a comparable scale. Are there any reason why we would want all values to be positive? For 2) I think I understand better now, thanks a lot. $\endgroup$
    – tomasn4a
    Nov 18, 2016 at 15:57
  • $\begingroup$ z-Transformation does not result in a limited scale at all. Any values are possible yet unprobable. For possible processing it can be valuable to have all data only within a certain limit (knowing both max and min). Therefore z-normalization is not done here. $\endgroup$ Nov 18, 2016 at 15:59

1 Answer 1

2
$\begingroup$

1) z-normalization would not result in target range of [0,1]. z-Transformation does not result in a limited scale at all. Any values are possible yet unprobable. For possible processing it can be valuable to have all data only within a certain limit (knowing both max and min). Therefore z-normalization is not done here.

2) q set to the 90th percentile only for a time series of standard white noise - thereby effectively asking that 90% of standard white noise results in an influence of less than 0.1 for q.

3) It is not a common precodure (to my knowledge)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.