0
$\begingroup$

I have read some papers on clustering application to detect outliers in the data set. In many places the euclidean distance between any 2 data points is calculated using minimum , maximum , mean and standard deviation of the data unit. Let's say the data set consists of multiple samples each of 1 hour duration. Now 1 hour of data is summarized to minimum , maximum , mean and standard deviation and euclidean distance is calculated on these parameters.
My doubt is what is the significance and rationale behind selecting the above 4 descriptive statistical parameters for subsequent use in clustering application.

$\endgroup$
4
  • 1
    $\begingroup$ Could you add a reference? This practice is non standard, though to be clear any distance metric can be used $\endgroup$
    – Harsh
    Feb 20, 2016 at 18:41
  • $\begingroup$ I have never heard of using min, max, mean & SD to calculate Euclidean distance, & I cannot see how it is possible. The Euclidean distance between 2 points is the square root of the sum of the squared distances on each dimension. Can you say more about your situation & your data? $\endgroup$ Feb 20, 2016 at 19:19
  • $\begingroup$ @Harsh and @gung It was used like d(X,Y) = sqrt( (avgx - avgy )^2 + (stdx - stdy )^2 + (maxx - maxy )^2 + (minx - miny )^2 $\endgroup$
    – Soumajit
    Feb 21, 2016 at 2:54
  • $\begingroup$ @Harsh [ arc.aiaa.org/doi/book/10.2514/MSPOPS12 ] Here you can find the paper "New Telemetry Monitoring Paradigm with Novelty Detection" $\endgroup$
    – Soumajit
    Feb 21, 2016 at 3:03

1 Answer 1

1
$\begingroup$

Sure. You can also add other moments such as kurtosis and skew. In essence, this is the approach to clustering time series suggested by Rob Hyndman in this paper, Dimension Reduction for Clustering Time Series Using Global Characteristics where he develops the rationale.

http://www.robjhyndman.com/papers/wang2.pdf

$\endgroup$
2
  • $\begingroup$ @Johnson Thanks for the reference. But the rationale for selecting "min,max,mean and std deviation" is not mentioned in the paper. $\endgroup$
    – Soumajit
    Feb 21, 2016 at 3:06
  • $\begingroup$ "Moments" refer to key statistics of any distribution, even if the metrics you're focused on aren't explicitly mentioned. $\endgroup$
    – user78229
    Feb 21, 2016 at 11:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.