I have read some papers on clustering application to detect outliers in the data set. In many places the euclidean distance between any 2 data points is calculated using minimum , maximum , mean and standard deviation of the data unit. Let's say the data set consists of multiple samples each of 1 hour duration. Now 1 hour of data is summarized to minimum , maximum , mean and standard deviation and euclidean distance is calculated on these parameters.
My doubt is what is the significance and rationale behind selecting the above 4 descriptive statistical parameters for subsequent use in clustering application.
$\begingroup$
$\endgroup$
4
1 Answer
$\begingroup$
$\endgroup$
2
Sure. You can also add other moments such as kurtosis and skew. In essence, this is the approach to clustering time series suggested by Rob Hyndman in this paper, Dimension Reduction for Clustering Time Series Using Global Characteristics where he develops the rationale.
answered Feb 20, 2016 at 18:45
-
$\begingroup$ @Johnson Thanks for the reference. But the rationale for selecting "
min,max,mean and std deviation" is not mentioned in the paper. $\endgroup$– SoumajitCommented Feb 21, 2016 at 3:06 -
$\begingroup$ "Moments" refer to key statistics of any distribution, even if the metrics you're focused on aren't explicitly mentioned. $\endgroup$ Commented Feb 21, 2016 at 11:58
d(X,Y) = sqrt( (avgx - avgy )^2 + (stdx - stdy )^2 + (maxx - maxy )^2 + (minx - miny )^2$\endgroup$