0
$\begingroup$

I have a time-series of a feature(metric) for 4 different servers each of length 2000. I want to use dbscan algorithm to figure out if all 4 machines fall in the same cluster or not using dbcscan on these 4 time-series.

I am using the dbscan package in R and my input is a 4 x 2000 matrix(inputMatrix) to the dbscan function. To determine the parameters I am determining the value of k/minpts as follows.

Calculation of k: 1.) There are 2000 points and 4 rows. Considering one column at a time, I am calculating the distance of each point from the remaining three points and then taking the mean. So this gives me 4 avg distances corresponding to 4 servers/rows at a particular time. So I again have a 4 x 2000 matrix of distances(distMatrix).

distmat<-function(x){
#each column of distance is the distances of each server with other servers.
distance<-as.matrix(dist(x = x,method = "euclidean",diag=T,upper=T))
return(apply(X = distance,MARGIN = 1,FUN = mean))
}

distMatrix<-apply(X = inputMatrix,MARGIN = 2,FUN = distmat)

2.) With each point as a center in the inputMatrix and corresponding avg dist in distMatrix as radius I calculated the maximum number of points that lie in the neighbourhood.

numberofpoints<-matrix(data = rep(x = 0,8000),nrow = 4,ncol = 2000)
for(i in 1:ncol(inputMatrix)){
    for(j in 1:nrow(inputMatrix)){
        numberofpoints[j,i]=length(which(inputMatrix[,i]<=inputMatrix[j,i]+distMatrix[j,i] & inputMatrix[,i]>=inputMatrix[j,i]-distMatrix[j,i]))
    }
}

Again taking a mean over the column first and then over the row yields the value of k/minpts.

meannumberofpoints<-apply(X = numberofpoints,MARGIN = 2,FUN = mean)
k=mean(meannumberofpoints)

k for my data is 2.167125

To find EPS: There is an inbuilt kNNdistplot function in dbscan package in R which plots the knee-like graph.

kNNdistplot

The horizontal line across the image corresponds to the eps value. However, I am not sure what variables it is plotting on the two axes. I want to automate this sorted k-graph calculation and plot it but I am not sure where to start.

Can anyone please explain what are the variables/values plotted on the x and y axis and how to calculate these. Thanks.

$\endgroup$
2
$\begingroup$

Something is very messed up here.

The usual approach would be to consider the four servers to be four "points". But at data this tiny DBSCAN does not make sense!

I'd suggest to use a time series distance function, and hierarchical clustering.

It appear that your approach is to treat the data as 2000 data sets, each with 4 points with 1 attribute each. But what magic do you expect DBSCAN to do there? And how did you manage ro end up with a non-integer k? What is the 2.12345 nearest neighbor supposed to be?!?

$\endgroup$
  • $\begingroup$ So my approach is wrong. I won't argue on this as I was experimenting. Now I know where I am going wrong. Do you think it would help if I use the entire 8000 datapoints as one data set and calculate my parameters based on the distance of every point from the remaining 7999 points? $\endgroup$ – tushaR Feb 25 '16 at 6:32
  • $\begingroup$ I think you first need to be clear about what question you are asking. I don't even understand what parameter you are trying to estimate, nor how you mean to use it. $\endgroup$ – Anony-Mousse Feb 25 '16 at 9:26
  • $\begingroup$ I am trying to calculate the parameters based on the algorithm provided in this paper: ijiset.com/v1s4/IJISET_V1_I4_48.pdf $\endgroup$ – tushaR Feb 25 '16 at 9:39
  • $\begingroup$ To do what? I don't see how you could use DBSCAN to solve anything on your data. Plus, that article is in a low reputation indian spam journal... I would not bother to even read it. $\endgroup$ – Anony-Mousse Feb 25 '16 at 9:56
  • $\begingroup$ Okay. My input is 4 time series of the same feature corresponding to 4 servers. The length of each time series is 2000. The objective is to determine that time-series of which servers fall in the same clusters using these series. Can we attain the objective using DBSCAN ? $\endgroup$ – tushaR Feb 25 '16 at 11:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.