2
$\begingroup$

I have used a clustering algorithm (ClusterONE) that uses 'cohesiveness of nodes' as a measure to find protein complexes in a PPI network. http://www.paccanarolab.org/clusterone/ Since I do not have a gold standard dataset to validate the results from this algorithm, I believe I should be using internal validation indices such as Dunn's index or Davies-Bouldin index or Silhouette value? However, these methods require a distance matrix or a similarity matrix to calculate. Are these internal validation indices applicable in my case considering that the algorithm used does not use a distance or similarity matrix? The input matrix that this algorithm uses for clustering is in the format:

Protein ID1     ProteinID2      Confidence Score for the interaction

Please correct me if I am wrong here in understanding any of the concepts.

Edited - my R code for calculating Dunn index:

df<-
        V1    V2   V3
1  ATP4A_HUMAN  1313 0.63
2  PR40A_HUMAN  6060 0.67
3  PR40A_HUMAN  6066 0.67
4  HOME1_HUMAN  7221 0.72
5   PTN6_HUMAN 10748 0.55
6   SYSC_HUMAN 23437 0.65
7  CAND1_HUMAN 26781 0.63
8   CSN5_HUMAN 26781 0.63
9   CUL1_HUMAN 26781 0.63
10   PAF_HUMAN 26781 0.63

library(reshape2)
library(clValid)

distmat <- acast(df, V1 ~ V2, value.var='V3', fun.aggregate = sum, margins=FALSE)
dismat <- as.dist(distmat)
nc <- 3056 # number of clusters by ClusterONE

clusterObj <- hclust(dismat,method = "average")
cluster <- cutree(clusterObj,nc)
dunn(dismat, cluster)
[1] NaN
Improve this question
$\endgroup$
6
  • $\begingroup$ If you could get the original feature data used for determining the confidence score you would be set, however, as you stated -- no such luck. However, you could generate a distance matrix based on all possible pairs of ID1-ID2 based on the scores -- and then try to twist the results into a hierarchical clustering heat map. Generate a heat map of the score matrix, and you may be able to infer patterns from it -- this is done all the time. $\endgroup$
    – user32398
    Dec 18, 2016 at 4:54
  • $\begingroup$ @ wrtsvkrfm Thank you for the response. I generated a distance matrix using the confidence scores and used the clValid R package to find the Dunn index using the hclust method. But I am getting 'NaN' as an answer. What does this mean ? My code is given above in the original post as an edit: $\endgroup$
    – DataStudent
    Dec 18, 2016 at 8:53
  • $\begingroup$ Davies-Bouldin can be computed both from case-by-variables data as well as from distance matrix. the other two require distance matrix. You say you have dataset and not distance matrix and your cluster analysis processes data, not distances. Well, many clustering algorithms are still implicitly based on some distance dispite they don't process them (see e.g.). Can ClusterONE be such one and the distance implied could be deduced and computed? $\endgroup$
    – ttnphns
    Dec 18, 2016 at 9:28
  • $\begingroup$ Because ClusterOne isn't very well known I might recommend you to read about its algorithm or to ask the authors directly "which distance is implied, if any". $\endgroup$
    – ttnphns
    Dec 18, 2016 at 9:35
  • $\begingroup$ NaN means "not a number" which could be occurring because of a zero score. The algorithm you fed the matrix to may not expect any zeroes in matrix elements. Find out in the description of the program if the matrix needs to be "positive definite", or "can't be singular". $\endgroup$
    – user32398
    Dec 18, 2016 at 12:24

1 Answer 1

Reset to default
1
$\begingroup$

The confidence score is a similarity, isn't it?

So why can't you use the standard indices?

What's the problem with your similarity matrix that makes them fail (and beware that similarity != distance).

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Thanks for the response. The confidence score is the value of 'strength' of that particular protein interaction. The range of the confidence score lies between 0 and 1, with 1 being a high confidence (strength) interaction. I now realize that maybe calculating a distance matrix was not the right approach. But I am still not sure how to depict this relationship through a matrix that can be used an input for calculating any internal clustering validation index. Any ideas will be of great help. $\endgroup$
    – DataStudent
    Dec 19, 2016 at 8:15
  • $\begingroup$ Well, you can try simply using 1-sim as distance... $\endgroup$
    – Has QUIT--Anony-Mousse
    Dec 19, 2016 at 21:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.