I am trying choose best $k$ from the consensus clustering using the Cophenetic Correlation Coefficient (CCC). I tried as follows. The correlation coefficients values are poor, i.e., k=2 (0.2110048)
, k=3 (0.1934558)
, k=4 (0.175295)
. Please suggest whether am following correct method.
# consensus clustering
library(Biobase)
data(geneData)
d = geneData
# median center genes
dc = sweep(d, 1, apply(d,1,median))
rcc = ConsensusClusterPlus(dc, maxK=4, reps=100, pItem=0.8, pFeature=1, title="example",
distance="pearson", innerLinkage="ward.D",
finalLinkage="ward.D", clusterAlg="hc")
# Cophenetic Correlation Coefficient
k2 <- rcc[[2]]$consensusMatrix
d1 <- as.dist(t(k2))
hc <- hclust(d1, "ward.D")
d2 <- cophenetic(hc)
cor(d1, d2)
# 0.2110048
k3 <- rcc[[3]]$consensusMatrix
d1 <- as.dist(t(k3))
hc <- hclust(d1, "ward.D")
d2 <- cophenetic(hc)
cor(d1, d2)
# 0.1934558
k4 <- rcc[[4]]$consensusMatrix
d1 <- as.dist(t(k4))
hc <- hclust(d1, "ward.D")
d2 <- cophenetic(hc)
cor(d1, d2)
# 0.175295