0
$\begingroup$

I am building an anomaly model and am confused between these distances below. What is the difference between these distances in self organized maps.

  1. som.iris$distances

  2. dist(som.iris$codes[[1]])

As far as I have read, 1) is between vector to Best Matching Unit, and 2) is the euclidean distance between adjacent neurons in the som topology (also seen in U-matrix) . When do i use 1. and when do i use 2.? How are they associated with each other? is 1. a more granular version of 2.? I'm interested to assign a score based on how 'different' one is against another.

thanks.

library('kohonen')
set.seed(1)

train <- iris
# --------- unsupervised Training - Train Model ------------
#preprocess
train.sc <- scale(train[,-5])

#train model
som_grid <- somgrid(xdim = 5
                    ,ydim=5
                    ,topo="hexagonal"
                    ,toroidal = F)  
som.iris<- som(train.sc
               ,grid=som_grid
               ,rlen=200
               ,alpha=c(0.05,0.01)
               ,keep.data = TRUE )

#different distances
#cells in descending order based on distance between vector and BMU 
head(som.iris$unit.classif[order(som.iris$distances,decreasing=T)],20)
#cells in descending order between adjacent neurons in som map
head(hclust(dist(som.iris$codes[[1]]))$order,20)
$\endgroup$

1 Answer 1

2
$\begingroup$

Sharing the research done to-date. Would love to hear your thoughts if I am on the right track.

  1. som.iris$distances defined as the mean distance of objects to the codebook vectors. This would be a good indicator on how 'similar' the object is against the code units it is assigned to. An anomaly is when the distance is greater relative to the others.

  2. dist(som.iris$codes[1]) is used primarily for hclust and the order is in the sequence of codes assigned based on the similarity between one to another.

To visualize 1. and 2. i have prepared a plot which describes the relationship of codes against the data points, and the measured dissimilarity when assigned against the codebook vectors.

As seen below, vector 16 has an object so dissimilar against other 16 objects, therefore the mean distance is much higher.

enter image description here

library('kohonen')
set.seed(1)

train <- iris

#preprocess
train.sc <- list(x=scale(train[,-5]),y=train[,5])

#train model
som_grid <- somgrid(xdim = 5
                    ,ydim=4
                    ,topo="hexagonal"
                    ,toroidal = F)
#unsupervised
som.iris<- supersom(train.sc
               ,grid=som_grid
               ,rlen=200
               ,alpha=c(0.05,0.01)
               ,keep.data = TRUE
               ,whatmap='x')

set_cluster <-3
dist_hc <-hclust(dist(som.iris$codes[[1]]))

## use hierarchical clustering to cluster the codebook vectors
som.iris.hc <- cutree(hclust(object.distances(som.iris, "codes")), set_cluster)
train_final <- cbind(iris,
                     cluster=som.iris.hc[som.iris$unit.classif], #assign clusters
                     dist=round(som.iris$distances,2),#vector to BMU level
                     cell_unit=som.iris$unit.classif)

table(train_final$Species,train_final$cluster)

library("plot3D")
# x, y and z coordinates
x <- sep.l <- train_final$Sepal.Length
y <- pet.l <- train_final$Petal.Length
z <- sep.w <- train_final$Sepal.Width
c <- clus <- train_final$cluster

# png("./charts/plots.png", width = 1920, height = 720, units = 'px', res = 120)
par(mfrow = c(1, 3))
plot(hclust(dist(som.iris$codes[[1]])),main='obj dist codes')
with(train_final, text3D(Sepal.Length, Petal.Length, Sepal.Width,
                         labels = som.iris$unit.classif, colvar = train_final$dist,
                         col = gg.col(100), theta = 45, phi = 0,
                         xlab = "Sepal.Length", ylab = "Petal.Length", zlab = "Sepal.Width",
                         main = "SOM Iris by Cell Grids", cex = 1.2,
                         bty = "g", ticktype = "detailed", d = 2,
                         clab = 'Vector to BMU Dist.', adj = 0.5, font = 2))
scatter3D(x, y, z, 
          clab = c("Vector to BMU Dist."),
          cex=1.2, d = 2,phi = 0, adj = 0.5,
          bty ="g",
          pch =c,
          colvar = train_final$dist,main='SOM Iris by 3 cluster', 
          theta = 45, col = gg.col(100),ticktype = "detailed")
par(mfrow = c(1, 1))
$\endgroup$
1
  • $\begingroup$ Great answer! I find its harder to make individual variable plots using supersom vs regular som in R: e.g. (this doesn't work with supersom - from : rpubs.com/loveb/som) var <- 1 #define the variable to plot plot(iris.som, type = "property", property = getCodes(iris.som)[,var], main=colnames(getCodes(iris.som))[var], palette.name=terrain.colors) $\endgroup$
    – stats_noob
    Jan 18, 2021 at 8:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.