I am looking for references (as I prefer to sit over the giant's shoulders...) to something it "seems" to work well...

When we do clustering to analyse some data, to understand its structure (in my case to understand people's preferences expressed in some survey), it is somehow like doing dimensionality reduction over a discrete space, given by the number of classes.

But I can also run an Autoencoder with a single dimension in output and print the density of the resulted encoded data. I can then see how the people surveys answers group together. In my study I retrieve a density that resembles very much the density of the cluster assignations, with a bit more "details" being the encoding over a continuous space....

Is this a death salmon kind of coincidence, or does it make sense to you ?

EDIT: I have tried it on the Iris dataset and got the same.. the density of the 1-D autoencoder output shows the presence of one modal clearly separated and the other two modals a bit merged between them, that is indeed the structure of the Iris dataset...

EDIT2: For the Iris dataset, I have associated the autoencoder output with the true classes (species). This is the outcomes.. as you can see the 1-d autoencoder is able to sort the classes very well, altought it is not evident the split between the virginica and vescicolor.

autoencoder output histogram

Code (Julia):

using DelimitedFiles, StatsPlots, BetaML

iris     = readdlm(joinpath(dirname(Base.find_package("BetaML")),"..","test","data","iris_shuffled.csv"),',',skipstart=1)
x        = convert(Array{Float64,2}, iris[:,1:4])
y        = convert(Array{String,1}, iris[:,5])
int_map = Dict("setosa"=>1, "virginica"=>2,"versicolor"=>3) 
intclasses = [int_map[i] for i in y]

function generate_bin_colours(data,classes,bins;)
    # Creation of shares per bin
    nbins   = length(bins)
    ndata = length(data)
    shares = [(0.0,0.0,0.0) for i in 1:nbins-1]
    for ib in  1:nbins-1
        ndata_per_bean = 0
        sums_rgb  = [0,0,0]
        bin_l = bins[ib]
        bin_u = bins[ib+1]
        for id in 1:ndata
            if data[id] >= bin_l &&  data[id] < bin_u
                sums_rgb[classes[id]] += 1
                ndata_per_bean += 1
        share = ndata_per_bean > 0 ? sums_rgb ./ ndata_per_bean : [1/3,1/3,1/3]
        shares[ib] = (share...,)
    return [RGB(x...) for x in shares]

ae       = AutoEncoder(encoded_size=1, epochs=300)
y        = fit!(ae,x)
idx      = sortperm(y[:,1])
ysorted  = y[idx]
clsorted = intclasses[idx]

bins = -4:0.1:2
histogram(fit!(Scaler(),ysorted),bins=bins,color=generate_bin_colours(fit!(Scaler(),ysorted),clsorted,bins),label="r: setosa\ng: virginica\nb:versicolor", title="1-D autoencoded data density")
  • $\begingroup$ Of course there are lots of clustering methods, so whoever wants to publish clustering by using this kind of autoencoder would need to show that it has advantages over whatever is already in the literature for clustering. I'm not sure that this is the case (and if so it will certainly depend on how exactly the autoencoder is set up), even though, as you correctly show, it may work well in some cases. But then so will lots of other existing clustering methods. $\endgroup$ Apr 12 at 12:45
  • $\begingroup$ @ChristianHennig But the difference is that (whatever) clustering method maps to a discrete space, while AE has the advantage to be a discriminant method that maps to a continuous space $\endgroup$
    – Antonello
    Apr 12 at 13:35
  • $\begingroup$ But clustering is essentially discrete and your question is about using the autoencoder for clustering!? $\endgroup$ Apr 12 at 14:49
  • $\begingroup$ Or maybe your suggestion is rather to first run the autoencoder and then run clustering on its output? In which case you'd use the autoencoder as dimension reduction technique before clustering and it'd have to compete with the likes of PCA, UMAP, t-SNE? $\endgroup$ Apr 12 at 14:53


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.