Does the Gaussian Mixture model require binary and multiclass response/target variable (classification), or can the target vector consist of all real numbers (continuous variable, regression)? Why is it called an unsupervised learning algorithm if it trains on target observations anyway? Or is that not a requirement and it can run without labels?
Here is a vignette on Gaussian Mixtures in R:
Here is the example R code from the "Gaussian Mixtures" library for a Gaussian Mixture Model, note in particular the lack of labels and the presence of pre-specified number of cluster components (4) and the specification that all variances are equal "V":
#load the data library(MASS) data(galaxies) X = galaxies / 1000 #this is the gmm library library(mclust, quietly=TRUE) #fit to a cluster fit = Mclust(X, G=4, model="V") #make summary of fit summary(fit)
This is the output of the code:
> summary(fit) --------------------------------------------------- Gaussian finite mixture model fitted by EM algorithm --------------------------------------------------- Mclust V (univariate, unequal variance) model with 4 components: log-likelihood n df BIC ICL -199.2545 82 11 -446.9829 -466.264 Clustering table: 1 2 3 4 7 35 32 8 >
The output of this is a model, in particular it is a set of parameters that can go into an analytic (aka symbolic) relationship.
Here is how we get the list of parameters
Here is the list of parameters:
$pro  0.08440664 0.38615546 0.36963554 0.15980236 $mean 1 2 3 4 9.707481 19.804456 22.877670 24.435158 $variance $variance$modelName  "V" $variance$d  1 $variance$G  4 $variance$sigmasq  0.1772940 0.4353339 1.2526260 34.1224385 $variance$scale  0.1772940 0.4353339 1.2526260 34.1224385
It, as in the parameter list, can be applied in a number of ways. It can be applied forward, backward, and several versions of sideways. Think of it as a collection of bubble-surfaces in a space. (bubble surface = surface of constant probability density)
Some things you could look at:
- in vs. out for any single bubble
- in vs. out for the collection of bubbles
- distance from bubble center for any single bubble
- distance from aggregated center for any permutation of bubbles
- are there too many or too few bubbles - how to get N correct
- is the space really a membrane, can you use PCA to reduce your bubble dimensionality
- do you have outlier bubbles where most are connected and some aren't
- are your bubble diameters close enough to equal to consider them equal (kmeans)
- how to the parameters vary when you repeat the process (is your fit robust)
- do you have pathological points
- do you have pathological bubbles
- how much of the volume of the convex hull of the space is shared by 2 or more bubbles
- what does the set of planes of equal probability of cluster membership look like (classification boundaries)
Consider this image, and imagine it was drawn better. Each family of 10 or so dots is really its own Gaussian distribution. It has its own mean and covariance matrix.
Together these 14 individual distributions in a 2d space can make a single Gaussian Mixture model. You weight each individual component by which fraction of total points it contains, and you would get about 1/14 for your average weight.
If you had a bigger space, and more time, you could make a 2d space with any number of individual Gaussian distributions, each having their own mean and covariance. You could then create a single Gaussian Mixture from all of that arbitrary number.
Check the image here out: