# Determining characteristics of peaks after mclust finite mixture model

I'm working with the mclust package in R (specifically using densityMclust). As output, I have a file with mixing probabilities, variances, and means for each normal distribution. The general format is:

foo prob1 var1 mean1 prob2 var2 mean2...
foo2 prob1 var1 mean1 prob2 var2 mean2...


for as many normal distributions as I can force.

This is all I have; I don't even have access to the actual raw data. My goal is to identify relative maximas (peaks). Would I go about doing this by finding the largest means?

I have a few more questions:

1. I'm having trouble understanding what exactly a mixing probability is. If anyone could please direct me to a resource that I couldn't find, I'd really appreciate it.

2. Since I only have the 3 data types listed above, how would I find standard deviation values without the range of raw data values?

3. And also any better description for a Gaussian finite mixture model than mine: forces data into multiple bell curves...?

4. Why are these mixture models called unsupervised learning? All the explanations I bumped into online were way too technical for me.

5. As shown in the data above, I often have >2 peaks. However I'm only concerned with the first two. Can I ignore the rest and safely call that part a bimodal distribution?

• Please don't cross-post. – gung - Reinstate Monica Jan 18 '15 at 18:50
• This question isn't very clear. Can you post your data? Even if it isn't the raw data, it would help. What are "foo" & "foo2", outputs from Mclust from 2 different input datasets? What do you want the SDs of, exactly? You are given variances of the clusters. Are you aware that $SD=\sqrt{\rm Var}$? – gung - Reinstate Monica Jan 18 '15 at 18:51
• I've deleted the duplicate post that someone migrated from Stack Overflow. – swetharevanur Jan 18 '15 at 18:58
• I can't post the data because it is proprietary. However I can be a bit more clear. I'm working with gene expression data, so foo and foo2 would be the names of some genes. I've clustered the actual expression data using a GMM and basically right now I have metadata for each subpopulation bell curve. As mentioned in the original question, I have mean, mixing probability, and variance. – swetharevanur Jan 18 '15 at 19:01
• @gung Correct me if I'm wrong - To find the peaks, take the relative maximums of the mean. To find the SD, take the square root of variance. I'm only interested in the first two peaks in expression level for each gene, so I can ignore the rest of them. The first two peaks form a bimodal distribution... And then I have some basic questions. (1) In layman's terms, what is a Gaussian FMM? (2) Why are mixture models called unsupervised learning? (3) In layman's terms, what is a mixing probability? Thanks. – swetharevanur Jan 18 '15 at 19:03

Your questions (reposted so I can be systematic and complete) with my answers inline follow:

Question1: I'm having trouble understanding what exactly a mixing probability is. If anyone could please direct me to a resource that I couldn't find, I'd really appreciate it.

Sweet and salty can go great together. So lets say you have a pretzel that you want to sprinkle a mix of salt and sugar over. Do you HAVE to do 50/50? No way. You can vary it from 1 molecule of salt and a spoon of sugar all the way to 1 molecule of sugar and a spoon of salt. The weights sum to 1.0 and really make up the proportions of the mix. If you have two components and the first weight is 0.00001 and the second is 0.99999, that means the first is only slight and the second is huge.

For most ML things, I love autonlab. Here is their materials on mixture models. (link). This stuff tends to be pretty good.

Question 2: Since I only have the 3 data types listed above, how would I find standard deviation values without the range of raw data values?

You can back it out of the parameter. Here is the documentation for mclust. (link) Reading through it, while booring, can also be highly informative. The first thing to try is the "summary" command.

est <- mclust(.... stuff ...)
summary(est)


If the parameter values show up in the displayed summary, then they are 1) stored somewhere, and 2) able to be extracted.

The fact that this selects the model with best parameters (link) means that they can be accessed. I recommend you read through the documentation - it is in there. I also recommend you use "Rstudio" because it has a variable explorer that allows you to look at fields and sub-fields of the "est".

Question 3. And also any better description for a Gaussian finite mixture model than mine: forces data into multiple bell curves...?

See the link in answer 1 for Autonlab. A finite mixture model is a linear weighted sum of probability densities. A gaussian mixture, implicitly homogenous, is comprised as a weighted sum of Gaussian distributions. There are non-Gaussian mixtures. There are non-homogenous mixtures. I particularly like the Zero Inflated Poisson by Diane Lambert. I felt it was quite revolutionary in thought and valuable in application - allowing yield to be pushed closer to unity than a homogenous model and much of the thought at the time of publication.

Question 4. Why are these mixture models called unsupervised learning? All the explanations I bumped into online were way too technical for me.

You pick "number of components" and click and it goes. That is unsupervised. If you knew something about classification, and that was part of the input, then it would qualify as supervised.

Question 5. As shown in the data above, I often have >2 peaks. However I'm only concerned with the first two. Can I ignore the rest and safely call that part a bimodal distribution?

Throw away results only when you know why you are doing it. Whether or not you realize it, you are doing voodoo and not science. You don't know why it works, and if you did it twice you might get radically different results. Science is about knowing "why" before doing. It is about repeatable results.

In production, after we had root-caused one of the modes, only then could we focus on the other. If you spent time to get input data, and you knew what you were doing when you specified mode count (aka number of components) then throwing one out without a good reason is a bad idea.

That said, you could look at the weights and make an argument like 95% of the weight is in the first 2 modes, so I am going to focus on them. Management who doesn't realize how stunningly bad of an idea it is, will often let themselves fall prey to arguments like 80/20 rule - therefore I'm only retaining 80%. Remember that they are the technical illiterate (aka non-technical) and you are serving them quite poorly - not doing due diligence in your area of expertise - if such things are allowed to occur. Math, Statistics, Data-analysis folks - we are the guardians of truth. Not religious or political truth but actual "the universe is speaking" truth. If you get loose with what you know or what you don't then you are doing a bad job, and you are defacing the hard work and excellence of your peers in the field. Don't do it. Being ruthlessly aggressive about what you don't know and do know - never letting a hint of falsehood pass your lips - is how to build a great credibility and how to make a career worth having. Don't be an ass to folks - but be as excellent as you are capable of in your work.

Best of luck

• Beautiful answers to all my questions, and thanks for some great links as well! Appreciate your help. – swetharevanur Jan 19 '15 at 4:39