# Fitting a Gaussian Mixture Model with known share of noise/outliers

A Gaussian Mixture model is fitted by the Expectation-Maximization algorithm.

This fairly simple iterative algorithm consists of two steps and the initialization.

1. Initialization (for k=2 Gaussians) Find (or guess) an initial muand sigma parameter value for both Gaussians.

2. E-step Use the current Gaussian parameters to estimate for each data point the likelihood that is comes from Gaussian A (or B). Dividing these likelihoods by the sum of both allows to obtain probabilities (or posteriors) that a point is rather from A than from B. Here, prior information could be introduced, if the share of the underlying classes is known.

3. M-step All mus and sigmas are updated. For example, the new mu value for Gaussian A can be calculated by the weighted sum of all data points. The weights are the posteriors of the previous step. Similarly the sigmas can be derived.

I wonder, how I could extend this algorithm to ignore a known share of data points. For example, I have the external knowledge that my dataset consists of two species A and B with known class probabilities of p(A)=70% and p(B)=15% and another 15% of unknown species. I want to fit two Gaussians to describe A and B distribution and want to allow the algorithm to ignore 15% of the data. This allows for smaller sigmas, as the remaining two Gaussians have no need to be stretched to cover data that is from another unknown distribution.

Note: Since the unknown species are possibly multiple species, I can not introduce a third Gaussian to "absorb" it. Also, the unknown species could be similarly distributed as A, so it is not necessarily the best thing to remove outliers.

So far, I use the known share of A and B as priors in the E-Step but I couldn't find an elegant way to ignore the unknown data. Any ideas? Thank you! Example with 70% species A, 15% species B and 15% unknown

(this was originally posted on stackoverflow, but I was advised to bring it here (and close the question back there)

• I do not understand the picture, as it seems to indicate four (or more) components. And "unknown" is too vague a qualitative to provide a clear answer. Jun 30 at 8:29
• The pic describes two attributes, say "height" and "width" of my species. I know that I studied 70% A 15% B and 15% other species. Now am interested in a good approximation of the height/width distribution of my species. Fitting two Gaussians gives me (hopefully) one large cluster (which must be A) and one small cluster (which must be B) and should ignore 15% of the data. I have many such samples with overlapping sets of species (sample 1 contains A,B,C sample 2 contains C,D,E etc). Mapping Gaussians to species becomes less ambiguuos as many species have high abundance in at least one sample. Jun 30 at 9:30
• If the 15% of the data is not identified as such, I cannot think of a sufficiently robust approach, except a Bayesian non-parametric approach that allows for a random number of components. Jun 30 at 12:22
• Could you elaborate on your idea? Number of components = number of species? Thank you Jun 30 at 14:33