0
$\begingroup$

A Gaussian Mixture model is fitted by the Expectation-Maximization algorithm.

This fairly simple iterative algorithm consists of two steps and the initialization.

  1. Initialization (for k=2 Gaussians) Find (or guess) an initial muand sigma parameter value for both Gaussians.

  2. E-step Use the current Gaussian parameters to estimate for each data point the likelihood that is comes from Gaussian A (or B). Dividing these likelihoods by the sum of both allows to obtain probabilities (or posteriors) that a point is rather from A than from B. Here, prior information could be introduced, if the share of the underlying classes is known.

  3. M-step All mus and sigmas are updated. For example, the new mu value for Gaussian A can be calculated by the weighted sum of all data points. The weights are the posteriors of the previous step. Similarly the sigmas can be derived.

I wonder, how I could extend this algorithm to ignore a known share of data points. For example, I have the external knowledge that my dataset consists of two species A and B with known class probabilities of p(A)=70% and p(B)=15% and another 15% of unknown species. I want to fit two Gaussians to describe A and B distribution and want to allow the algorithm to ignore 15% of the data. This allows for smaller sigmas, as the remaining two Gaussians have no need to be stretched to cover data that is from another unknown distribution.

Note: Since the unknown species are possibly multiple species, I can not introduce a third Gaussian to "absorb" it. Also, the unknown species could be similarly distributed as A, so it is not necessarily the best thing to remove outliers.

So far, I use the known share of A and B as priors in the E-Step but I couldn't find an elegant way to ignore the unknown data. Any ideas? Thank you!

Example with 70% species A, 15% species B and 15% unknown

Example with 70% species A, 15% species B and 15% unknown

(this was originally posted on stackoverflow, but I was advised to bring it here (and close the question back there)

Improve this question
$\endgroup$
4
  • $\begingroup$ I do not understand the picture, as it seems to indicate four (or more) components. And "unknown" is too vague a qualitative to provide a clear answer. $\endgroup$
    – Xi'an
    Jun 30 at 8:29
  • $\begingroup$ The pic describes two attributes, say "height" and "width" of my species. I know that I studied 70% A 15% B and 15% other species. Now am interested in a good approximation of the height/width distribution of my species. Fitting two Gaussians gives me (hopefully) one large cluster (which must be A) and one small cluster (which must be B) and should ignore 15% of the data. I have many such samples with overlapping sets of species (sample 1 contains A,B,C sample 2 contains C,D,E etc). Mapping Gaussians to species becomes less ambiguuos as many species have high abundance in at least one sample. $\endgroup$
    – Klops
    Jun 30 at 9:30
  • $\begingroup$ If the 15% of the data is not identified as such, I cannot think of a sufficiently robust approach, except a Bayesian non-parametric approach that allows for a random number of components. $\endgroup$
    – Xi'an
    Jun 30 at 12:22
  • $\begingroup$ Could you elaborate on your idea? Number of components = number of species? Thank you $\endgroup$
    – Klops
    Jun 30 at 14:33

1 Answer 1

Reset to default
1
$\begingroup$

Look at the tclust package in R for clustering based on the Gaussian distribution with trimming a certain percentage of observations, its vignettes and references.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you, this looks indeed promising and lead me to relevant papers and keywords. Despite this being very valuable to me, maybe other answers will go more into detail $\endgroup$
    – Klops
    Jun 29 at 23:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.