0
$\begingroup$

I know it can happen for it to decrease dramatically as it overfits on a single datapoint. But I've never read about a component "taking everything over". See the following images (circles are stddevs). Could this be (relatively) normal behaviour?

edit: For some reason changing my gaussian function made the behaviour sane, note both return the same output for given inputs:

iter 300! iter 300!

$\endgroup$
  • $\begingroup$ How are you fitting the mixture model.. is it EM? What does the circles stand for... a contour plot of sort? $\endgroup$ – Suren Jan 23 '18 at 0:33
  • $\begingroup$ Yes EM. Contour is stddev of the component (root of variance, covar matrix is diagonal). $\endgroup$ – Nimitz14 Jan 23 '18 at 9:11
  • $\begingroup$ It is very much possible to end up with smaller size cluster, if you are fitting an incorrect model, say larger number of clusters. By looking at this data, even though it is 2D, it is difficult to say how many clusters would be appropriate or what type of mixture model you should fit. I'd recommend trying something like mclust software to see what it does, and then compare that with outcomes of your algorithm. $\endgroup$ – Suren Jan 24 '18 at 0:27
0
$\begingroup$

I think what is happening is that in the event of a component stepping towards the center of all the datapoints while already having the largest variance(s), as the posteriors/responsibilities of all the datapoints will increase (as its coming close to the center), the variance will get larger, which will make it take another step towards the center and so on.

At the same time, it seems something is wrong with my implementation as the component that is getting "swallowed" should not be staying still, as the culprit gets the larger variance and moves closer, the posteriors/responsibilities of the swallowed one should make it move away, yet it barely budges. So it seems to just be a bug of my code.

edit: Don't think there's a mistake anymore. The large variance makes the density quite low and so the responsibilities of the swallowed cluster one barely decrease.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.