I have datasets of increasing sizes identically distributed. I have tried to fit a gaussian mixture to these datasets using Expectation-Maximization algorithm.
To check the quality of this fit, I have run goodness of fit tests between the density of the known points and the estimated density. It provides very good results. Tests hypothesis are accepted in more than 90% of the cases.
The acceptation rate first increases with the number of points in the learning dataset (as I was expected). And then for the last dataset size, the acceptation rate falls from 95% to about 85%... I have run tests many times and it is always the same...
Do you have some hints about what is going on ? Why would the quality of the model be so low (compared to other sizes) for one particular learning dataset size ?... Do you know articles where the behavior of Expectation-Maximization is studied in function of the sample size ?
EDIT : As I can simulate as many datasets as I want. I do not use cross-validation. To test the quality of the density estimated with a dataset $A$, I run the goodness-fo-fit tests between a test dataset $B$, simulated in the same way that dataset $A$ and a sample of points simulated from the estimated gaussian mixture. The size of the dataset $B$ is constant through my tests and thus does not depend on the size of dataset $A$.