7
$\begingroup$

I just estimated the parameters for a mixture of two gaussians with different means and different sigmas, I would like to test if the data adjusts well to the explicit form of the mixture, do I necessarily need to simulate data from the mixture or how can I test the goodness of fit? I am using the mixtools package.

$\endgroup$
  • $\begingroup$ Question flagged for migration, so you don't have to add a new question to that site. $\endgroup$ – Joris Meys May 21 '12 at 10:04
  • $\begingroup$ A nonparametric option is the Kolmogorov-Smirnov test but there are probably some ad hoc tools for the case of Gaussian mixtures. $\endgroup$ – user10525 May 21 '12 at 15:57
8
$\begingroup$

You can write a function that calculates the relevant value for a given test under the null hypothesis based on the outputs from, say, normalmixEM, then do the test using that. For example, for the Kolmogorov-Smirnov test, we need the CDF given a set of parameters:

# CDF of mixture of two normals
pmnorm <- function(x, mu, sigma, pmix) {
  pmix[1]*pnorm(x,mu[1],sigma[1]) + (1-pmix[1])*pnorm(x,mu[2],sigma[2])
}

We then run the K-S test in the usual way:

# Sample run
x <- c(rnorm(50), rnorm(50,2))

foo <- normalmixEM(x)
test <- ks.test(x, pmnorm, mu=foo$mu, sigma=foo$sigma, pmix=foo$lambda)
test

    One-sample Kolmogorov-Smirnov test

data:  x 
D = 0.0559, p-value = 0.914
alternative hypothesis: two-sided 

Always keeping in mind the fact that we estimated the parameters from the same data we are using to do the test, thus biasing the test towards failure to reject H0.

We can overcome that latter bias to some extent via a parametric bootstrap - generating many samples from a mixture of normals parameterized by the estimates from normalmixEM, then estimating the parameters of the samples and calculating the test statistics for each sample using the estimated parameters. Under this construction, the null hypothesis is always true. In the code below, I'm helping the EM algorithm out by starting at the true parameters for the sample, which is also cheating a little, as it makes the EM algorithm more likely to find values near the true values than with the original sample, but greatly reduces the number of error messages.

# Bootstrap estimation of ks statistic distribution
N <- length(x)
ks.boot <- rep(0,1000)
for (i in 1:1000) {
  z <- rbinom(N, 1, foo$lambda[1])
  x.b <- z*rnorm(N, foo$mu[1], foo$sigma[1]) + (1-z)*rnorm(N, foo$mu[2], foo$sigma[2])
  foo.b <- normalmixEM(x.b, maxit=10000, lambda=foo$lambda, mu=foo$mu, sigma=foo$sigma)
  ks.boot[i] <- ks.test(x.b, pmnorm, mu=foo.b$mu, sigma=foo.b$sigma, pmix=foo.b$lambda)$statistic
}

mean(test$statistic <= ks.boot)
[1] 0.323

So instead of a p-value of 0.914, we get a p-value of 0.323. Interesting, but not particularly important in this case.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Does anything change in the following case? I have 100 simulated obs from a mixture of two Normals. So i know the means and variances of the "true" mixture. I generate from the predictive distribution 20000 samples. How can i test if those samples are from the specific mixture? I believe i have to skip the foo variable and substitute the known means and variances in the ks.test right? or i am missing something bigger here? $\endgroup$ – Christos Nov 20 '13 at 14:48
  • $\begingroup$ @Christos - Your intuition is correct - better to substitute the known means and variances in the ks test, and, if you do so, you don't need to bootstrap the ks test statistic either. $\endgroup$ – jbowman Nov 20 '13 at 14:53
  • $\begingroup$ Could you please explain how a $p=0.914$ implies that the data from the mixture distribution? I thought failure to reject H0 did not mean that. $\endgroup$ – Jacob May 5 '14 at 11:58
  • $\begingroup$ @Jacob - where did I say that a p-value of 0.914 implies that the data is from a mixture distribution? $\endgroup$ – jbowman May 5 '14 at 17:30
  • $\begingroup$ @jbowman: I'm sorry, you obviously don't. However, is there a statistical test to establish equivalence? I.e. given a CDF and a small sample of data, is there a statistical test which can prove that the data follows the CDF? $\endgroup$ – Jacob May 5 '14 at 20:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.