I have a set of 2D positions, obtained by tracking an object with an rgb camera. For various reasons, I want to train a model so that given a new position I can estimate how likely it is that it was generated by the same underlying process as the original set.

I've modeled the distribution as a 2D normal and for the system I'm building it works fine, but I'm being asked to perform a normality test on the set of 2D positions to ensure this is indeed reasonable, or if for example I should be using a GMM.

I've found the Peacock Test for multivariate ks testing. However from what I could understand, it was developed to test two sets of samples come from the same distribution, or to check the fit of a single set of samples to a normal distribution with known parameters ($\mu$ and $\Sigma$).

  • How could I test for the hypothesis that it comes from some normal distribution, even if I don't know $\mu$?
  • I though about estimating $\mu$ and $\Sigma$ from the datapoints, then generating samples with the estimated parameters for a 2D normal, and performing the peacock test on the two sets of samples (the original and generated). This is intuitive but hardly rigorous.
  • Would it be better to use BIC or similar and try to estimate the number of gaussians for a GMM, even if in most cases there would be just a single one?
  • 2
    $\begingroup$ Testing is probably not the best choice -- your data almost certainly aren't normal, but a normal may be a perfectly good approximation, entirely suitable for your purposes. However, no matter how reasonable that approximation, with enough data you will reject normality. On the other hand if you don't have enough data, failure to reject is no comfort, because it in no way indicates that the approximation is good. Hypothesis testing addresses the wrong question. The question you will care about is more like an effect size (am I close enough to normal for my purposes) than a hypothesis test $\endgroup$
    – Glen_b
    Commented Jul 11, 2016 at 0:15
  • 1
    $\begingroup$ However, the question may have broader interest... $\endgroup$
    – Glen_b
    Commented Jul 11, 2016 at 0:15
  • 1
    $\begingroup$ I agree, for my purposes I only care that it is a reasonable approximation. I guess in that case the appropriate measure of how good an approximation it is is the performance of the model on the training set, but the reviewer doesn't see it that way. $\endgroup$ Commented Jul 11, 2016 at 12:18


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.