How to test which data match model at best

Question

Description

I have 1D data with $N$ normally distributed clusters. I have to find a cluster, which is the worst (differs at most from the normal distribution).

My approach

I calculate $sq = \frac{(f(x) - y)^2}{\#y}$, where $f(x)$ is value of normal PDF with mean equal to the center of the cluster and sigma equal to cluster's "radius", $\#y$ is the total number of observations. Cluster with highest value of $sq$ is considered to be the worst one.

Question

Problem is that number of points per cluster differs a lot (one cluster could have 3000 points and other 300). And imho I think that if I had small errors and many points, I would end up with larger $sq$, then if I had bigger errors and small amount of points.

Can you point me the right way?

Answer 1

Have you looked at Kolmogorov-Smirnov tests?

These are meant to measure how well an empirical CDF matches the theoretical CDF.

You can't really rely on the p-values (as you used the same data for constructing your model, I assume?) and may also be biased towards rejecting larger data sets, though. There are some alternatives, and plenty of literature for you to read (A starting point is "See also" in the Wikipedia article above).