I have a large number of data sets. Each data set has something 200K data points lying in a square times a circle. The square is solid $I\times I$. The circle $S^1$ is hollow (dim 1). By reasoning from the experimental setup that produces the data, I am convinced that this collection of points is a sample from mixed noise plus gaussian, and the gaussian is confined to a small region. (We can assume this at first, but more realistic reasoning would give a mixture of a small number of gaussians, with centres pretty close together. I'll treat it as a single gaussian for the moment.) The problem is that the noise is far from uniform. Moreover, the noise distribution definitely differs from one dataset to the next.
Given one of my datasets D with k points (k varies with D) and underlying noise distribution $Noise(D)$, I have a way of rapidly producing a sample of k points drawn from $Noise(D)$. In other words, the information that comes with D is sufficient to identify $Noise(D)$ in the sense that it is possible to generate a sample of k points from $Noise(D)$.
My guess is that there are something like 100 points explained by the Gaussian, though this could be optimistic.
I'm not a statistician. How could I now estimate my unknown gaussian? It seems that this should be possible, since I "know" $Noise(D)$. Could someone explain to me how I could do something like the EM algorithm in my current situation. Also, could someone please recommend an online explanation of how EM is usually carried out?
