Estimating variance of center-censored Normal samples

Question

I have normally-distributed processes from which I get small samples (n typically 10-30) that I want to use to estimate variance. But frequently the samples are so close together that we can't measure individual points near the center.

I have this vague understanding that we should be able to construct an efficient estimator using ordered samples: E.g., if I know the sample contains 20 points, and that 10 are clustered near the center too tightly to measure individually, but I have discrete measurements of 5 on either tail, is there a standard/formulaic approach for estimating the process variance that makes optimal use of such samples?

(Note that I don't think I can just weight the center average. For example, it's possible for 7 samples to cluster tightly while another three are asymmetrically skewed to one side but close enough we can't tell that without more tedious single sampling.)

If the answer is complicated any tips on what I should be researching would be appreciated. E.g., is this an order-statistic problem? Is there likely to be a formulaic answer, or is this a computational problem?

Updated detail: The application is analysis of shooting targets. A single underlying sample is the point of impact (x,y) of a single shot on the target. The underlying process has a symmetric bivariate normal distribution but there is no correlation between axes, so we are able to treat the {x} and {y} samples as independent draws from the same normal distribution. (We could also say the underlying process is Rayleigh-distributed, but we can't measure the sample Rayleigh variates because we can't be certain of the coordinates of the "true" center of the process, which for small n can be significantly distant from the sample center ($\bar{x}$, $\bar{y}$).)

We are given a target and the number of shots fired into it. The problem is that for n>>3 precise guns will typically shoot a "ragged hole" surrounded by distinct shots. We can observe the x- and y-width of the hole, but we don't know where in the hole the non-distinct shots impacted.

Here are some examples of more problematic targets:

(Granted, in an ideal world we would change/switch targets after each shot and then aggregate the samples for analysis. There are a number of reasons that's often impractical, though it is done when possible.)

Further Notes following WHuber's clarifications in the comments: Shots produce target holes that are of uniform and known diameter. When a shot is outside any "ragged group" we know the projectile radius and so we can measure the precise center $x_i$. In each "ragged group" we can discern some number of peripheral "balls" and again mark the precise center of those outer shots based on the known projectile radius. It is the remaining "center-censored" shots that we only know impacted somewhere in the interior of a "ragged group" (which is typically -- and if necessary let us assume -- one per target).

To facilitate solution I believe it will be easiest to reduce this to a set of one-dimensional samples from the normal, with a central interval of width w > d, where d is the projectile diameter, containing c < n "censored" samples.

Answer 1

That is an interesting problem. First, I would not make the assumption of a normal distribution. It seems what you are really looking for is some estimate of dispersion that you apply fairly to many different shooters or guns or ammo or whatever.

I'd try to turn this around. You don't know exactly where all the bullets went unless you see 10 separate holes (assuming 10 shots). But you do know where they didn't go. This could be used to constrain the distribution assuming Bayesian statistics if you want to start with a distribution.

An idea that might be best here is to stop trying to do it mathematically and just do something sensible like this. Take the target and run an image processing routine to mark the shot through area which may be unconnected. Measure the mean and second moment of this and use these are an estimator. If you want to go a little further and try to Gaussianize it, you can run simple monte carlo experiment to get a calibration factor.

Answer 2

From another vantage point, one could view this in the light of the field of Spatial Statistics, which has created an assortment of metrics, many of which have been placed in toolboxes (see, for example, ArcGIS).

Wikipedia (link: http://en.m.wikipedia.org/wiki/Spatial_descriptive_statistics ) actually has a good introductory page discussing such concepts as measures of spatial central tendency and spatial dispersion. To quote Wikipedia on the latter:

For most applications, spatial dispersion should be quantified in a way that is invariant to rotations and reflections. Several simple measures of spatial dispersion for a point set can be defined using the covariance matrix of the coordinates of the points. The trace, the determinant, and the largest eigenvalue of the covariance matrix can be used as measures of spatial dispersion. A measure of spatial dispersion that is not based on the covariance matrix is the average distance between nearest neighbors.

Related concepts include measures of spatial homogeneity, Ripley's K and L functions, and perhaps most relevant for the analysis of bullet clusters, the Cuzick–Edwards test for clustering of sub-populations within clustered populations. The latter test is based on the comparison (using "nearest-neighbor" analyses to tabulate statistics) to a control population, which in the current context could be based on actual observed targets classified as not displaying clustering, or per a theoretical simulation, from say the Rayleigh distribution.