Statistics for a Particle finding experiment? I am running an experiment where I use software to count the number of particles on a filter. The software is not always perfect and has different parameters etc. that I can set, I am also using different backgrounds to improve contrast. There can be particles missed, particles counted multiple times, background counted as particles etc.
I thought of collecting the false positive (background counted as a particle or particle counted >1), false negatives(particle missed), true positive(particle identified correctly), and true negative (background identified correctly?)
I am sure I am using the wrong method or confused about using this method as the TN will be infinite/0?
Is there a better way to prove which settings/backgrounds are better statistically? Can I ignore TN and use the rest of the TP, FP, FN  statistics?
 A: This type of particle analysis doesn't usually identify true negative objects. It tries to identify objects of a certain type based on things like size and shape. So you can get calls that are true positive, false positive (calling an object that isn't there) or false negative (missing a real object). But the entire rest of the area analyzed is simply called negative. There are no true negative objects.
This might be the one type of classification scheme in which an F-score is very useful. An F-score:

is calculated from the precision and recall of the test, where the precision is the number of correctly identified positive results divided by the number of all positive results, including those not identified correctly, and the recall is the number of correctly identified positive results divided by the number of all samples that should have been identified as positive.

If you look carefully, you will note that true negatives aren't considered at all. That can be a problem if there really are individual objects that are truly negative. But you don't have any true negative objects, so an F-score seems quite applicable here.
The usual $\textsf{F}_1$-score is the harmonic mean of the precision and recall, treating them equally. If one of precision or recall is more important for your application, the page linked above shows how to calculate an $\textsf{F}_\beta$-score, for any positive factor $\beta$ representing the relative importance of recall over precision.
