1
$\begingroup$

I am running an experiment where I use software to count the number of particles on a filter. The software is not always perfect and has different parameters etc. that I can set, I am also using different backgrounds to improve contrast. There can be particles missed, particles counted multiple times, background counted as particles etc.

I thought of collecting the false positive (background counted as a particle or particle counted >1), false negatives(particle missed), true positive(particle identified correctly), and true negative (background identified correctly?)

I am sure I am using the wrong method or confused about using this method as the TN will be infinite/0?

Is there a better way to prove which settings/backgrounds are better statistically? Can I ignore TN and use the rest of the TP, FP, FN statistics?

Improve this question
$\endgroup$
2
  • $\begingroup$ Maybe look at ROC curve. // Maybe choose the method that gets closest to the true prevalence of particles. // Possibly relevant: This Q&A or its links $\endgroup$
    – BruceET
    Commented Sep 14, 2020 at 18:30
  • 1
    $\begingroup$ Please say more about just what you would mean by a "true negative" and how you would count them in this case. Usually this type of image-segmentation software reports where objects of interest are but simply leaves the background blank. For example, if there were just a few particles on the filter most of the image would be negative, but there wouldn't be any negative objects identified.. $\endgroup$
    – EdM
    Commented Sep 14, 2020 at 18:30

1 Answer 1

Reset to default
0
$\begingroup$

This type of particle analysis doesn't usually identify true negative objects. It tries to identify objects of a certain type based on things like size and shape. So you can get calls that are true positive, false positive (calling an object that isn't there) or false negative (missing a real object). But the entire rest of the area analyzed is simply called negative. There are no true negative objects.

This might be the one type of classification scheme in which an F-score is very useful. An F-score:

is calculated from the precision and recall of the test, where the precision is the number of correctly identified positive results divided by the number of all positive results, including those not identified correctly, and the recall is the number of correctly identified positive results divided by the number of all samples that should have been identified as positive.

If you look carefully, you will note that true negatives aren't considered at all. That can be a problem if there really are individual objects that are truly negative. But you don't have any true negative objects, so an F-score seems quite applicable here.

The usual $\textsf{F}_1$-score is the harmonic mean of the precision and recall, treating them equally. If one of precision or recall is more important for your application, the page linked above shows how to calculate an $\textsf{F}_\beta$-score, for any positive factor $\beta$ representing the relative importance of recall over precision.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.