I'm calculating the F-Score for a sandbox dataset: 100 medical patients, 20 of which have cancer. Our classifier mis-classifies 20 healthy patients as having cancer, and 5 patients with cancer as healthy, the rest it gets right.
We compute True Positives; True Negatives; False Positives; and False Negatives.
We ran into a debate about which class comes first, those that test "Positive" for cancer, or the majority class, e.g. those that are "Healthy".
Explicit Question: What is the correct true-positive rate in this dataset? Is it:
- # of predicted healthy patients over # of actual healthy patients
- # of predicted cancer patients over # of actual cancer patients
Bonus points if you can reference some literature that supports one supposition or the other.
Note, I've skimmed through a few texts on f-scores but haven't seen an explicit discussion of this point:
Wikipedias text on precision and recall seem to suggest that "true positive" be defined by whatever "test" is being performed, and thus in this case defined as the minority class because the "test" is for cancer. However I don't find the discussion rigorous enough to convince me. If I simply describe the test in terms of testing for "healthy" patients I change the f-score, but this was just a semantic change. I would expect the f-score to have a mathematically rigorous definition.