11
$\begingroup$

I've encountered an interesting discussion at work on interpretation of precision (confusion matrix) within a machine learning model. The interpretation of precision is where there is a difference of opinion so my description centers a little bit simplified around precision.

The problem discussed with some numbers for reference:

Suppose we have a machine learning model A for binary classification. The training dataset has 100.000 datapoints. The data is quite imbalanced: 5% is classified as 1, 95% as 0. The model is tested on another (unseen) dataset of 50.000 datapoints. For evaluation a confusion matrix is made to evaluate model A. Precision (TP/(TP+FP) = 30%. From here on there is a difference in view between the data scientists (both camps are intelligent people).

Group 1: We think the model is useful. While precision is low (30%), it is quite higher than random (5%). Therefore the model has some value. We can use the output the model generates and can expect the model to pinpoint datapoints which on average have a 30% probability of being a 1.

Group 2: You can not use this model. Precision needs to be at minimum 70-80% for a model to be useful. The point is, precision only measures the model, and not the underlying data. Therefore one can only use models with a minimum of 70-80% precision. Balanced or imbalanced data doesn't matter.

I myself find myself more in group 1. So if someone can explain why group 2 is right (if they are right) I would be happy.

More context: We produced a model to pinpoints locations where inspectors can find objects with on error. In the past ( al the datapoints) inspectors found on average in 5% of their visits an error. Model A had a precision of 30%. So the reasoning of group 1 is if inspectors only go to these 30% locations pinpointed by the model they will on average find more errors than the historic 5%. I should ad that only about 500 visits per year will be done, false negatives have no associated costs. So any precision gain more than 5% would be good.

$\endgroup$
17
  • 5
    $\begingroup$ You should consider the specificity as well to get a fuller picture. Certainly the model is providing additional information beyond the base rate ($30\% > 5\%$). Regarding whether it's useful, you need to ask the people who will use it. The idea that there is a fixed rule that a model is useful when precision is at least $70\%$ is just dumb. $\endgroup$ Commented Mar 7, 2023 at 20:50
  • 1
    $\begingroup$ Does this answer your question? Is my model any good, based on the diagnostic metric ($R^2$/ AUC/ accuracy/ RMSE etc.) value? $\endgroup$
    – mkt
    Commented Mar 7, 2023 at 21:02
  • 6
    $\begingroup$ A precision of $30\%$ means that, when the model predicts a member of the positive class, there is a $30\%$ chance of it being a member of that class. Does that sound useful? I agree with @gung-ReinstateMonica that a blanket rule about $70$ or $80\%$ is silly, but that group $2$ might have good reason to require such performance, despite the legitimate claim from group $1$ that the model is better than random. (Better than random is the bare minimum, not necessarily the target.) // Does your model output discrete classes? Neural networks and logistic regressions don’t predict discrete classes. $\endgroup$
    – Dave
    Commented Mar 7, 2023 at 21:05
  • 2
    $\begingroup$ I don’t follow that, but if there is a category where predicting that category and getting it wrong incurs no cost (a scenario I struggle to believe, but let’s go with it), there is a strong argument to predict that category every time, especially if getting it right provides a gain. $\endgroup$
    – Dave
    Commented Mar 7, 2023 at 22:07
  • 1
    $\begingroup$ @Dave - And if you get every prediction wrong in a binary model...then you have a model that perfectly models the underlying phenomenon. Just pick the opposite of whatever it suggests. $\endgroup$
    – Obie 2.0
    Commented Mar 10, 2023 at 0:06

8 Answers 8

30
$\begingroup$

As Dave argues, if "false negatives have no associated costs", then your best course of action would be not to classify anything as positive, i.e., as 1. You inspect nothing at all, you incur zero cost, and all your inspectors can do something different (or be fired).

Yes, of course this makes no sense. Which is because it makes no sense to claim that false negatives incur no costs. They do, it's the cost of uncaught errors.

This is an example of why precision, sensitivity etc. are all as misleading as accuracy as evaluation metrics, especially (but not only) in "unbalanced" situations.

What I would strongly recommend you do is scrap "hard" classifications, use probabilistic classifications instead and separate the decision aspect from the statistical modeling aspect. The decisions (whether to inspect or not) should not only be driven by the probabilistic classification, but also by the cost structure.

$\endgroup$
4
  • $\begingroup$ I do agree that there would be value gained from looking at the probabilistic output of the model and letting those plus the costs of wrong decisions guide how to proceed. However, this seems just to delay addressing the question: if, after considering the probabilities of event outcomes and the costs associated with misclassification, the optimal classification scheme only gives $30\%$ precision, is that enough? $\endgroup$
    – Dave
    Commented Mar 8, 2023 at 12:03
  • 11
    $\begingroup$ @Dave: I would argue that precision is meaningless. Whether a given probabilistic model adds any value will depend on the alternative. If we have budget and resources to do 500 inspections, we can either do them at random, or as guided by a model (e.g., by picking the 500 highest predicted probabilities), or by some other rule (e.g., inspecting those installations where the last inspection was the longest time ago). If we then have a handle on the cost structure (!), we can say whether the decisions (!) made in either of these ways are better than others. $\endgroup$ Commented Mar 8, 2023 at 12:27
  • $\begingroup$ Thanks fot the links Stephan. If i understand correctly, probabilistic classifications would label or 'rate' a future inspection. The idea is also to divide all possible visits in 2 groups where 1 group will still be picked randomly, and the other group by the model. So e.g. if 500 inspections per year are possible, simply let the model pick 250 and the other 250 are still randomly chosen. This also provides a nice benchmark group to see model performance, and still one is able to pick up changes in the environment from the random group $\endgroup$
    – wmmwmm
    Commented Mar 11, 2023 at 12:04
  • $\begingroup$ That is definitely one possibility. I would also suggest you do not limit your decision to the output of the model alone - if there is an "important" site (important customer, big installation) with a lower predicted break down probability, then it might be worthwhile to inspect that rather than a "less important" site with a lower predicted probability. Similar thoughts could apply once you look at costs of inspections, e.g., in terms of distance - maybe you can inspect five installations in a day if they are close together, rather than three that are far apart. $\endgroup$ Commented Mar 11, 2023 at 12:30
18
$\begingroup$

I would say neither group is entirely correct. The question is what do you want to do with the model, and what will happen for positive or negative model predictions?

There are screening tests used in medical practice that have precision (positive predictive value) that low -- mammograms for breast cancer, prostate-specific antigen for prostate cancer. The positive predictive value of low-dose spiral CT for detecting lung cancer in smokers is way lower than 30%.

What these have in common is that you really want to detect cases, so you care much more about sensitivity (recall) than precision. The benefit of a true positive is much more than 3 times the cost of a false positive.

So, what you need to know to decide is the two costs ('losses' in decision theory). You can then work out the expected loss from using the algorithm and from not using it, and see which is lower.

$\endgroup$
7
  • $\begingroup$ Exactly! I also introduced costs of FN and FP in my presentation. But group 2 stays unconvinced. They keep claiming "you just can't expect a probability of 30% the model will find new cases in new data" $\endgroup$
    – wmmwmm
    Commented Mar 7, 2023 at 21:35
  • $\begingroup$ @wmmwmm What justification do they give for that? Precision is literally the probability of a case belonging to the predicted category ($P(Y=1\vert \hat Y=1)$). Unless there is reason to believe the new data are inherently different (could be, but that’s a very different discussion), that group appears to reject the standard definition of precision in machine learning. $\endgroup$
    – Dave
    Commented Mar 7, 2023 at 22:10
  • 1
    $\begingroup$ Of course you can't expect a probability of 30%... unless you have good reason to believe that your test data will be distributed identically to your training data, rarely true in practice. "Unless there is reason to believe the new data are inherently different" -- Yes Dave, and the more complicated the ML model, the more likely it is to be relying on things that are likely to be different. I've seen tons of models that do great on training data (30% > heuristic), pass validation (say 25% better), and fall apart on the real new data which is gathered independently (2% worse vs. heurestic). $\endgroup$
    – ttbek
    Commented Mar 8, 2023 at 10:29
  • $\begingroup$ To rephrase, the default assumption should be that there is likely a fair amount of variation in the distribution. Yes, it's obnoxious, why can't they have gathered even larger data and included the entirety of what data we might see? Sometimes they even did gather almost everything out there... but the distribution drifts with time, welcome to non-stationary data hell. $\endgroup$
    – ttbek
    Commented Mar 8, 2023 at 10:32
  • $\begingroup$ @Dave; the argument is basically the precision only can say something about the model, and not the underlying data. The outher group(number 1) says the data is representative for the population, and both training and test dataset are evenly distributed. $\endgroup$
    – wmmwmm
    Commented Mar 11, 2023 at 11:48
9
$\begingroup$

This primarily depends on how the model is supposed to be used. From your context it seems you have an alternative test which has an almost perfect classification rate but is very expensive to use because it essentially consists of sending a qualified human to do a manual check. You want to use your model (which in comparision is extremely cheap) to decide where to perform the human tests.

If I understand you correctly than a) your goal is to find as many instances evaluated by a human as 1 as possible (implying instances evaluated as 0 are not valuable) and b) if some instances that are 1 are not checked by a human because the model thinks they are unlikely candidates, this is not a problem because the number of human checks is very limited anyway.

If these assumptions are correct than any model that has a bigger than 5% chance to find instances qualified as 1 is useful and your model with a 30% hit rate will increase the number of instances qualified as 1 sixfold so is a major improvement compared to not using a the model.

There are of course plenty of other situations where such a model would be useless or even actively bad if applied. It just depends on what you want to do with the model.

$\endgroup$
7
$\begingroup$

As with most things, neither of two polar opinions is wholly correct

If a model can select an area needed for inspection better than random chance then for a singular inspection run you're using your inspector's time more usefully.

As an example, lets say your inspectors are checking welds on a pipeline. A classification model for error/no error may pick up early signs of corrosion but not spot a more subtle error like porosity or an undercut. In the long term, dependence on the model to inform inspectors could mean other error type get inspected less.

I'd recommend trying to look into your false negatives here, I know you say they incur no cost but you've got inspectors for a reason. Is there a particular type of defect that your model isn't picking up? Is there more data you can bring in to better account for the other error types?

tldr;

Better than random sounds like it would be more effective, but if your model is blind to an error type it could be increasing the likelihood that that error goes unnoticed (without knowing the specific situation we can't say more). At the same time 70-80% is just a number picked from thin air. A lot is down to your application.

$\endgroup$
3
  • $\begingroup$ The only truly correct answer on this thread so far. The reason is that if we want to prevent drastic failures, our goal should not be to maximize number of correct finds of defects are but rather minimize cost of defects that we fail to find, and as you so nicely pointed out, different kinds of defects can have very different costs. In fact, defects often get more serious over time, and so it may very well be that maximizing successful correct finds leads to finding obvious defects all the time but failing to find all the obscure defects that get more and more serious... $\endgroup$
    – user21820
    Commented Mar 10, 2023 at 14:10
  • $\begingroup$ I agree with you in this. If inspections are only held in the 'suspicious' cases this will feed the model with it's own data. New trends and datadrift will not be picked up. The plan is to split all possible inspections in 2 groups; 1 group by the model and 1 group still random. The random group is used to pick up new trends and can be used as benchmark to check and feed the model $\endgroup$
    – wmmwmm
    Commented Mar 11, 2023 at 11:40
  • $\begingroup$ @wmmwmm: Yes, that is one way to avoid the problem! By the way, you can click on the tick to mark this answer as accepted. $\endgroup$
    – user21820
    Commented Mar 11, 2023 at 11:54
3
$\begingroup$

5% would not be the performance of the model making predictions “at random” or if you just classified everything as 1’s.

I'm not sure if comparing it to the base rate makes sense here. It means comparing to the most primitive alternative possible. Why not try some other simple model (decision tree, logistic regression, $k$NN) as a benchmark?

Moreover, there's nothing magical about “70-80% precision”. For some problems, this would not be achievable, but for others way too low. Those numbers are arbitrary and there's no reason whatsoever to aim at them.

$\endgroup$
6
  • 2
    $\begingroup$ @wmmwmm ok, so what? If this is the best you can get, it's the best you can get. If it's not enough and the alternative not to use a machine learning is better, let it be. $\endgroup$
    – Tim
    Commented Mar 7, 2023 at 21:03
  • $\begingroup$ I Agree, but if a model with precision is discarded while being usefull that will not a good decision $\endgroup$
    – wmmwmm
    Commented Mar 7, 2023 at 21:27
  • $\begingroup$ We have tried a lot of models and the best model produced a precision of 30%. Group 2 discards the model nevertheless – $\endgroup$
    – wmmwmm
    Commented Mar 7, 2023 at 21:50
  • 1
    $\begingroup$ @wmmwmm many good models are never used, in other cases, many hours are spent on building models that never reach the level of performance that would make them useful. $\endgroup$
    – Tim
    Commented Mar 7, 2023 at 21:57
  • $\begingroup$ @Tim: I don't understand your point about $5$ percent. If $X$ (the real data) is a random variable such that $\mathbb{P}[X=1] = 1 - \mathbb{P}[X=0]$, and if $Y$ (the classification) is any random variable independent from $X$, then $\frac{\mathbb{P}[X=1 \vert Y = 0]}{\mathbb{P}[X=1\vert Y = 0] + \mathbb{P}[X=0 \vert Y = 0]} = \mathbb{P}[X=1]$. The random variable $Y$ could be constant equal to $1$, constant equal to $0$, or anything else, right? Or are we talking about different things? $\endgroup$
    – Plop
    Commented Mar 9, 2023 at 9:46
2
$\begingroup$

As hashed over a bit in the comments. The usefulness of the model here is going to hinge heavily on how likely it is that the distribution of your future data is to match the training/validation data.

I think it would be a serious mistake to assume they will match unless you could think of a super obvious reason why they wouldn't. I would assume there will be differences and test that hypothesis. If the generation of this data is stationary and highly likely to remain similar, then the model is probably useful, but you should make an attempt to quantify this (e.g. if your current data was gathered at different times, separate it into the earliest and latest data and check model performance within each subgroup and see if there are time-based trends in the data itself (look for time/geography/population-based trends in a wide range of statistics, means, medians, maxes, mins, curl, etc...) or if there is any other potential reason the data might change, e.g. new professional guidelines). I wouldn't recommend assuming the model will generalize even with good looking generalization on train/test/validation sets until you get a new real-world independent validation set. And then the model might work just for your company with exactly how you gather data right now, if you outsource the data gathering any sort of new semi-systemic idiosyncrasy might throw a serious wrench in the works. Likewise, it may not work at another branch office, etc...

This sort of issue is a plague in precision medicine, in part because the large whole genome and exome data sets and GWAS studies are so biased towards white Europeans but when you want to go to clinical application you're suddenly not treating only white Europeans. Then there are false positives... you can have a SNP associated with their descent look associated with disease but have it turn out to be socioeconomic rather than genetic, etc... This is in part why ML hasn't obliterated much simpler statistical tests in that field. One of the rationales some have offered for including more diverse populations in GWAS is merely to reduce the false positives showing up for the currently available data. I try not to read that too cynically.

Also, I'm aware of various attempts to use more recent ML methods (deep learning, gradient boosted trees, xgboost) for imputation in this field, but none of them have broken into mainstream use despite very flattering initial papers. Largely because when they are applied to new independent data they don't perform better than the HMMs and often quite a bit worse.

When group B says they want to see much better performance, the actual threshold is arbitrary, but I think the sentiment is that they expect some loss of utility due to data differences and, unless the initial performance was strong, expect it is likely for the edge the model has to evaporate or even be harmful.

Edit: I read now the comments where you say it is a logistic regression. Usually people tend not to say "My machine learning model" when it's a logistic regression even though it can indeed be considered machine learning. You are certainly less likely to have some of the issues described above when using simpler models like that, but it can still happen. I do lean towards it being useful, but it can still be worth validating the common assumptions (e.g. stationality).

$\endgroup$
1
$\begingroup$

It depends.

A decision model can only be considered "useful" or not given a particular setting. You can see this in everyday life, where different decision processes result in a wide range of performance measures, from near-perfect to slightly-better-than-random. Take, for example, a few different decision processes, like 1) diagnosing a severe medical issue, 2) returning top search hits, and 3) selecting batches of parts for inspection. In each of these cases, one should conider the relative costs of false positives and negatives, and tune the decision process to optimize for the desired metric.

A medical screening test, for example, should have high recall, catching all true case of disease at the cost of some false diagnoses. A search engine, on the other hand, should have high precision, as it doesn't need to find every relevant webpage, but the ones it does return must be relevant. An industrial decision tool to direct parts inspectors may be useful even if it is only slightly better than random, as virtually any improvement makes the inspectors more efficient.

As you can see, there is not a numerical threshold for any performance measure that separates useful and non-useful models. It all depends on the context. A decision model with 30% precision may indeed be useful in some settings, but not others.

$\endgroup$
3
  • $\begingroup$ Perhaps this is a matter of customers wanting perfect performance and not liking any mistakes (a reasonable sentiment, even if not a reasonable expectation), but when I worked on EEG machine learning, the main complaint by clinicians was too high of a false positive rate. They did not like when the system cried wolf. I think this really speaks to your point, though, that everything is context-dependent. What you wrote about wanting high recall in medical diagnosis sounds reasonable, yet my project had doctors unhappy about the sensitivity! (It would wake up on-call staff too often. I get it.) $\endgroup$
    – Dave
    Commented Mar 9, 2023 at 21:40
  • $\begingroup$ @Dave On the one hand, you really do want to catch any potential problem. On the other hand, of course the doctors are frustrated if it cries wolf because it's a lot of stress on the doctor and on the patient. And the followup can be more invasive, e.g. biopsy if the system were doing cancer detection, maybe injecting radioactive dyes for better imaging after EEG, etc... Strictly speaking, missing something entirely is worse, but it takes all the decision making pressure off the doctors and patients, ignorance is bliss as they say. Continued... $\endgroup$
    – ttbek
    Commented Mar 14, 2023 at 8:47
  • $\begingroup$ Maybe the solution there is a more nuanced output? If the reading isn't super high confidence, rather than flagging as a case, maybe it can flag that the RR interval was weird because such and such and let the clinician figure out how bad he thinks that is. Again, they have the pressure of the decision, but if they decide it likely isn't a problem, then the lawsuit isn't going to say "the machine said they had arrhythmia and you did nothing." $\endgroup$
    – ttbek
    Commented Mar 14, 2023 at 8:52
0
$\begingroup$

This question requires answers to two sub questions. When is a model useful? What is the cost function to determine usefulness?

Group 1 says that the model is useful because it is doing better than nothing.

Group 2 says that the model is not useful because the cost is only reduced in a meaningful way when the performance is above some level (apparently 70-80% for your colleagues).

The two group's conclusion don't really contradict, they just look at it with different perspectives based on answers to the subquestions.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.