Return to Answer

Frank Harrell has written about this on his blog: Classification vs. Prediction and Damage Caused by Classification Accuracy and Other Discontinuous Improper Accuracy Scoring Rules.

As Frank Harrell notes, accuracy is an improper scoring rule. (More precisely, accuracy is not even a scoring rule at all: see my answer to Is accuracy an improper scoring rule in a binary classification setting?) This can be seen, e.g., if we have no predictors at all and just a flip of an unfair coin with probabilities $(0.6,0.4)$. Accuracy is maximized if we classify everything as the first class and completely ignore the 40% probability that any outcome might be in the second class. (Here we see that accuracy is problematic even for balanced classes.) Proper scoring-rules will prefer a $(0.6,0.4)$ prediction to the $(1,0)$ one in expectation. In particular, accuracy is discontinuous in the threshold: moving the threshold a tiny little bit may make one (or multiple) predictions change classes and change the entire accuracy by a discrete amount. This makes little sense.

Frank Harrell has written about this on his blog: Classification vs. Prediction and Damage Caused by Classification Accuracy and Other Discontinuous Improper Accuracy Scoring Rules.

As Frank Harrell notes, accuracy is an improper scoring rule. (More precisely, accuracy is not even a scoring rule at all: see my answer to Is accuracy an improper scoring rule in a binary classification setting?) This can be seen, e.g., if we have no predictors at all and just a flip of an unfair coin with probabilities $(0.6,0.4)$. Accuracy is maximized if we classify everything as the first class and completely ignore the 40% probability that any outcome might be in the second class. (Here we see that accuracy is problematic even for balanced classes.) Proper scoring-rules will prefer a $(0.6,0.4)$ prediction to the $(1,0)$ one in expectation. In particular, accuracy is discontinuous in the threshold: moving the threshold a tiny little bit may make one (or multiple) predictions change classes and change the entire accuracy by a discrete amount. This makes little sense.

As Frank Harrell notes, accuracy is an improper scoring rule. This can be seen, e.g., if we have no predictors at all and just a flip of an unfair coin with probabilities $(0.6,0.4)$. Accuracy is maximized if we classify everything as the first class and completely ignore the 40% probability that any outcome might be in the second class. (Here we see that accuracy is problematic even for balanced classes.) Proper scoring-rules will prefer a $(0.6,0.4)$ prediction to the $(1,0)$ one in expectation. In particular, accuracy is discontinuous in the threshold: moving the threshold a tiny little bit may make one (or multiple) predictions change classes and change the entire accuracy by a discrete amount. This makes little sense.

As Frank Harrell notes, accuracy is an improper scoring rule. (More precisely, accuracy is not even a scoring rule at all: see my answer to Is accuracy an improper scoring rule in a binary classification setting?) This can be seen, e.g., if we have no predictors at all and just a flip of an unfair coin with probabilities $(0.6,0.4)$. Accuracy is maximized if we classify everything as the first class and completely ignore the 40% probability that any outcome might be in the second class. (Here we see that accuracy is problematic even for balanced classes.) Proper scoring-rules will prefer a $(0.6,0.4)$ prediction to the $(1,0)$ one in expectation. In particular, accuracy is discontinuous in the threshold: moving the threshold a tiny little bit may make one (or multiple) predictions change classes and change the entire accuracy by a discrete amount. This makes little sense.

EDIT. My answer to Example when using accuracy as an outcome measure will lead to a wrong conclusion gives a hopefully illustrative example where maximizing accuracy can lead to wrong decisions even for balanced classes.