Skip to main content

how to measure a classifier's performance when close to 100% of the class labels belong to one class?

In my data, I have a class variable, denoted as $C$. This class variable values are ${0, 1}$ (binary). Almost all observations of $C$ are 0 (close to 100%, more precisely, 97%). I would like a "performance" test on different classification models (it could be accuracy). What I am afraid of happening is that if I have a classification model that always classifies any observation into class 0, then that model will be 97% accurate (even though it never considers any other variables).

Are there any well known performance tests for classification models on data dealing with very rare events?

Jane Wayne
  • 1.4k
  • 2
  • 17
  • 26