Background: I am working on the problem of classifying objects found in some biological images. Time and again, we encounter objects which do not fall into any of the categories/classes we are interested in and we would like to filter out these uninteresting objects before analyzing the interesting ones. I look at this as an outlier detection problem. I have prepared a training set with inliers (all interesting object categories pooled together) and outliers (objects we are not interested in). Visually, the outliers have a different (but very diverse/surely-multimodal) appearance characteristics when compared to the objects in the inlier class (which is also intuitively multimodal). So, I have computed some features that quantify an object's appearance in various aspects including texture.
Now, I am faced with the question of whether to solve this as a two-class classification problem or a one-class classification problem. My training set is quite imbalanced, with 3500 examples (~96%) from the inlier class and 150 examples (~4%) from the outlier class.
Question: I want to ask at which degree of class-imbalance between the positive and negative classes would you strongly prefer solving such problems as a one-class problem as opposed to a two-class classification problem?