First of all, I would like to describe some common layouts that Data Mining books use explaining how to deal with Unbalanced Datasets. Usually the main section is named Unbalanced Datasets and they cover these two subsections: Cost-Sensitive Classification and Sampling Techniques.
It seems that facing a problem with a rare class you can perform both cost-sensitive classification and sampling. Instead, I think that one should apply cost-sensitive techniques if the rare class is also the target of the classification and a misclassification of a record of that class is costly.
On the other hand, sampling techniques, such as over-sampling and under-sampling, are useful if the target of the classification is a good accuracy overall, without focusing on a particular class.
This belief comes from the rationale of MetaCost that is a general way to make a classifier cost-sensitive: if one want to make a classifier cost-sensitive in order to penalise a misclassification error of the rare class he should over-sample the other class. Roughly speaking, the classifier tries to adapt to the other class and it becomes specific to the rare class.
This is the opposite of over-sampling the rare class, that is the usually suggested way to deal with this problem. Over-sampling of the rare class or the under-sampling of the other class is useful to improve the overall accuracy.
Please, it would be great if you confirmed my thoughts.
Stated this, the common question facing an unbalanced dataset is:
Should I try to get a dataset that counts as many rare records as other ones?
My answer would be, in case you are looking for accuracy: OK. You can perform it either finding out more rare class examples or deleting some records of the other class.
In case you are focusing on the rare class, with a cost-sensitive technique, I would answer: you can only find out more rare class example but you shouldn't delete records of the other class. In the latter case you will not be able to let the classifier adapt to the other class, and the rare class misclassification error could increase.
What would you answer?