I am solving a problem of binary classification with up to 50 continuous and categorical predictors, where the class of interest is quite rare (1-5%). In addition, I would like to be very specific about the result, that is, I am more interesting of keeping the false positives really low, even if there are a lot of false negatives.
Are there in general any rules of thumb for method selection for tasks where high specificity is more important than low sensitivity?
Sounds like cost sensitive learning: you have some idea of what your cost ratio of false negatives/false positives is and you want to incorporate that into your model building. Most models can be tuned to the specific cost ratio fairly easily. The method of incorporating cost can depend on the model, but nothing you've mentioned suggests a specific method as far as I can tell.
Addition
By fairly easily I mean it is often incorporated in packages. So for logistic regression you can change the probability cut-off for classification, C50 and rpart have cost functions where you can define the value of FN/FP. But for many packages it isn't that easy. Depends on package.
You can undersample/oversample, but (1) that isn't data efficient - so utility depends on sample size (2) often you have to tune undersampling to get appropriate cost function.
For randomForests you can do "internal" undersampling that is more data efficient. In Max Kuhn's appliedpredictivemodelling package for his book he has code for "internal" undersampling as well as other cost sensitive methods (Chapter 16). I haven't tried the RUSBoost package on gitbub.
