When does rules based classifier outperforms decision trees? I am trying to identify approximate 3% of the population for some characteristic feature. Standard decision tree or logistic regression gives too many false positives. Is there a chance that rules based classifier can improve performance? I would like to get approx 75% of recall with 95% of precission (i.e. FalsePositives <= 5% of Positives) 
 A: Probably you just have unbalanced classes (3% to 97%, if I understood well) -- try balancing them (get this 3% of true ones and about equal number of false ones) and check the classifier build on this case. If you are worried that you have thrown out most of your data, iterate it few times and connect them with some simple blender, like voting. (More complex blenders will also suffer from unbalanced classes). You should also check some better classifiers than a single tree or logistic regression -- like SVM or Random Forest.
Of course you can also use some classifier immune to imbalance problem, like kNN or as you say some rule-based approach.
A: This is only valid for the logit: you can use another link function (complementary log-log or cloglog in short). This is a variation of the classical logit function that allows for assymetry (when one tail of the link function does not go to 0 at the same speed as the other tail goes to 1). I had a very good experience fitting one of these to a database with about 1% of 'ones'.
this is a good starting point reference:
Link
These can be fitted using the zelig package in R.
edit: there is a good comparaison with a logit link here:
http://rss.acs.unt.edu/Rdoc/library/VGAM/html/cloglog.html
A: One strategy would be to use margin based methods with uneven margins (see this paper). Or you can use active learning to provide the learner with more balanced classes. Besides this, actually there are a number of other ways too that you can use to deal with imbalanced dataset. See this survey paper which discusses a number of techniques such as resampling, cost-sensitive approaches, active learning, etc (and evaluation methods).
