I am trying to make a classifier that would predict whether an item is a "buy" (positive) or "not a buy" (negative). My dataset is ~65% positive examples, and ~35% negatives. It is more important for my model to keep false positives down (at the cost of reducing true positives). Let's say that the acceptable level of false positives is 5%. I am OK with my model missing a lot of "buy" opportunities, but when it recommends me to buy, I want to have a great degree of confidence (>95%) it actually is a buy.
I train my classifier, draw a ROC curve, and select the threshold such that false positives are less than 5% (which gives me true positive rate of 35%).
Now, am I correct in saying that a dummy classifier (i.e. one that ignores the features when making its prediction, but only looks at the distribution of positive and negative examples) will never be able to get true positive rate better than 5% if false positive rate is to be kept less than 5%? That is, its precision can never be better than 0.5, regardless of whether the dataset is balanced or unbalanced?