weak classifier with weak features I have a set of weak features, and I am looking to create classifier based on them (it will only be weak given the noisy data). I am only trying to be right in ~60% of the case (that's enough for my problem).
Is there a literature on this kind of problem? I can only find articles where the output accuracy is very high.
Thank you
 A: Your question makes me ask the following:


*

*Why are you 'looking to create a weak classifier'? As in why are you deliberately looking for a 'weak classifier', when potentially you could train a 'strong classifier'?

*Why would you deliberately try to be right only about 60% of the time, when potentially a model could be trained to be right about say 66% of the time?
Your self-imposed '60%' accuracy limit suggests that you are aware of working in some noisy domain, (perhaps financial time series).
If that is the case, it seems that you are wary of over-fitting a classifier and seeing too-good-to-be-true accuracy on in-sample data. In which case, a better way is to simply to penalize model complexity when training your model and keep aside a pristine subset of data for out-of-sample testing.
In case, 60% accuracy is some lucky magic number, then stop training your classifier once it reaches 60% accuracy on the subset of data used for validation; which you can later check against the as yet pristine out-of-sample test data.
Having said all that, I think better to use regularization. In say boosting, not using interaction terms in the base learners and a low learning rate and a smaller number of iterations will inevitably to lower in-sample accuracy than when using greater interaction depth, fast learning rate and high number of iterations, holding other things constant.
