Learning from one positive Say that in a binary classification problem you have several negatives and only one positive. 
What types of models are good to learn from this data, and predict the label for a new instance? Anything available in Python? From what I read here, scikit-learn doesn't support it.
 A: If you only have one example of your positive class, I'm afraid your situation is pretty dire--it's quite hard to reliably generalize from a single example. 
There are "one-class" classifiers that can learn a description of a single class and tell you whether a new example is likely to belong to it. These are sometimes called outlier/novelty/anomoly detectors. You could train one for the class where you have the most data and go from there. There's a decent thesis (Tax, 2001) that would be a good place to start, even if it is a bit old. This review might bring you more up to date, but I've just skimmed it.
However, I can imagine two problems:


*

*One-class classifiers are usually trained on the class of interest (e.g., "is fraudulent/cancerous/suspicious"), since those presumably have some features in common, whereas there are probably lots of ways for something to be not-fradulent (etc). Still might be worth a shot.

*If you really only have five or ten examples of the other class, you're still going to run into problems. Get more data! (even if it is expensive or a hassle...)

*Even if you could learn a model from so little data, how could you possibly evaluate its performance?


For what it's worth, a lot of the one-shot learning stuff does use fairly little data from each class. However, it often has either 1) a lot of classes or 2) a previously-acquired prior or "baseline" model, which helps tremendously. Is there any chance you could steal that idea?
A: You can train an exemplar classifier, when you have only single positive and a lot of negatives (which could be randomly sampled, i.e. when working on images). The idea is to learn from negatives, "what your positive is not", as you do not have enough positive data instances.


*

*If handcrafted features and shallow methods will be enough for your application, you can have a look at Malisiewicz et al.'s Exemplar SVM paper. Their source code is in Matlab. The idea is based on linear SVM classification, and you can easily do it using scikit's SVM package. However, you should write your hard negative mining code after training classifier. If you have chance to get a few more positive data instance (~ tens of), ensemble of exemplar classifiers will perform better than single exemplar classifier. In addition, the number of negatives should be at least a few thousands.

*Another approach is to learn an exemplar CNN. If you do not have experience in deep learning, it might be a bit more tricky. But, you can check following reference, exemplar CNN.
