# Resampling to get equal predictive power per observation

Cross posted from data science due to lack of response

This is probably a thing I am just not searching for correctly, but essentially my idea is this: given some machine learning classification $$C$$ based on an input dataset $$D$$, certain observations in $$D$$ are more likely to be misclassified than others because they are "less common". So what we would want to do is oversample observations like that, until $$C$$ is trained such that all observations have the probability of being classified correctly.

Is there a resampling method that does such a thing? My idea would be to

1. train classifier $$C$$ on base dataset $$D$$
2. take all observations misclassified by $$C$$ in $$D$$ and insert into new dataset $$D'$$
3. retrain $$C$$ on $$D = D + D'$$ (or add a new batch with $$D'$$ in SGD etc)
4. iterate that way until some convergence happens

Has someone formalized something along these lines? Intuitively, we want to overweight under-represented types of data.

Update: An additional thought I had that something along these lines might be useful in Q-learning - that instead of just randomly exploring off-policy, explore in areas where the model is less confident.