I want to predict using statistical/machine-learning methods the probability of a person to buy a product on a website knowing the characteristics of the product and the other products it is compared to on the same website.
Each "line" of my dataset is the characterics of a product, a summary of the characterics of the other proposed products (mean, minimum, maximum of prices and other characteristics) and what product has been chosen. The number of bought products is much lower than the number of non-bought ones: ratio between the two is several hundreds. As this dataset is unbalanced, I have implemented a subsampling method that
I have used the following algorithm (see Easy-Ensemble, basically averaging of under-sampling):
- Subset $N$ times the non-bought products (majority class)
- Define $N$ training datasets as each of the $N$ subsets combined with the whole set of bought products (minority class)
- Train $N$ models on these $N$ datasets
- The probability of a product to be bought is the mean of the probabilities given by the $N$ models.
The models are random forests.
Models have been trained on data with much more bought products than non-bought ones, therefore one could think that the obtained quantity is biased towards bought products.
- Can we consider that the probability obtained in 4 is an estimate of the real probability?
- If not, is there a way to "scale" the quantity obtained in 4 to get an estimate of the probability of the product to bought.
What is the effect of the biased sampling used for the modelling? How could I transform back the obtained model to counter-balance this effect?