2
$\begingroup$

This question already has an answer here:

This is my first question here, so please bear with me.

I am working on a regression model to estimate the long term value of users of a certain website. For example, I would like to estimate the value of user $A$ after $180$ days, given their revenue data up to day $30$. To be more specific, denote by $x_{ij}$ the average daily amount of money spent by user $A_i$ up to day $j$. The problem can then be rephrased to predicting $y_i := x_{i, 180}$ from $(x_{i1}, \ldots, x_{i,30})$. After applying some inverse trigonometric transformation to the data to reduce its range, I have tried a few classifiers (linear regression, k-nearest neighbours, random forests) that perform reasonably well, though I feel that my procedure is not optimal.

As you might have guessed, the data is very skewed towards non-paying users. Actually, only 1%-1.5% of the users are paying users, which means that roughly $99$ out of $100$ rows in my dataset consist of all zeroes. How should I deal with this situation?

Option 1: Should I include the non-paying users when training the classifiers? In that case, even a sample of the data would have to be huge to actually include a reasonable amount of paying users (e.g., ~ $100000$ data points would contain ~ $1000$ paying users). More importantly, many models could be biased towards zero, given the disproportionate amount of zero observations. Last but not least, training the model becomes (perhaps unnecessarily) costly.

Option 2: Disregard non-paying users altogether and train the classifier only on paying users. However, I'm unsure whether the regressor would be artificially biased towards nonzero values. This is certainly the case for users having $x_{ij} = 0$ for $j=1, \ldots, 30$, as the prediction will be some nonzero $y'_i$, while the true value $y_i$ is almost surely $0$.

Option 3: Some combination of the above two. Something that comes to mind is to first address the binary nature of the data (paying vs. non-paying) by straight up estimating $y'_i = 0$ for users $A_i$ that are non-paying up to day $30$. On the other hand, if $x_{i,30} > 0$, we predict $y_i$ using a classifier trained only on users that have already paid at day $30$.

Any suggestions about what option above is most suitable? More generally, does anybody have some insight into how this type of problem is usually tackled? Thank you for your time.

$\endgroup$

marked as duplicate by mkt, mdewey, kjetil b halvorsen, Peter Flom regression Feb 2 at 11:01

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

0
$\begingroup$

If you want to approach the problem as a simple regression, you can simply balance your classes by downsamplinging negative examples or upsampling positives. That is a valid technique, though it may depend on how much positive data you have.

If instead you choose to pursue the classification-first method, consider class weights to penalize false negatives.

$\endgroup$
  • $\begingroup$ Thank you very much for your comments. I do have enough positives examples. However, it is unclear to me what ratio positive/negative should I strive for. Any comments on this? $\endgroup$ – marioga Nov 25 '15 at 19:02

Not the answer you're looking for? Browse other questions tagged or ask your own question.