Best way to combine binary and continuous response

Question

I am trying to come up with the best way to predict payment amount for a collections agency. The dependent variable is only non-zero when a payment has been made. Understandably, there are an overwhelming number of zeros because most people cannot be reached or cannot pay back the debt.

There is also a very strong negative correlation between the debt amount and the probability of making a payment. Typically, I would create a logistic model to predict the probability of pay / non-pay but this has the unfortunate consequence of finding the people with the lowest balances.

Is there a way to combine a logistic pay/non-pay model with a separate model predicting the payment amount?

Answer 1

The idea of gui11aume of building a two-stage model is the right way to go, however, one needs to consider the special difficulty of your setup which is the very strong negative correlation between the debt amount and the probability of making a payment

The primary issue of building a two-stage model here is, that the second model (for prediction of the debt), when built upon the "non-zeros" only, is built on a most likely non-random sample of the population (i.e. the whole dataset), but the combined model has to be applied on the whole population again. This means that the second model will have to make predictions for parts of the data which it has never seen before, resulting in a loss of accuracy. This is called Sample Selection Bias (for a overview from a ML perspective I recommend A Bayesian Network Framework for Reject Inference by Smith and Elkan).

The KDD-Cup-98 dealt with a similar issue where one should predict whether a donor for a veterans organization is likely to donate again and how much it is likely to donate. In this dataset, the probability of donating again was negatively correlated with the expected amount of money, too. The Sample Selection Bias also appeared.

The solution which impressed me the most can be found in Learning and Making Decisions When Costs and Probabilities are Both Unknown by Bianca Zadrozny and Charles Elkan. They have created a cost sensitive solution based upon the Heckman correction, which is to my knowledge the first systematic approach to correct the (sample) selection bias.

Answer 2

That's a very nice question (+1).

Why not treat the 0s as if they were NAs?

You could add a dummy response indicating whether any money has been recovered (i.e.equal to 0 when the value is 0, and 1 when the value is positive) and fit a logistic model on this binary response with the same predictors. You would fit 2 models: the binary response using all the data points, and the continuous response using only the non zeron data points (in line with the idea of treating 0 as NA).

You can still test the nullity of parameters in each model and compute expected gain by using both sets of paramters.