Using standard machine learning tools on left-censored data

Question

I'm developing a forecasting application whose purpose is to allow an importer to forecast demand for its products from its customer network of distributors. Sales figures are a pretty good proxy for demand, so long as there is adequate inventory to fill the demand. When inventory gets drawn down to zero, though (the situation we're looking to help our customer avoid), we don't know much much we missed the target by. How many sales would the customer have made, had they had sufficient supply? Standard regression-based ML approaches that use Sales as a simple target variable will produce inconsistent estimates of the relationship between time, my descriptive variables, and demand.

Tobit modeling is the most obvious way to approach the problem: http://en.wikipedia.org/wiki/Tobit_model. I am wondering about ML adaptations of random forests, GBMS, SVMs, and neural networks that also account for a left-handed censored structure of the data.

In short, how do I apply machine learning tools to left-censored regression data to get consistent estimates of the relationships between my dependent and independent variables? First preference would be for solutions available in R, followed by Python.

Cheers,

Aaron

Answer 1

In short, how do I apply machine learning tools to left-censored regression data to get consistent estimates of the relationships between my dependent and independent variables?

If you can write up a likelihood and flip the sign to minus then you have your self a loss function which can be used for many machine learning models. In gradient boosting this is commonly refereed to as model boosting. See e.g., Boosting Algorithms: Regularization, Prediction and Model Fitting.

As an example with the Tobit model see Gradient Tree Boosted Tobit Models for Default Prediction paper. The method should be available with the scikit-learn branch mentioned in the paper.

The same idea is used for right censored data in e.g., the gbm and mboost packages in R for right censored data.

The above idea can be applied with other methods (e.g., neural network). However, it is particularly easy with Gradient boosting since you just need to be able to compute the gradient of the loss function (the negative log likelihood). Then you can apply whatever method you prefer to fit the negative gradient with an $L2$ loss.