I'm trying to make predictions based on customer service samples using randomForest and GBM, I'm leveraging on 2 stage modeling, i.e. 1st stage to predict whether customer will react or not, 2nd stage will predict how much they will spend. My 2nd stage model is training on those who spend>0 following is the prediction histogram vs actual test histogram (pink is actual, blue is prediction)

enter image description here

my question is why they deviates this far? I have checked my training samples, their histogram is same as test samples, any way that I can improve??

Many thanks!

  • $\begingroup$ I can't comment so I have to write it here: You have to furnish more details. What does your data set look like? Why have you decided to use tree-based methods (the answer shouldn't be something like xgboost is the most popular algorithm on kaggle)? $\endgroup$ – sntx Mar 29 '17 at 21:40
  • $\begingroup$ The two stage process seems correct given the picture you have here. What computations go into producing the pink histogram? Please edit details into your question. $\endgroup$ – Matthew Drury Mar 29 '17 at 22:06
  • $\begingroup$ the pink histogram is the plain histogram of the test data set dependent variable y, no transformations $\endgroup$ – user2926523 Mar 29 '17 at 22:36
  • $\begingroup$ the tree based methods are just my first pick, I haven't tried other models, the dependent variable has huge amount of 0s and a long tail as in the picture. $\endgroup$ – user2926523 Mar 29 '17 at 22:38
  • 1
    $\begingroup$ The most likely problem here is a bug in your code. I can't really give further advice unless you post the code. $\endgroup$ – AaronDefazio Mar 30 '17 at 2:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.