Best way to "parse" survey data for predictive value?

Question

Straightforward question. Kind of like bucketing too.

Say you have a customer survey. The customer rates 1-10, or 1-5.

Say you want to use this to predict other behaviors. Reorder rate, refund rate, whatever. I know data science is a massive field but --

Is there a specific art, or methodology, whereas --- you may discover --- encoding the input variable differently might offer greater predictive value?

For instance instead of looking at 1-10 .... bucketing 1-5 and 6-10 turns out to have more predictive power. Or some other formula. Does that make sense? And is there a way to discover some useful encoding besides trial and error?

Is it simply a -- re-run the predictive model with hold-out sets and see what generally performs better?

Answer 1

Little background about me, I've spent the first 2 years working on survey data in big retail company for performance evaluation, customer satisfaction improvement and staffing optimization.

It depends on what you want from the predictive model in terms of accuracy and interpretability. and of course, the quality and quantity of your dataset.

If you have huge amount of records which are clean (beware of fraud survey records. Detecting and handling those records is another huge topic), and you don't care about interpretation and just want to max out the accuracy of your prediction, then you try models like xgboost, random forests, neural networks, .... set up training/test/validation datasets and pick the one that gives you the best test/validation accuracy.

More often, you don't have much data when it comes to survey data. Then, you might do a bit more work on data exploration like check distribution, check correlation, check trending, check demographics, feature engineering (bucketing that you mentioned is one of the methods). In this step, we are actually weigh in your domain knowledge and can also get some business insights at this step. If your engineered features are strong, the final model can just be a simply regression or decision tree. This path is a bit painful than the previous one, but it has less limits on your dataset, and often higher interpretability.