Is a predictive model the best way to generalize about my current data?

Question

Suppose I have a customer dataset. My independent variables are various types of customer attributes (age, where they live, gender, price of the item they are potentially buying) and my dependent variable is "not buy" vs "buy" (0 or 1).

What I'm trying to understand is, how much each customer attribute affects the probability of buying. For example, I want to be able to say "customer 25 years old is x% more likely to buy then....".

My first thought of this is to some sort of classification model (logistic regression, trees etc). In my mind, if I can fit this model as best as possible on both the training and test set, then the coefficients of my model will tell me how much impact each customer attribute has on the probability of buying.

Is this the best way to think about this? Is using a predictive model for explanatory purposes accurate?

My end goal is not to predict a future customer's action, but just to understand my current customer's attribute. To me, they sound the same, but I want to make sure that they are one and the same.

Are there other statistical techniques I can use to do what I want to do?

Answer 1

Strictly speaking, a predictive model does not necessarily confer any ability to generate understanding of the relationship between the predictors and target variable. In other words, the model can be a black box. It takes some input and produces some output, but there need not be any clear way for you to see how input maps to output. Complex machine learning algorithms such as neural networks function like this, being much harder to interpret than coefficients from a regression model.

But recent years have seen a large increase in interest in making such machine learning models interpretable. For most (probably all) machine learning models, it is possible to quantify the relative importance of different predictors to understand which ones contribute to predictive power, and also examine partial-plots to understand how individual predictors relate to the target variable.

So nowadays, you can use most (probably all) predictive models to learn from your data. Whether it is the best approach depends on your goals, your data, and the kinds of claims you want to make. Since you seem less interested in making actual predictions, you could also consider using models designed for inference, such as regression. Incidentally, logistic regression is one such type of model - it is not a classification algorithm, though it can be used for classification.

Answer 2

This question presents a very important problem faced by a great many analysts.

I take it you want to use a multivariate model to control for a set of predictors, in order to isolate the role each one plays in affecting the purchase decision. In this way you would identify the amount of leverage each predictor gives you over the outcome.

Before you examine the various types of models you might use -- logistic regression, random forest, neural network, etc. -- think carefully, and perhaps ask colleagues to work with you, to consider how the causal pathways might work. Draw a diagram if it helps. Then think about what will happen if you use a model that employs statistical control.

Suppose your customers from Region X disproportionately tend to consider Item Z in their purchase decision. Then, if you control for Region, you might be misleadingly deflating the coefficient describing the role of Item. Or vice versa.

Similarly, suppose men below a certain age disproportionately tend to buy your products. If you control for age, you might misleadingly deflate the coefficient for gender. Or vice versa.

No statistical model or software can take care of such dynamics for you. And there may not be a clear, correct answer to any question that comes up; you may need to try multiple avenues of analysis to yield the best information.

Kudos for recognizing (I believe) that good explanatory analysis requires more than simply using an applicable statistical model. Also, kudos for planning to cross-validate, though I'll caution you that a single iteration of analysis via training set and test set is not nearly enough to provide reliable results in most circumstances.

Answer 3

I agree with the previous answer given by mkt. I can add a couple of 2 cents from my basket here.

All depends from why you need to know what you want to know; i.e. what you or your client is going to do with this information (assuming this is not an academic assignment!).

From the setting of your problem, I understand that what you are trying to solve is a typical problem people in market research have been addressing since customer records where kept and a relatively strong computing power has been available – probably since the 70ies of last century.

Before trying the hard way, always try simplest ways. Analysis of the frequencies and bivariate correlations are going to tell you a lot about the data you have, and suggesting the way to approach your problem from a multivariate perspective.

You need to understand if what you are after is an R squared, for example, or an odd ratio. This changes the modelling approach from linear to logistic. You might want to summarise your learnings in a single importance score for each independent variable, or measure the impact that the independent variable has on the dependent.

Generally speaking, binary outcome variables like buy/ not buy are very difficult to predict with accuracy, unless of course you use among the predictors something that is extremely similar to the outcome (for example: time spent on the website). If you go on the multivariate path, you need to carefully assess the predictors you use in the model, to make sure your results are robust and interpretable.

Hope this is not too confusing and it helps! (and this is my first answer here on StackExchange!)