Difference between descriptive and predictive modelling

Question

I was wondering if anyone could help me clear up the difference between descriptive and predictive modelling. I am trying to build a model to predict where house prices will go up. To do this I need to examine historical data to identify predictors of house price increase, and how they relate to predict an increase.

To me this seems like it fits the description of descriptive modelling and predictive modelling. I am looking at historical data and trying to find the set of rules that summarise how we get from the variables to the current house price, so that I can use the same rules to predict from current conditions to future unknown house prices.

However, this also seems like predictive modelling because we are finding rules to predict an outcome.

So my question is, what is this: descriptive or predictive? And how can I determine whether something is descriptive or predictive modelling?

Answer 1

The best graphic I've seen summarizing this is from here:

The bigger focus is on whether or not the statistical model you're building is for statistical inference, or predictive accuracy. Breiman has a good paper explaining the differences, Statistical Modeling - A Tale of Two Cultures. In Breiman's paper, "data modeling" is equivalent to models built for statistical inference, whereas "algorithmic modeling" is closer to models built for predictive accuracy.

In the comment below, @boscovich mentions G. Shmueli's paper, To Explain or Predict? The paper looks to speak directly to your original question (Sections 1.2 and 1.3), and offers a trove of information beyond, for example: model evaluation and selection is also covered (Section 2.6), and two examples are provided (Section 3). This paper is probably a much more digestible starting point than Breiman's.

The biggest difference IMO is related to model fit and assessment. With models built for statistical inference, you are looking at in-sample fit (i.e. the entire sample population). With models built for predictive accuracy, you are looking at out-of-sample fit (i.e. the dataset which represents your sample population is split into a training and test set, and you judge the predictive accuracy by a measure of error - such as MSE, RMSE, MASE, etc. - on the test set). Rob Hyndman has a good paper covering measures of error, Another Look at Measures of Forecast Accuracy.

One thing that gets a bit confusing is that the terms in-sample fit and out-of-sample fit are often relative terms. Suppose your goal is a statistical model built for predictive accuracy. You randomly sample your dataset, splitting it as a 70/30 training-test set. Your model is built on the 70% training set, and scored (judged) by performance on the 30% test set. The training-test set approach is used to avoid overfitting or misspecification. Ultimately, you may want to deploy your model to the population, or even to a validation set. These can also be considered "out-of-sample". The validation set is data that are (read: should be) from the population, but that you never see during the model building process (either in training or test; for all you know it may not exist), but ultimately one where your model is deployed and scored on. A validation set approach is often used in Kaggle competitions. You build your model on the training set, score on the test set, and then it is scored again on the validation set.

Both model types are making "predictions" but it is the domain of which those predictions are judged that make a difference. For statistical inference, the fit is assessed by examining residuals, p-values at specific levels of alpha, etc. For predictive accuracy, you tend to be less concerned with those, and are more aligned with answering the question "Does this model make accurate predictions out-of-sample?"

Answer 2

You are doing predictive modeling. You know that you're doing predictive modeling when you don't care much (or at all) about how your model is arriving at its predictions as long as its predictions are as accurate as possible without overfitting. Easy examples of this include using any of the following models: gradient boosted trees, random forests, or neural networks.

These models are so complex that even though you could look at their weights and splits if you really wanted to, the amount of information would be too overwhelming to make sense of. It's straight forward enough to make sense of one decision tree but not thousands. The same can be said of neural networks, particularly deep learning for object detection in images. A deep neural network can have millions of weights.

Answer 3

1. Descriptive Analytics, which use data aggregation and data mining to provide insight into the past and answer: “What has happened?”

2. Predictive Analytics, which use statistical models and forecasts techniques to understand the future and answer: “What could happen?”

3. Prescriptive Analytics, which use optimization and simulation algorithms to advice on possible outcomes and answer: “What should we do?”